How to Initialize a New kcmd Catalog Snapshot: A Complete Guide

Use kcmd init with the --bigquery-dataset, --entry-group, or --kb flag to create a local snapshot directory containing a manifest.json file and the standard catalog structure, optionally pulling data immediately with the --pull flag.

The kcmd CLI is the command-line interface bundled with the GoogleCloudPlatform/knowledge-catalog toolbox. When you initialize a new kcmd catalog snapshot, you create a local representation of your cloud metadata—BigQuery datasets, Dataplex EntryGroups, or Knowledge Bases—that can be version-controlled, inspected, and synchronized back to Google Cloud using kcmd push.

Supported Catalog Sources

The init command supports three mutually exclusive source types defined in toolbox/mdcode/src/tool/commands.ts. You must specify exactly one source type per initialization:

• --bigquery-dataset – Accepts one or more BigQuery datasets in project.dataset format
• --entry-group – Accepts a single Dataplex EntryGroup identifier
• --kb – Accepts a single Knowledge Base identifier

The Initialization Process

Internally, the kcmd init command executes a four-step workflow as implemented in toolbox/mdcode/src/tool/commands.ts and toolbox/mdcode/src/libts/manifest.ts:

1. Parse CLI options – Validates that only one source type is selected and processes the --output directory path (defaults to catalog-snapshot).

2. Create a CatalogManifest – Instantiates the appropriate factory method from toolbox/mdcode/src/libts/manifest.ts:

   • CatalogManifest.initWithBigQuery(datasets, ctx) for BigQuery sources
   • CatalogManifest.initWithEntryGroup(name, ctx) for Dataplex EntryGroups
   • CatalogManifest.initWithKnowledgeBase(name, ctx) for Knowledge Bases

3. Initialize the snapshot layout – The manifest’s _layout object (implemented in toolbox/mdcode/src/libts/layouts/standard.ts) creates the directory structure and writes the manifest.json file containing source identifiers and API context.

4. Optional pull – If the --pull flag is present, the command immediately executes kcmd pull to populate the snapshot with current catalog entries.

Command Examples

Initialize a snapshot from different source types using these patterns:

# Initialize from a single BigQuery dataset

kcmd init --bigquery-dataset my-project.my_dataset

# Initialize from multiple BigQuery datasets (repeat the flag)

kcmd init \
  --bigquery-dataset my-project.dataset_one \
  --bigquery-dataset my-project.dataset_two

# Initialize from a Dataplex EntryGroup

kcmd init --entry-group my-project.us-central1.my_entry_group

# Initialize from a Knowledge Base

kcmd init --kb my-project.my_knowledge_base

# Initialize and immediately pull catalog entries

kcmd init --bigquery-dataset my-project.my_dataset --pull

# Specify a custom output directory

kcmd init --bigquery-dataset my-project.my_dataset --output ./my-snapshot

Understanding the Generated Snapshot Structure

After initialization, the target directory (default: catalog-snapshot) contains the following structure defined in toolbox/mdcode/src/libts/layouts/standard.ts and toolbox/mdcode/src/libts/layouts/documents.ts:

• manifest.json – Records source identifiers, API context, and metadata required for subsequent pull and push operations.
• catalog/ – Directory containing the actual metadata files and entries.
• index/ – Helper indexes for quick lookups and local operations.

The CLI entry point in toolbox/mdcode/src/tool/main.ts registers the init command, while the implementation details are documented in toolbox/mdcode/docs/design.md and toolbox/mdcode/docs/spec.md.

Summary

• kcmd init creates a local snapshot of BigQuery datasets, Dataplex EntryGroups, or Knowledge Bases.
• Use factory methods (initWithBigQuery, initWithEntryGroup, initWithKnowledgeBase) in toolbox/mdcode/src/libts/manifest.ts to handle different source types.
• The command generates a standard directory layout with manifest.json, catalog/, and index/ directories.
• Add the --pull flag to immediately populate the snapshot with current cloud data.
• Source options are mutually exclusive—you cannot mix --bigquery-dataset with --entry-group in a single command.

Frequently Asked Questions

Can I initialize a snapshot from multiple BigQuery datasets at once?

Yes. According to the source code in toolbox/mdcode/src/tool/commands.ts, the --bigquery-dataset flag accepts multiple values. Repeat the flag for each dataset you want to include in the snapshot. However, you cannot combine BigQuery datasets with Dataplex EntryGroups or Knowledge Bases in the same initialization command.

What is the difference between kcmd init and kcmd pull?

kcmd init creates the local directory structure and writes the manifest.json file that records the connection to your cloud source. It does not download metadata by default. kcmd pull populates the catalog/ directory with the actual entries from the cloud source. When you use the --pull flag with init, the command executes both steps sequentially.

Where is the manifest file stored and what does it contain?

The manifest.json file is stored in the root of your snapshot directory (default: catalog-snapshot/). As implemented in toolbox/mdcode/src/libts/manifest.ts, it contains the source identifiers (BigQuery dataset IDs, EntryGroup names, or Knowledge Base names), the API context needed for authentication, and metadata required to synchronize changes back to Google Cloud via kcmd push.

Can I specify a custom output directory for the snapshot?

Yes. Use the --output or -o flag followed by your desired path. If you do not specify an output directory, kcmd init defaults to creating a directory named catalog-snapshot in the current working directory. The target directory will contain the standard layout defined in toolbox/mdcode/src/libts/layouts/standard.ts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}