# How to Initialize BigQuery Dataset Metadata Snapshots Using kcmd init with Multiple Entry Types

> Initialize BigQuery dataset metadata snapshots with kcmd init using multiple entry types. Learn how to create a catalog.yaml manifest for efficient knowledge cataloging.

- Repository: [Google Cloud Platform/knowledge-catalog](https://github.com/GoogleCloudPlatform/knowledge-catalog)
- Tags: how-to-guide
- Published: 2026-07-14

---

**You can initialize snapshots for multiple BigQuery datasets by passing the `--bigquery-dataset` flag multiple times or as a comma-separated list, which creates a [`catalog.yaml`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/catalog.yaml) manifest that records the scope for the Knowledge Catalog snapshot.**

The `kcmd` CLI in the GoogleCloudPlatform/knowledge-catalog repository implements a *Metadata as Code* workflow that mirrors remote metadata sources into version-controlled local files. When working with BigQuery, the `init` sub-command bootstraps a local snapshot directory containing YAML for structured metadata and Markdown sidecars for unstructured content, enabling you to capture one or many datasets in a single operation.

## Understanding the kcmd init Command

The `kcmd init` command serves as the entry point for creating **CatalogManifest** files that define your snapshot scope. Located in [`toolbox/mdcode/src/tool/commands.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/toolbox/mdcode/src/tool/commands.ts), this command accepts mutually exclusive flags for different source types: `--entry-group` for Dataplex EntryGroups, `--kb` for Knowledge Bases, or `--bigquery-dataset` for BigQuery datasets.

When you specify BigQuery datasets, the CLI invokes `CatalogManifest.initWithBigQuery()` from [`toolbox/mdcode/src/libts/manifest.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/toolbox/mdcode/src/libts/manifest.ts). This method constructs a manifest that records dataset identifiers and sync configuration, then serializes it to [`catalog.yaml`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/catalog.yaml) in your current working directory.

## Handling Multiple Entry Types

For **multiple entry types** of the same source kind—specifically multiple BigQuery datasets—the CLI accepts dataset identifiers in two formats. Both approaches result in a comma-joined string passed to the manifest initialization logic.

### Using Comma-Separated Values

Pass multiple datasets as a single comma-delimited string:

```bash
kcmd init --bigquery-dataset my-project.ds1,my-project.ds2,my-project.ds3

```

### Using Repeated Flags

Alternatively, repeat the flag for each dataset:

```bash
kcmd init \
  --bigquery-dataset my-project.ds1 \
  --bigquery-dataset my-project.ds2 \
  --bigquery-dataset my-project.ds3

```

Both methods trigger the same underlying logic in [`commands.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/commands.ts) that handles arrays:

```typescript
const datasets = Array.isArray(options.bigqueryDataset)
  ? options.bigqueryDataset.join(',')
  : options.bigqueryDataset;
manifest = await kcmd.CatalogManifest.initWithBigQuery(datasets, ctx);

```

## The Snapshot Layout and Immediate Sync

After creating the manifest, `kcmd init` writes the **Standard Layout** to disk. This layout organizes metadata into YAML files for structured aspects (schemas, labels) and sidecar Markdown files for unstructured content like overviews.

To populate these files immediately rather than just creating the manifest, append the `--pull` flag:

```bash
kcmd init --bigquery-dataset my-project.ds1,my-project.ds2 --pull

```

This combination creates the [`catalog.yaml`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/catalog.yaml) manifest and immediately executes the `pull` sub-command, downloading current metadata from BigQuery into the snapshot directory.

## Complete Implementation Details

The initialization workflow in [`toolbox/mdcode/src/tool/commands.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/toolbox/mdcode/src/tool/commands.ts) (lines 25-51) performs these specific steps:

1. **Source Type Detection** – Validates that exactly one source flag is provided among `--entry-group`, `--kb`, or `--bigquery-dataset`.
2. **Manifest Construction** – Calls the appropriate `CatalogManifest` factory method based on the source type.
3. **Persistence** – Saves the manifest as [`catalog.yaml`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/catalog.yaml) in the current directory.
4. **Optional Pull** – If `--pull` is specified, chains to the `pull()` function to hydrate the snapshot.

The `CatalogManifest` class persists sync configuration and dataset scope, enabling subsequent `kcmd pull` and `kcmd push` operations to synchronize changes between your local files and the Knowledge Catalog service.

## Summary

- **Multiple datasets** are supported via comma-separated strings or repeated `--bigquery-dataset` flags when initializing BigQuery snapshots.
- The **source of truth** for initialization logic resides in [`toolbox/mdcode/src/tool/commands.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/toolbox/mdcode/src/tool/commands.ts), while manifest definitions live in [`toolbox/mdcode/src/libts/manifest.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/toolbox/mdcode/src/libts/manifest.ts).
- **CatalogManifest** factory methods like `initWithBigQuery()` convert CLI arguments into persistent configuration stored in [`catalog.yaml`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/catalog.yaml).
- The **Standard Layout** produces YAML/Markdown files that enable version-controlled metadata editing.
- Use the **--pull** flag to immediately populate the snapshot after manifest creation.

## Frequently Asked Questions

### Can I initialize snapshots for multiple BigQuery datasets in one command?

Yes. You can pass multiple dataset identifiers either as a comma-separated list (`--bigquery-dataset project.ds1,project.ds2`) or by repeating the flag multiple times. Both approaches generate a single [`catalog.yaml`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/catalog.yaml) that encompasses all specified datasets.

### Where is the snapshot configuration stored after running kcmd init?

The command creates a [`catalog.yaml`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/catalog.yaml) file in your current working directory. This file contains the **CatalogManifest** that records the source type, dataset identifiers, and sync configuration required for subsequent `pull` and `push` operations.

### Can I mix BigQuery datasets with Dataplex EntryGroups in a single init command?

No. The CLI enforces mutually exclusive source types through conditional logic in [`commands.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/commands.ts). You must choose exactly one source type per initialization: either `--bigquery-dataset` for BigQuery, `--entry-group` for Dataplex, or `--kb` for Knowledge Base.

### Does kcmd init immediately download the metadata files?

By default, `kcmd init` only creates the manifest. To immediately populate the snapshot directory with actual metadata files, include the `--pull` flag. This triggers the pull workflow immediately after manifest creation, downloading YAML and Markdown representations of your datasets.