# How kcmd Manages Metadata Using Source Code Artifacts: A Complete Guide to the Knowledge Catalog CLI

> Learn how kcmd manages metadata using source code artifacts. This guide explores its approach to catalog entries, aspects, and manifests stored as TypeScript objects synchronized with Google Cloud Data Catalog.

- Repository: [Google Cloud Platform/knowledge-catalog](https://github.com/GoogleCloudPlatform/knowledge-catalog)
- Tags: how-to-guide
- Published: 2026-07-15

---

**`kcmd` treats catalog metadata as first-class source-code artifacts by modeling entries, aspects, and manifests as plain TypeScript objects that are persisted to the filesystem using well-defined layouts, then synchronized with Google Cloud Data Catalog.**

The `kcmd` CLI from the GoogleCloudPlatform/knowledge-catalog repository enables teams to manage metadata using source code artifacts through a declarative, version-controlled workflow. Unlike traditional database-centric approaches, this tool materializes BigQuery datasets, Dataplex entry groups, and Knowledge Bases as ordinary YAML and Markdown files. This architecture allows developers to apply standard IDE tooling, linting, and Git workflows to metadata management while maintaining bidirectional synchronization with Google Cloud Data Catalog.

## Understanding the kcmd Architecture

The `kcmd` architecture consists of a small set of TypeScript libraries that model catalog entities as code representations, orchestrated by a CLI that dispatches to these core components.

### CatalogManifest: Declaring the Source

The **`CatalogManifest`** class describes the source of a catalog—whether a BigQuery dataset, Dataplex entry group, or Knowledge Base—and optionally limits which entries and aspects are captured. According to the source code in [`src/libts/manifest.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/libts/manifest.ts), this manifest is persisted as a YAML file named [`catalog.yaml`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/catalog.yaml) that serves as the source-code artifact declaring what metadata will be materialized locally.

### Entry and Aspect Models

At the heart of the system are the **`Entry`** and **`Aspect`** interfaces defined in [`src/libts/metadata.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/libts/metadata.ts). These simple TypeScript interfaces model the shape of catalog entries and their attached aspects, representing the code-level abstraction of the metadata. When loaded into memory, these plain objects enable deterministic checksums and validation before persistence.

### CatalogSnapshot: The Persistence Layer

The **`CatalogSnapshot`** class in [`src/libts/snapshot.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/libts/snapshot.ts) provides the high-level API for reading, writing, listing, and validating entries on disk. Rather than handling file formats directly, it delegates to a *layout* strategy—such as YAML side-cars or Markdown front-matter—allowing the same metadata to be stored in different textual formats depending on team preferences.

### CatalogSync: The Synchronization Engine

The **`CatalogSync`** class in [`src/libts/sync.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/libts/sync.ts) implements the pull and push operations that compare local snapshots with the remote Google Cloud Data Catalog service. This component handles checksum computation, dry-run capabilities, and conflict resolution, ensuring that only explicitly declared artifacts (respecting the manifest's `snapshot` and `publishing` whitelists) are propagated to the cloud service.

## The Metadata-as-Code Workflow

The `kcmd` tool implements a five-stage workflow that treats metadata as version-controlled source code.

### 1. Manifest Creation with `kcmd init`

When initializing a new catalog, the `CatalogManifest.initWithBigQuery` method constructs a `CatalogSource` object based on user-provided parameters like `--bigquery-dataset PROJECT.DATASET`. This method, implemented in [`src/libts/manifest.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/libts/manifest.ts), saves the configuration to [`catalog.yaml`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/catalog.yaml), creating the source-code artifact that declares the metadata scope.

### 2. Snapshot Generation with `kcmd pull`

The `CatalogSnapshot.fromPath` method reads the manifest from the repository root, resolves the source connection, and walks the chosen layout strategy to produce in-memory `Entry` objects. The snapshot maintains a map of entries and aspects, enabling deterministic checksums for change detection.

### 3. Local Modification

Users edit the generated files directly—modifying [`my_table.yaml`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/my_table.yaml) or [`my_table.md`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/my_table.md) using standard editors. Because these are ordinary source files, teams can apply linting, code review, and version control practices that are impossible with API-only metadata management.

### 4. Synchronization with `kcmd push`

The `CatalogSync` class compares the local snapshot with the remote Data Catalog, detecting changes via checksums and performing create, update, or delete operations via the `gcp.CatalogClient`. The sync respects the manifest's whitelists, ensuring only declared artifacts are propagated.

### 5. MCP Server Exposure

When invoked as `kcmd mcp`, the binary runs a lightweight HTTP server defined in [`src/tool/mcp.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/tool/mcp.ts) that exposes sync primitives as tools for external agents. This allows automated systems to treat the catalog as "metadata-as-code" and invoke the same operations programmatically.

## CLI Implementation and MCP Server

The command-line interface wires user-facing commands to the library classes through a structured entry point and command implementations.

### Command Routing in main.ts

The CLI entry point in [`src/tool/main.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/tool/main.ts) uses the `cac` argument parser to dispatch subcommands (`init`, `pull`, `push`, `status`, `mcp`) to their respective handlers. This binary (`dist/kcmd`) serves as the unified interface for both human operators and automated systems.

### Command Logic in commands.ts

The concrete CLI logic resides in [`src/tool/commands.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/tool/commands.ts), where implementations create a `CatalogManifest`, load a `CatalogSnapshot`, and invoke `CatalogSync` methods. This separation ensures that the CLI layer remains thin, delegating all business logic to the reusable library classes.

### MCP Server for External Agents

The MCP server implementation in [`src/tool/mcp.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/tool/mcp.ts) exposes the same `CatalogSnapshot` and `CatalogSync` logic via HTTP RPC endpoints such as `list-entries`, `lookup-entry`, `pull`, and `push`. This architecture allows external agents to interact with metadata artifacts without shelling out to the CLI.

## Working with Source Code Artifacts Programmatically

The following examples demonstrate how to interact with `kcmd`'s source-code artifact system using the TypeScript API.

Create a manifest for a BigQuery dataset:

```typescript
import * as kcmd from 'kcmd';
import { gcp } from 'kcmd';

const ctx = gcp.ApiContext.default();
const manifest = await kcmd.CatalogManifest.initWithBigQuery(
  'my-project.my_dataset', ctx);
manifest.save('catalog.yaml');

```

Load a local snapshot from the filesystem:

```typescript
const snapshot = await kcmd.CatalogSnapshot.fromPath(
  '.',  // repo root containing catalog.yaml + entry files
  ctx);
console.log(snapshot.entries.map(e => e.name));

```

Push local changes back to Cloud Data Catalog:

```typescript
import { CatalogSync } from 'kcmd/libts/sync';

const catalog = new kcmd.gcp.CatalogClient(ctx);
const sync = new kcmd.CatalogSync(catalog, snapshot);
await sync.push();  // Handles checksums, dry-run, etc.

```

Run the MCP server for external tool integration:

```typescript
import * as http from 'node:http';
import { startMcp } from 'kcmd/tool/mcp';

http.createServer(startMcp({ ctx })).listen(8080);

```

## Summary

- **`kcmd`** manages metadata through source-code artifacts by treating BigQuery datasets and other catalog sources as declarative YAML configurations and entry files.
- The **`CatalogManifest`** in [`src/libts/manifest.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/libts/manifest.ts) defines the source and scope, persisted as [`catalog.yaml`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/catalog.yaml).
- **`CatalogSnapshot`** from [`src/libts/snapshot.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/libts/snapshot.ts) materializes metadata into the filesystem using pluggable layouts (YAML side-cars or Markdown front-matter).
- **`CatalogSync`** in [`src/libts/sync.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/libts/sync.ts) handles bidirectional synchronization with checksum validation and whitelist filtering.
- The CLI in [`src/tool/main.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/tool/main.ts) and command implementations in [`src/tool/commands.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/tool/commands.ts) provide the user interface, while [`src/tool/mcp.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/tool/mcp.ts) exposes these capabilities as HTTP tools for external agents.

## Frequently Asked Questions

### How does kcmd store metadata on the local filesystem?

`kcmd` stores metadata using layout strategies delegated by `CatalogSnapshot`. Depending on configuration, entries are saved as YAML side-car files or Markdown documents with front-matter, allowing the metadata to exist as ordinary source files alongside application code.

### What is the purpose of the catalog.yaml file?

The [`catalog.yaml`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/catalog.yaml) file is the `CatalogManifest` that declares the source of the catalog (such as a BigQuery dataset or Dataplex entry group) and defines which entries and aspects should be captured. It serves as the source-code artifact that configures the synchronization scope.

### How does kcmd detect changes between local files and the remote Data Catalog?

`CatalogSync` computes checksums of the local `Entry` and `Aspect` objects managed by `CatalogSnapshot`, then compares these against the remote state via the `gcp.CatalogClient`. This allows precise detection of create, update, and delete operations before applying them.

### Can external tools or agents interact with kcmd programmatically?

Yes. When started with `kcmd mcp`, the tool runs an HTTP server defined in [`src/tool/mcp.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/tool/mcp.ts) that exposes the snapshot and sync operations as RPC endpoints. External agents can invoke `list-entries`, `lookup-entry`, `pull`, and `push` operations directly without using the CLI interface.