Directory Structure for kcmd Metadata: A Complete Guide to GoogleCloudPlatform/knowledge-catalog

The kcmd metadata directory structure resides in toolbox/mdcode/src/libts/ and organizes TypeScript interfaces, source adapters, GCP utilities, and layout engines into a modular hierarchy that supports BigQuery, Dataplex, and Knowledge Base integrations.

The GoogleCloudPlatform/knowledge-catalog repository provides a TypeScript-based metadata management toolkit for data catalog operations. Understanding the directory structure for kcmd metadata is essential for developers extending the catalog or integrating custom data sources, as each subdirectory serves a specific architectural purpose within the metadata pipeline.

Core Metadata Interfaces

The foundation of the kcmd metadata system lies in type definitions that describe catalog objects.

Entry and Aspect Definitions

In toolbox/mdcode/src/libts/metadata.ts, the library defines the core TypeScript interfaces that describe catalog objects. This file exports the Entry and Aspect types, which represent the fundamental units of metadata storage. The Entry interface captures resource identifiers, types, and relationships, while Aspect handles specific metadata attributes attached to each entry.

Catalog State Management

The directory structure separates concerns between loading existing catalog states and defining new catalog groupings.

Snapshot Loading

The toolbox/mdcode/src/libts/snapshot.ts file contains logic for reading a local catalog snapshot from disk. It implements the CatalogSnapshot class, which provides methods to parse and validate stored metadata representations. This module handles the deserialization of catalog entries from the file system into runnable TypeScript objects.

Manifest Organization

Catalog manifests—groupings of related entries—are defined in toolbox/mdcode/src/libts/manifest.ts. The CatalogManifest class manages collections of entries as coherent units, enabling operations that treat multiple catalog items as a single deployable artifact. This file orchestrates how metadata collections are serialized and persisted to specific paths.

Metadata Source Architecture

The structure distinguishes between abstract source definitions and concrete implementations for different data platforms.

Abstract Base Class

The toolbox/mdcode/src/libts/source.ts file defines the abstract base class for all metadata sources. This class establishes the contract that specific implementations must follow, whether pulling metadata from BigQuery, Dataplex, or Knowledge Base systems. It standardizes initialization parameters, authentication flows, and data transformation methods.

Concrete Implementations

The toolbox/mdcode/src/libts/sources/ directory contains specific adapters for different catalog backends:

• bq-dataset.ts – Handles BigQuery dataset metadata extraction
• entrygroup.ts – Manages Dataplex entry group synchronization
• kb.ts – Processes Knowledge Base entries

Each file extends the abstract base class to implement platform-specific API interactions and data normalization logic.

Google Cloud Platform Integration

Google Cloud-specific helpers are isolated in a dedicated utilities directory.

API Clients and Utilities

The toolbox/mdcode/src/libts/gcp/ directory contains low-level infrastructure code:

• api.ts – Provides the low-level API client and request handling
• bigquery.ts – Contains BigQuery-specific utilities and data type mappers
• dataplex.ts – Implements Dataplex service integrations
• crm.ts – Cloud Resource Manager helpers for project and organization resolution
• context.ts – Manages ApiContext instances for authentication and request configuration

These modules handle GCP authentication, rate limiting, and service-specific error handling, isolating cloud dependencies from the core metadata logic.

Layout and Storage Strategies

The directory structure includes sophisticated layout engines that map raw snapshots into file-system-compatible trees.

Layout Engines

The toolbox/mdcode/src/libts/layouts/ directory contains layout strategies:

• standard.ts – Implements the default hierarchical layout for metadata storage
• documents.ts – Provides a document-centric layout optimized for documentation workflows

These engines determine how in-memory metadata structures translate to directory hierarchies and file naming conventions on disk.

Layout Orchestration

The toolbox/mdcode/src/libts/layout.ts file provides high-level orchestration of the layout process. It coordinates between the raw snapshot data and the specific layout engines, selecting appropriate strategies based on user configuration and catalog type.

Synchronization and Public API

The final components handle reconciliation and module exports.

Sync Engine

The toolbox/mdcode/src/libts/sync.ts file implements the sync engine that reconciles local metadata with the remote catalog. It detects changes between the local snapshot and remote state, generating diff reports and applying updates atomically to prevent catalog corruption.

Barrel Exports

The toolbox/mdcode/src/libts/index.ts file serves as the public barrel file that re-exports the library's public API. This centralizes the exposed functionality, allowing consumers to import from kcmd without referencing internal file paths directly.

Practical Implementation Examples

Working with the kcmd metadata structure involves importing types from the core files and instantiating the provided classes.

Import core metadata types and load a local snapshot:

// Import the core metadata types
import { Entry, Aspect } from 'kcmd/libts/metadata';

// Load a snapshot from the current directory
const ctx = kcmd.gcp.ApiContext.default();
const snapshot = await kcmd.CatalogSnapshot.fromPath('.', ctx);

// Access a specific entry
const entry: Entry | undefined = snapshot.entries.find(e => e.name === 'myDataset');
if (entry) {
  console.log('Entry type:', entry.type);
  console.log('Aspects:', entry.aspects);
}

Create a manifest from an existing BigQuery dataset:

// Create a manifest from a BigQuery dataset
import * as kcmd from 'kcmd';

const ctx = kcmd.gcp.ApiContext.default();
const manifest = await kcmd.CatalogManifest.initWithBigQuery(
  ['my-project.my_dataset'],
  ctx
);
await manifest.saveToPath('./catalog');

Summary

• The kcmd metadata source code resides in toolbox/mdcode/src/libts/ within the GoogleCloudPlatform/knowledge-catalog repository.
• Core interfaces (Entry, Aspect) are defined in metadata.ts, while snapshot.ts and manifest.ts handle data persistence.
• The sources/ directory contains concrete implementations for BigQuery, Dataplex, and Knowledge Base integrations.
• GCP-specific utilities in gcp/ provide API clients and authentication contexts.
• Layout engines in layouts/ and layout.ts map metadata to file system structures.
• The index.ts barrel file exposes the public API, and sync.ts manages remote reconciliation.

Frequently Asked Questions

Where is the kcmd metadata library located in the repository?

The kcmd metadata library is located at toolbox/mdcode/src/libts/ in the GoogleCloudPlatform/knowledge-catalog repository. This path contains all TypeScript source files, configuration files, and subdirectories that constitute the metadata management system.

What is the purpose of the sources/ directory in kcmd metadata?

The toolbox/mdcode/src/libts/sources/ directory houses concrete implementations of the abstract source class defined in source.ts. It contains platform-specific adapters including bq-dataset.ts for BigQuery, entrygroup.ts for Dataplex, and kb.ts for Knowledge Base entries, each handling unique API interactions for their respective services.

How do layout engines in the layouts/ directory function?

The toolbox/mdcode/src/libts/layouts/ directory contains layout engines that transform in-memory metadata snapshots into file system hierarchies. The standard.ts engine provides the default tree structure, while documents.ts offers a document-centric organization, with layout.ts orchestrating which strategy to apply based on configuration.

Which file defines the core Entry and Aspect interfaces?

The toolbox/mdcode/src/libts/metadata.ts file defines the Entry and Aspect interfaces that form the type foundation of the catalog system. These interfaces describe the structure of catalog objects and their associated metadata attributes throughout the kcmd library.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}