How to Push Local Changes to the Catalog Service Using kcmd

Use the kcmd push command to synchronize your local metadata files with Google Dataplex, creating new entries or updating existing ones based on your local snapshot.

The kcmd CLI tool, part of the GoogleCloudPlatform/knowledge-catalog open-source repository, enables metadata-as-code workflows by treating Dataplex catalogs as version-controlled files. When you finish editing local entries or the catalog.yaml manifest, you must push local changes to the catalog service using kcmd to propagate modifications to the remote Dataplex API. This command compares your local state against the remote service and applies incremental updates.

How the kcmd Push Workflow Works

The push operation follows a structured pipeline that moves from CLI parsing to API communication. Understanding these steps helps troubleshoot synchronization issues and integrate the logic into custom automation.

CLI Entry Point and Command Parsing

In toolbox/mdcode/src/tool/main.ts, the CLI entry point parses the push sub-command and forwards the parsed options to the appropriate handler. This module handles argument validation and ensures flags like --force or --validate-only are recognized before execution begins.

Command Handler Initialization

The push handler in toolbox/mdcode/src/tool/commands.ts (lines 82-99) orchestrates the preparation phase. It performs three critical actions:

1. Creates a default ApiContext using Application Default Credentials
2. Loads the local CatalogSnapshot from the current directory (reading catalog.yaml and entry files)
3. Instantiates a CatalogClient to wrap the Dataplex REST API

Sync Logic and State Comparison

The CatalogSync.push method in toolbox/mdcode/src/libts/sync.ts (lines 63-99) implements the core synchronization algorithm. This method:

• Retrieves the list of entry names from the local snapshot
• Compares each entry against the remote state using lookupEntry
• Calls createEntry for new resources or modifyEntry for existing ones
• Returns a SyncResult indicating success or detailed error messages

Dataplex API Communication

The CatalogClient in toolbox/mdcode/src/libts/gcp/catalog.ts executes the actual HTTP requests to Google Cloud. It provides methods including lookupEntry, createEntry, and modifyEntry that translate local metadata into Dataplex API calls. If any operation fails, the client returns error details that propagate back to the CLI exit codes.

Practical Usage Examples

Basic CLI Workflow

Follow this standard workflow to initialize, edit, and push your catalog:

1. Initialize a local snapshot (one-time setup):

   kcmd init --bigquery-dataset my-project.my_dataset

2. Edit the generated files (catalog.yaml or entry JSON/YAML files)

3. Push changes to Dataplex:

   kcmd push

CLI Options and Flags

Control push behavior with optional flags:

• Normal push ( incremental updates only):

  kcmd push

• Force push (overwrites remote changes without conflict checks):

  kcmd push --force

• Validate only (dry-run without modifying remote state):

  kcmd push --validate-only

• Dry run (shows what would be sent without performing writes):

  kcmd push --dry-run

Programmatic Push Implementation

You can replicate the kcmd push behavior programmatically using the TypeScript API:

import * as kcmd from 'kcmd';

// Create an API context using Application Default Credentials
const ctx = kcmd.gcp.ApiContext.default();

// Load the local snapshot from the catalog root directory
const snapshot = await kcmd.CatalogSnapshot.fromPath('.', ctx);

// Initialize the Dataplex client
const catalog = new kcmd.gcp.CatalogClient(ctx);

// Build the sync coordinator
const sync = new kcmd.CatalogSync(catalog, snapshot);

// Execute the push with optional flags
const result = await sync.push({ force: false, validateOnly: false });

if (result.success) {
  console.log('✅ Push succeeded');
} else {
  console.error('❌ Push failed:', result.details);
  process.exit(1);
}

Summary

• kcmd push synchronizes local metadata files with Google Dataplex by comparing your snapshot against the remote state.
• The workflow parses commands in main.ts, initializes resources in commands.ts, and executes sync logic in sync.ts before calling the Dataplex API via gcp/catalog.ts.
• Use --force to overwrite remote conflicts, --validate-only to dry-run changes, or --dry-run to preview modifications without writing.
• The TypeScript API exposes CatalogSync.push() for custom automation pipelines requiring programmatic control.

Frequently Asked Questions

What does kcmd push do exactly?

The kcmd push command reads your local CatalogSnapshot (including catalog.yaml and entry files), compares each entry against the current state in Google Dataplex, and applies necessary changes. It creates new entries that don't exist remotely and updates existing ones that differ from your local version. The command exits with a non-zero status if any operation fails.

How do I force push changes without conflict checks?

Add the --force flag to your push command: kcmd push --force. This instructs the CatalogSync class in toolbox/mdcode/src/libts/sync.ts to skip conflict detection and overwrite remote entries regardless of their current state. Use this option with caution, as it may destroy concurrent modifications made by other users or processes.

Can I validate changes before pushing to Dataplex?

Yes, use the --validate-only flag to perform a dry-run that validates your local files without contacting the Dataplex service. You can also combine this with --dry-run to see exactly what API calls would be executed. These flags are processed in commands.ts and prevent the CatalogClient from sending actual write requests.

What happens if a push fails partway through?

If any entry fails to synchronize during the sync.push() operation, the command logs the error details and exits with a non-zero status code. Successfully pushed entries remain committed in Dataplex; the operation does not roll back previously applied changes. Check the error output to identify which specific entry caused the failure, fix the local metadata, and rerun kcmd push.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}