# How Does Bi-Directional Sync Work in kcmd? A Technical Deep Dive

> Discover how bi-directional sync works in kcmd. Learn the three phases of reconciling local JSON snapshots with remote Google Cloud catalogs for efficient data management.

- Repository: [Google Cloud Platform/knowledge-catalog](https://github.com/GoogleCloudPlatform/knowledge-catalog)
- Tags: deep-dive
- Published: 2026-07-15

---

**Bi-directional sync in `kcmd` reconciles local JSON snapshots with remote Google Cloud catalogs through a three-phase process: loading states, computing checksum-based differences, and applying changes via the `CatalogSync` class.**

The `kcmd` command-line tool from the [GoogleCloudPlatform/knowledge-catalog](https://github.com/GoogleCloudPlatform/knowledge-catalog) repository enables developers to manage Knowledge Catalog metadata as code. Understanding how **bi-directional synchronization** works requires examining the `CatalogSync` class and its reconciliation engine that bridges local file systems with BigQuery, Dataplex, and Knowledge Base APIs.

## The Three-Phase Synchronization Architecture

The `CatalogSync` class implements bi-directional sync through a deterministic reconciliation workflow. According to the source code in [`src/libts/sync.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/libts/sync.ts), the process treats both local snapshots and remote catalogs as hierarchical trees of `Entry` objects, then computes and applies differences.

### Phase 1: Loading Local and Remote States

The sync process begins by materializing both data sources into memory.

**Local State**: The `CatalogSnapshot.fromPath('.')` method reads on-disk metadata from JSON files in the current directory. Implemented in [`src/libts/snapshot.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/libts/snapshot.ts), this creates a hierarchy of `Entry` objects representing the local catalog state.

**Remote State**: The `CatalogManifest` class loads the remote catalog via GCP client libraries. Depending on the target service, it uses `initWithEntryGroup()`, `initWithKnowledgeBase()`, or `initWithBigQuery()` to fetch entries and construct an in-memory representation. This logic resides in [`src/libts/manifest.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/libts/manifest.ts).

### Phase 2: Computing Checksum-Based Differences

Once both states are loaded, `CatalogSync.diff()` compares the two trees. The algorithm:

1. Builds lookup maps keyed by entry IDs for both local and remote entries
2. Invokes `CatalogSync._compareEntries()` to evaluate each pair
3. Generates `CatalogDiff` objects categorizing changes into three sets:
   - **Added**: Entries present only in the remote catalog (for pull) or local snapshot (for push)
   - **Deleted**: Entries missing from the opposite side
   - **Modified**: Entries with identical IDs but differing content

The comparison relies on **SHA-256 checksums** calculated from each entry's JSON representation. By comparing `entry.checksum` values rather than performing deep object equality checks, the system efficiently detects any metadata drift.

### Phase 3: Applying Changes with Pull and Push

The final phase applies the computed diff in the direction specified by the command:

**Pull Operations** (`kcmd pull`): The `CatalogSync.applyPull()` method writes new or changed entries to the local snapshot root as JSON files and deletes local files corresponding to remote deletions. This synchronizes the remote state to the local filesystem.

**Push Operations** (`kcmd push`): The `CatalogSync.applyPush()` method invokes GCP APIs to mutate the remote catalog. It calls `catalogEntries.create`, `catalogEntries.update`, and `catalogEntries.delete` RPCs to make the remote catalog match the local snapshot.

Both commands support `--dry-run` to preview changes without applying them, and `--force` to override checksum conflicts.

## Conflict Resolution and Safety Mechanisms

When executing `kcmd push`, the system performs strict conflict detection. If a remote entry's checksum differs from the local version, `CatalogSync` flags this as a conflict and aborts the operation with a clear error message unless `--force` is specified.

To inspect pending changes without applying them, use `kcmd status`. This command runs the same diff logic as the sync operations but outputs a concise table of added, modified, and deleted items, providing visibility into drift before reconciliation.

## Command-Line Workflow

The typical bi-directional sync workflow follows this pattern:

```bash

# Initialize a local snapshot from a BigQuery dataset

kcmd init --bigquery-dataset my-project.my_dataset

# Preview changes before applying

kcmd status          # Display diff between local and remote

kcmd pull --dry-run  # Preview remote-to-local updates

kcmd push --dry-run  # Preview local-to-remote updates

# Apply changes

kcmd pull            # Update local files from remote

kcmd push            # Push local edits to Google Cloud

```

This workflow enables GitOps-style catalog management where metadata changes undergo code review before deployment to production environments.

## Programmatic Sync with TypeScript

Beyond the CLI, `kcmd` exposes the synchronization engine as a TypeScript library. The following example demonstrates programmatic bi-directional sync:

```typescript
import * as kcmd from 'kcmd';

// Initialize API context
const ctx = kcmd.gcp.ApiContext.default();

// Load local snapshot from current directory
const snapshot = await kcmd.CatalogSnapshot.fromPath('.', ctx);

// Load remote catalog (BigQuery example)
const remote = await kcmd.CatalogManifest.initWithBigQuery(
  ['my-project.my_dataset'], ctx);

// Instantiate sync manager
const sync = new kcmd.CatalogSync(remote, snapshot);

// Compute and display differences
const diff = sync.diff();
console.log('Added:', diff.added.length);
console.log('Modified:', diff.modified.length);
console.log('Deleted:', diff.deleted.length);

// Apply changes
await sync.applyPush();   // Push local to remote
// await sync.applyPull(); // Or pull remote to local

```

The `CatalogSync` class handles all checksum validation and API communication, allowing developers to integrate catalog synchronization into custom pipelines or CI/CD systems.

## Summary

- **Bi-directional sync** in `kcmd` uses the `CatalogSync` class to reconcile local JSON snapshots with remote Google Cloud catalogs.
- The process follows three phases: loading states via `CatalogSnapshot` and `CatalogManifest`, computing SHA-256 checksum differences, and applying changes through `applyPull()` or `applyPush()`.
- Key implementation files include [`src/libts/sync.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/libts/sync.ts) for the core logic, [`src/libts/snapshot.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/libts/snapshot.ts) for local state, and [`src/libts/manifest.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/src/libts/manifest.ts) for remote API integration.
- Safety features include `--dry-run` for previewing changes, `kcmd status` for diff inspection, and checksum-based conflict detection with `--force` override support.

## Frequently Asked Questions

### What is the difference between `kcmd pull` and `kcmd push`?

`kcmd pull` executes `CatalogSync.applyPull()`, which updates local JSON files to match the remote catalog state, effectively importing changes from Google Cloud. `kcmd push` executes `CatalogSync.applyPush()`, which updates the remote catalog via GCP APIs to match the local snapshot, exporting your local edits to BigQuery, Dataplex, or Knowledge Base.

### How does `kcmd` detect conflicts during synchronization?

The system calculates a SHA-256 checksum for each entry's JSON representation and stores it in `entry.checksum`. When pushing, if the remote entry's checksum differs from the local version (indicating someone else modified the remote catalog), `kcmd` detects a conflict and aborts the operation unless you provide the `--force` flag to overwrite the remote version.

### Can I preview changes before applying them in `kcmd`?

Yes. The `kcmd status` command runs the diff logic without applying changes, displaying a table of added, modified, and deleted entries. Additionally, both `kcmd pull` and `kcmd push` accept the `--dry-run` flag, which executes the full synchronization logic but stops before calling write operations, allowing you to verify the proposed changes safely.

### How do I resolve checksum conflicts when pushing changes?

When `kcmd push` encounters a checksum mismatch, it aborts with an error indicating which entry has diverged. You can resolve this by either pulling the remote changes first with `kcmd pull` to merge updates, or by forcing the local version to overwrite the remote entry using `kcmd push --force`. The `--force` flag bypasses the checksum validation and applies your local state regardless of remote modifications.