# What Is CatalogSnapshot and How Does It Manage Local Catalog Entries?

> Learn what CatalogSnapshot is and how it manages local catalog entries by loading Dataplex catalog manifests and delegating file-system operations.

- Repository: [Google Cloud Platform/knowledge-catalog](https://github.com/GoogleCloudPlatform/knowledge-catalog)
- Tags: deep-dive
- Published: 2026-07-15

---

**A CatalogSnapshot is the core abstraction that loads a Dataplex catalog manifest locally, builds in-memory maps of entry and aspect types, and delegates file-system operations to a layout implementation that manages how entries are stored on disk.**

The **CatalogSnapshot** class serves as the bridge between remote Dataplex metadata and the local file-based catalog used by the GoogleCloudPlatform/knowledge-catalog toolbox. It ensures that only entry types and aspects declared in the [`catalog.yaml`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/catalog.yaml) manifest are available, while respecting whether the catalog is ingested (read-only) or user-managed (editable).

## Understanding the CatalogSnapshot Architecture

In [`toolbox/mdcode/src/libts/snapshot.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/toolbox/mdcode/src/libts/snapshot.ts), the `CatalogSnapshot` class encapsulates the entire lifecycle of a local catalog instance. Unlike direct API calls to Dataplex, the snapshot maintains a **local copy** of the catalog manifest and metadata definitions, providing offline-capable operations with synchronization capabilities.

The architecture centers on three pillars: manifest loading, type resolution, and layout abstraction. When you initialize a snapshot via `CatalogSnapshot.fromPath()`, it reads the [`catalog.yaml`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/catalog.yaml) file, fetches type definitions from the Dataplex API, and selects a concrete `CatalogLayout` implementation based on the manifest's source configuration.

## Core Responsibilities of CatalogSnapshot

### Loading the Manifest

The entry point `CatalogSnapshot.fromPath()` (lines 32-38 in [`snapshot.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/snapshot.ts)) reads the [`catalog.yaml`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/catalog.yaml) file from the specified directory and instantiates a `CatalogManifest` object. This manifest defines the catalog's structure, including whether it uses standard YAML files or document-centric storage.

### Building Type Maps

After loading the manifest, the snapshot calls `_buildTypes()` (lines 35-76) to query the Dataplex API for each entry-type and aspect-type listed in `manifest.snapshotConfig`. The results are cached as `Map<string, EntryType>` and `Map<string, AspectType>` on the snapshot instance, ensuring type validation can occur locally without additional API calls.

### Selecting the Layout

Based on `manifest.source.layout`, the snapshot uses `createLayout()` (lines 28-30) to obtain a `CatalogLayout` implementation. As defined in [`toolbox/mdcode/src/libts/layout.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/toolbox/mdcode/src/libts/layout.ts) (lines 25-33), this layout determines whether entries are stored as separate YAML files (`StandardLayout` in [`layouts/standard.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/layouts/standard.ts)) or as document collections (`DocumentsLayout` in [`layouts/documents.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/layouts/documents.ts)).

### CRUD Operations

The snapshot delegates all file-system operations to the layout while enforcing manifest rules:

- **List entries**: `listEntries()` (lines 54-62) returns all entry names from the local `catalog/` directory
- **Read entries**: `lookupEntry()` retrieves specific entries via the layout
- **Create/Update/Delete**: `createEntry()`, `updateEntry()`, and `deleteEntry()` (lines 9-33, 64-84) manipulate on-disk representations while respecting "user-managed vs. ingested" rules
- **Sync conversion**: Private helpers `_storeEntry()` and `_fetchEntry()` (lines 78-108, 86-108) translate between the Dataplex service model and the local metadata model (`md.Entry`)

## Working with CatalogSnapshot

The following examples demonstrate how to load a catalog snapshot and perform operations on local entries using the implementation from [`toolbox/mdcode/src/libts/snapshot.ts`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/toolbox/mdcode/src/libts/snapshot.ts).

### Loading a Snapshot from Local Directory

```typescript
import { CatalogSnapshot } from './snapshot';
import * as gcp from './gcp/context';

async function openCatalog(dir: string) {
  const ctx = await gcp.createApiContext();   // creates an authenticated GCP client
  const snapshot = await CatalogSnapshot.fromPath(dir, ctx);
  return snapshot;
}

```

### Listing and Reading Entries

```typescript
// List all locally stored entries
async function listAll(dir: string) {
  const snap = await openCatalog(dir);
  const names = await snap.listEntries();   // e.g. ['mydataset.mytable', 'myview']
  console.log(names);
}

// Read a specific entry
async function readEntry(dir: string, name: string) {
  const snap = await openCatalog(dir);
  const entry = await snap.lookupEntry(name);
  console.log(entry);
}

```

### Updating Local Entries

```typescript
// Update the description of an entry's resource metadata
async function updateDescription(dir: string, name: string, newDesc: string) {
  const snap = await openCatalog(dir);
  const entry = await snap.lookupEntry(name);
  entry.resource = entry.resource || {};
  entry.resource.description = newDesc;
  await snap.updateEntry(entry, ['resource']);
}

```

## Summary

- **CatalogSnapshot** acts as the local bridge between Dataplex cloud metadata and the file-based catalog in the GoogleCloudPlatform/knowledge-catalog repository.
- The class loads [`catalog.yaml`](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/catalog.yaml) via `fromPath()` and caches type definitions from the Dataplex API to enable offline validation.
- File-system operations are delegated to a **CatalogLayout** implementation (either `StandardLayout` or `DocumentsLayout`) selected based on the manifest configuration.
- CRUD operations respect the catalog's mode (ingested vs. user-managed) as defined in the manifest, preventing unauthorized modifications to read-only catalogs.
- Internal translation methods `_storeEntry()` and `_fetchEntry()` handle conversion between the Dataplex service model and the local `md.Entry` format.

## Frequently Asked Questions

### What is the difference between ingested and user-managed catalogs?

**Ingested catalogs** are read-only local copies of Dataplex metadata that cannot be modified through the snapshot, while **user-managed catalogs** allow local edits that can be pushed back to Dataplex. The `CatalogSnapshot` enforces these rules based on the manifest configuration, preventing accidental writes to ingested entries.

### How does CatalogSnapshot handle entry type validation?

The snapshot validates entry types against cached `Map<string, EntryType>` and `Map<string, AspectType>` objects built during initialization. These maps are populated by querying the Dataplex API for each type listed in `manifest.snapshotConfig`, ensuring that only declared types can be created or modified locally.

### What file formats does CatalogSnapshot support?

According to the source code in `toolbox/mdcode/src/libts/layouts/`, the snapshot supports two storage formats: **StandardLayout** stores each entry as a separate YAML file under the `catalog/` directory, while **DocumentsLayout** stores entries as document collections for document-centric catalogs. The specific layout is determined by the `manifest.source.layout` configuration.

### How does CatalogSnapshot sync with Dataplex?

The snapshot uses private helper methods `_storeEntry()` and `_fetchEntry()` to translate between the Dataplex service model and the local `md.Entry` metadata model. While the snapshot maintains local copies for offline work, these conversion methods ensure compatibility when synchronizing changes back to the Dataplex API or fetching updates from remote sources.