Standard Layout vs Documents Layout in Google Cloud Knowledge Catalog: Choosing the Right Metadata Format

Choose the Standard Layout for structured resources like BigQuery datasets that store metadata in dedicated YAML files, and the Documents Layout for knowledge-base assets where the primary content is unstructured text managed in Markdown with YAML front-matter.

Google Cloud Knowledge Catalog treats metadata as version-controlled code artifacts, offering two distinct on-disk arrangements for organizing entries. The repository implements both the Standard Layout and the Documents Layout in the GoogleCloudPlatform/knowledge-catalog codebase to accommodate different ratios of structured versus unstructured data, with both layouts conforming to the same underlying md.Entry metadata model.

What Are the Knowledge Catalog Layouts?

The Knowledge Catalog stores metadata as code so developers and agents can version-control, review, and automate changes through Git workflows. The repository defines two filesystem arrangements in src/libts/layout.ts via the CatalogLayout interface:

• Standard Layout: Stores each entry as a dedicated YAML file containing complete resource-level metadata
• Documents Layout: Stores each entry as a single .md file with structured fields embedded in YAML front-matter

Both layouts abstract the filesystem structure behind the CatalogLayout interface, ensuring downstream agents remain agnostic to which layout you choose.

The Standard Layout: YAML for Structured Resources

The Standard Layout works best for large, structured resources such as BigQuery datasets or Dataplex entry groups. In this arrangement, each entry gets a dedicated YAML file that holds the complete resource-level metadata, including the name, description, and aspects like schema or profile data.

Unstructured text content—such as a detailed overview—can be kept in side-car Markdown files that sit next to the YAML file. This separation keeps the YAML parsing simple and enables bulk operations (for example, using grep across all entries) while allowing large textual fields to be edited without bloating the YAML structure.

The implementation resides in src/libts/layouts/standard.ts, where the saveEntry() method writes a YAML file under the catalog root.

The Documents Layout: Markdown for Knowledge Bases

The Documents Layout shines for knowledge-base (KB) style assets where the primary unstructured content is the entry's description. In this format, the entry is a single .md file where structured fields live in a YAML front-matter block at the top, while the main body holds the primary unstructured aspect (overview.content).

This layout aligns with common static-site conventions, making it easier for non-technical contributors to edit entries directly in the GitHub UI or documentation workflows. Additional aspects that are too large for front-matter may still be stored in side-car Markdown files automatically.

The implementation in src/libts/layouts/documents.ts handles the front-matter parsing and body extraction, ensuring the overview.content field maps to the Markdown body while other metadata remains in the YAML header.

Comparing Layouts: When to Use Which

The choice hinges on how much structured versus unstructured data each entry contains and on the resource type you are modeling.

Standard Layout works best when you have many aspects (schema, profiling, lineage) that are naturally expressed as separate structured objects. Keeping them in a YAML file keeps parsing simple and enables bulk operations. Side-car Markdown files let you edit large textual fields without bloating the YAML.

Documents Layout is optimal for KB-style content where the overview is the central artifact. By embedding metadata in front-matter, the entry remains a single Markdown file that can be edited directly in documentation workflows. This format also integrates cleanly with static-site generators.

Working with Both Layouts in Code

Both layouts share the same metadata model (md.Entry defined in src/libts/metadata.ts), so switching between them does not affect downstream agents. The Snapshot class in toolbox/mdcode/src/libts/snapshot automatically detects the layout when loading entries.

Loading an entry regardless of layout:

import { Snapshot } from 'toolbox/mdcode/src/libts/snapshot';

const snapshot = new Snapshot(snapshotPath);
await snapshot.init();                     // Detects layout automatically
const entry = await snapshot.loadEntry('my_entry_id');
console.log(entry.resource.displayName);

Saving an entry using the Standard Layout creates a <name>.yaml file:

import { md } from 'toolbox/mdcode/src/libts/metadata';

const entry: md.Entry = {
  name: 'projects/myproj/datasets/mydataset',
  resource: { displayName: 'My Dataset', description: 'Raw events' },
  aspects: { schema: { fields: [...] } },
};

await snapshot.saveEntry(entry.name, entry);   // Writes a YAML file under the catalog root

Saving an entry using the Documents Layout creates a <name>.md file with front-matter:

import { md } from 'toolbox/mdcode/src/libts/metadata';
import { DocumentsLayout } from 'toolbox/mdcode/src/libts/layouts/documents';

const layout = new DocumentsLayout('/path/to/snapshot');
await layout.init();

const entry: md.Entry = {
  name: 'kb/articles/getting-started',
  resource: { displayName: 'Getting Started', description: 'Intro to Knowledge Catalog' },
  // Aspects that are not overview go into side-car markdown files automatically
};

await layout.saveEntry(entry.name, entry);   // Writes <name>.md with front-matter and body

Technical Implementation Details

The layout system is defined across several key files in the repository:

• src/libts/layout.ts: Defines the CatalogLayout abstraction used by both layouts
• src/libts/layouts/standard.ts: Implements the Standard Layout (YAML per entry)
• src/libts/layouts/documents.ts: Implements the Documents Layout (Markdown with YAML front-matter)
• src/libts/metadata.ts: Central TypeScript definitions for Entry, Aspect, and related types
• docs/spec.md: Human-readable specification of the two layouts and the overall metadata-as-code model

Using the appropriate layout ensures that your metadata remains easy to author, version, and synchronize with the Knowledge Catalog service while matching the natural structure of the underlying assets.

Summary

• Standard Layout uses dedicated YAML files per entry (<name>.yaml) with optional side-car Markdown files, ideal for structured resources like BigQuery datasets.
• Documents Layout uses single Markdown files with YAML front-matter (<name>.md), optimal for knowledge-base content where the overview is the primary artifact.
• Both layouts implement the CatalogLayout interface and use the same md.Entry model, ensuring interoperability.
• The Snapshot class auto-detects the layout on initialization, while the DocumentsLayout class allows manual control when needed.

Frequently Asked Questions

When should I use the Standard Layout versus the Documents Layout?

Use the Standard Layout when modeling structured resources with multiple complex aspects (schemas, lineage, profiling data) that benefit from separate YAML files. Choose the Documents Layout for knowledge-base entries where the primary content is unstructured documentation that authors will edit frequently in Markdown workflows.

Can I switch between layouts without breaking downstream agents?

Yes. Both layouts conform to the same md.Entry interface defined in src/libts/metadata.ts and implement the CatalogLayout abstraction from src/libts/layout.ts. The Snapshot class abstracts filesystem differences, so agents consuming the catalog remain agnostic to which layout generated the files.

What happens to large aspect data in the Documents Layout?

While the Documents Layout stores primary metadata and the overview.content in the front-matter and body of a single .md file, additional aspects that are too large for front-matter are automatically stored in side-car Markdown files. This prevents the front-matter from becoming unwieldy while keeping the entry conceptually unified.

How does the Snapshot class detect which layout to use?

The Snapshot class in toolbox/mdcode/src/libts/snapshot initializes the appropriate layout implementation during the init() call by inspecting the filesystem structure. When you call loadEntry(), it uses the detected layout to resolve the entry file regardless of whether it is stored as YAML (Standard) or Markdown with front-matter (Documents).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}