How OKFDocument Parses YAML Frontmatter in Bundle Documents

OKFDocument parses YAML frontmatter by detecting opening and closing --- delimiters, extracting the intermediate lines, and parsing them with yaml.safe_load, while everything following the closing delimiter becomes the document body.

The OKFDocument class in the GoogleCloudPlatform/knowledge-catalog repository serves as the core mechanism for reading and writing bundle markdown files containing YAML frontmatter. Understanding how this parser processes delimiter-separated metadata blocks is essential for developers working with the Knowledge Catalog's document ingestion pipeline.

The Frontmatter Parsing Pipeline

The parsing implementation resides in okf/src/reference_agent/bundle/document.py. The algorithm follows a strict five-step sequence to separate metadata from content.

Detecting the Opening Delimiter

The parser first checks if the document begins with the line ---, stored internally as _FRONTMATTER_DELIM. If the first line does not match this delimiter, the entire text is treated as the document body and the method returns an empty frontmatter dictionary (lines 24-27).

Locating the Closing Delimiter

Starting from the second line, the parser scans sequentially until it encounters another ---. The index of this closing line is captured as end_idx. Failure to locate a closing delimiter triggers an OKFDocumentError with the message "Undterminated YAML frontmatter block" (lines 28-34).

Parsing the YAML Block

The lines between the delimiters—specifically lines[1:end_idx]—are joined into a single string called fm_text. This string is passed to yaml.safe_load for parsing. If YAML parsing fails, the exception is wrapped in an OKFDocumentError stating "Invalid YAML in frontmatter" (lines 38-40). Additionally, if the parsed object is not a dictionary (YAML mapping), the parser raises OKFDocumentError with "Frontmatter must be a YAML mapping" (lines 41-42).

Extracting the Document Body

All lines following the closing delimiter (lines[end_idx + 1:]) constitute the document body. The parser strips a leading newline from this section to ensure clean formatting before storing it in the dataclass (lines 44-47).

Error Handling and Validation

The implementation includes strict validation to ensure data integrity across the Knowledge Catalog ecosystem.

Invalid YAML and Type Checking

Beyond basic syntax validation, the parser enforces that frontmatter must resolve to a key-value mapping structure. Scalar values or lists at the root level are rejected immediately, preventing malformed documents from entering the processing pipeline as implemented in okf/src/reference_agent/bundle/document.py.

Required Field Validation

The validate() method (lines 56-61) checks for mandatory keys: type, title, description, and timestamp. If any required field is missing, the method raises an error, ensuring that all documents meet the catalog's metadata standards before serialization or indexing.

Serializing Bundle Documents

The class provides a serialize() method that reverses the parsing process, reconstructing the original delimiter-separated format. This enables round-trip editing where documents can be parsed, modified programmatically, and written back to disk while preserving the frontmatter structure required by the Knowledge Catalog.

Practical Implementation Example

Here is how to parse and validate bundle documents using the class:

from reference_agent.bundle.document import OKFDocument, OKFDocumentError

# Example markdown with YAML frontmatter

md = """---
type: article
title: Understanding OKFDocument
description: A short guide
timestamp: 2024-07-16
---

# Introduction

This document explains how parsing works.
"""

# Parse the markdown

try:
    doc = OKFDocument.parse(md)
except OKFDocumentError as e:
    print("Parse failed:", e)

print(doc.frontmatter)  # {'type': 'article', 'title': ...}

print(doc.body)         # "# Introduction\n\nThis document..."

# Validate required keys

doc.validate()          # Raises exception if keys missing

# Serialize back to markdown

serialized = doc.serialize()

Summary

• Delimiter detection: The parser requires documents to start with --- and identifies the closing delimiter to isolate the frontmatter block according to the logic in okf/src/reference_agent/bundle/document.py.
• YAML parsing: The extracted block is parsed using yaml.safe_load with strict type checking to ensure dictionary output.
• Error handling: Specific OKFDocumentError exceptions indicate unterminated blocks, invalid YAML syntax, or incorrect data types.
• Body extraction: Content following the closing delimiter becomes the document body with normalized whitespace handling.
• Validation: The validate() method enforces required metadata fields (type, title, description, timestamp).

Frequently Asked Questions

What happens if a bundle document lacks the opening --- delimiter?

If the first line is not ---, the parser treats the entire text as the document body and returns an empty dictionary for the frontmatter field. No exception is raised for missing opening delimiters in the GoogleCloudPlatform/knowledge-catalog implementation.

Which required fields does OKFDocument.validate() check for?

The validation method verifies the presence of four mandatory keys: type, title, description, and timestamp. If any are missing, it raises an OKFDocumentError immediately.

How does the parser handle malformed YAML syntax?

When yaml.safe_load fails to parse the frontmatter block, the parser catches the exception and re-raises it as an OKFDocumentError with the message "Invalid YAML in frontmatter".

Can OKFDocument reconstruct the original markdown format after parsing?

Yes, the serialize() method reconstructs the document by combining the frontmatter dictionary (rendered as YAML) and the body text, reinserting the --- delimiters to produce valid bundle markdown compatible with the Knowledge Catalog.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}