OKF Agent Prompt Templates for BigQuery and Web Ingestion Tasks

The OKF agent relies on two specialized markdown prompt templates—reference_instruction.md for BigQuery metadata enrichment and web_ingestion_instruction.md for web crawling workflows—located in the okf/src/reference_agent/prompts/ directory and dynamically loaded at runtime.

The GoogleCloudPlatform/knowledge-catalog repository implements an Open Knowledge Format (OKF) agent that automates the creation of structured documentation for data assets. Understanding the specific prompt templates for BigQuery and web ingestion tasks is essential for customizing the agent’s behavior or debugging its output. These templates provide the LLM with precise instructions for metadata extraction, content synthesis, and citation handling.

BigQuery Enrichment Prompt Template

Template Location and Purpose

The BigQuery enrichment workflow uses the reference_instruction.md template stored at okf/src/reference_agent/prompts/reference_instruction.md. This file provides the instruction set for the reference agent that creates or augments OKF documents for BigQuery assets such as datasets and tables.

The template directs the agent to read existing documentation, fetch raw metadata from BigQuery, optionally sample rows, and emit a single write_concept_doc call containing the required front-matter and body sections. It ensures consistent formatting and content structure across all generated BigQuery asset documentation.

Runtime Loading Mechanism

At runtime, the agent loads this template via the _load_prompt utility function defined in the codebase. The function leverages Python’s importlib.resources to read the bundled markdown file:

from importlib import resources

def _load_prompt(filename: str) -> str:
    # Resources are bundled with the package; the function loads the markdown file.

    return resources.files("reference_agent.prompts").joinpath(filename).read_text()

In the reference agent implementation (okf/src/reference_agent/agent.py), the template is loaded as follows:

# In the reference agent (e.g., okf/src/reference_agent/agent.py)

instruction = _load_prompt("reference_instruction.md")

The resulting instruction string is then interpolated into the LLM request’s system or user messages to guide the agent’s BigQuery-specific behavior.

Web Ingestion Prompt Template

Template Location and Purpose

For web ingestion tasks, the OKF agent utilizes web_ingestion_instruction.md, located at okf/src/reference_agent/prompts/web_ingestion_instruction.md. This template defines the workflow for the web-ingestion agent responsible for crawling seed URLs and extracting authoritative reference concepts.

The template specifies budgeting constraints, link-filtering criteria, augmentation rules, and required citation handling. It enables the agent to either enrich existing concepts or mint new reference concepts—such as metrics, dimensions, and joins—derived from web content.

Integration with the Agent Pipeline

Similar to the BigQuery template, the web ingestion prompt is loaded using the same _load_prompt function:

instruction = _load_prompt("web_ingestion_instruction.md")

This instruction string configures the LLM to execute the multi-step web crawling and content extraction workflow defined in the template.

Technical Implementation and Supporting Files

The prompt loading architecture centers on the _load_prompt function, which reads files from the reference_agent.prompts package namespace. This approach ensures that prompt templates are treated as package resources, maintaining portability across different deployment environments.

Beyond the prompt files themselves, the okf/src/reference_agent/agent.py module orchestrates the loading and assembly of these templates into final LLM prompts. Additionally, okf/src/reference_agent/bundle/synthesizer.py utilizes a _PROMPT_TEMPLATE constant alongside the loaded instruction files to synthesize final document bundles.

File	Role
okf/src/reference_agent/prompts/reference_instruction.md	Prompt template for BigQuery metadata-driven enrichment.
okf/src/reference_agent/prompts/web_ingestion_instruction.md	Prompt template for crawling and ingesting web pages.
okf/src/reference_agent/agent.py	Loads templates and assembles the final prompt sent to the model.
okf/src/reference_agent/bundle/synthesizer.py	Uses _PROMPT_TEMPLATE together with loaded instruction files for bundle synthesis.

Summary

• The OKF agent employs two distinct prompt templates: reference_instruction.md for BigQuery tasks and web_ingestion_instruction.md for web ingestion workflows.
• Both templates reside in okf/src/reference_agent/prompts/ and are loaded at runtime via the _load_prompt function using importlib.resources.
• The reference_instruction.md template guides the creation of structured documentation for BigQuery datasets and tables, including metadata extraction and row sampling.
• The web_ingestion_instruction.md template governs web crawling logic, link filtering, and citation handling for deriving reference concepts from URLs.
• Core implementation files include okf/src/reference_agent/agent.py for orchestration and okf/src/reference_agent/bundle/synthesizer.py for document synthesis.

Frequently Asked Questions

What is the difference between the BigQuery and web ingestion prompt templates?

The BigQuery template (reference_instruction.md) focuses on metadata-driven enrichment of database assets, instructing the agent to fetch schema metadata and sample rows. The web ingestion template (web_ingestion_instruction.md) handles URL crawling, content extraction, and citation management for deriving knowledge from web sources.

Where are the OKF agent prompt templates stored in the repository?

Both templates are located under okf/src/reference_agent/prompts/ within the GoogleCloudPlatform/knowledge-catalog repository. Specifically, reference_instruction.md handles BigQuery workflows, while web_ingestion_instruction.md manages web ingestion tasks.

How does the OKF agent load prompt templates at runtime?

The agent calls the _load_prompt function, which uses Python’s importlib.resources to read markdown files from the reference_agent.prompts package. This method ensures reliable access to template files regardless of the execution environment.

Which source files handle the prompt template loading for the OKF agent?

The primary loading logic resides in okf/src/reference_agent/agent.py, which invokes _load_prompt to retrieve template content. The okf/src/reference_agent/bundle/synthesizer.py file additionally uses these templates alongside internal constants like _PROMPT_TEMPLATE to generate final output documents.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}