How the `sample_rows` Function Handles Non-TABLE BigQuery Objects: Views, Materialized Views, and External Tables

The sample_rows function detects when a BigQuery object is not a base table—such as a view, materialized view, or external table—and automatically falls back to executing a SELECT * ... LIMIT n query instead of using the list_rows API, which only supports native tables.

The sample_rows method in the GoogleCloudPlatform/knowledge-catalog repository provides unified row sampling across diverse BigQuery resource types. Located within the BigQuerySource class in okf/src/reference_agent/sources/bigquery.py, this function inspects the underlying object type to determine whether it can use efficient storage APIs or must materialize data through a query execution.

Understanding the sample_rows Implementation

File Location and Core Logic

The sampling logic resides in okf/src/reference_agent/sources/bigquery.py within the BigQuerySource class. When invoked, sample_rows first retrieves metadata for the target concept to determine the actual BigQuery object type:

table_type = (getattr(tbl, "table_type", None) or "TABLE").upper()

This table type detection determines whether the target is a standard table or a specialized object like a view, materialized view, external table, or snapshot.

The Two-Path Execution Strategy

Based on the table_type value, sample_rows selects between two distinct retrieval mechanisms:

• Native tables (TABLE): Use the streaming list_rows API
• Non-table objects (VIEW, MATERIALIZED_VIEW, EXTERNAL, SNAPSHOT): Execute a materialization query

How sample_rows Processes Different BigQuery Object Types

Native Tables: The Streaming API Path

When table_type equals "TABLE", the function calls the efficient list_rows method, which streams rows directly from BigQuery storage:

row_iter = self.client.list_rows(table_ref, max_results=n)

This approach provides optimal performance because it reads directly from the table's underlying storage without invoking the query engine.

Views and Non-Table Objects: The Query Fallback

For any non-TABLE type, sample_rows cannot use list_rows because the underlying tabledata.list REST endpoint refuses to read from non-base tables. Instead, the function constructs and executes a lightweight query that materializes the rows:

# VIEW / MATERIALIZED_VIEW / EXTERNAL / SNAPSHOT — the

# tabledata.list REST endpoint refuses non‑base‑tables, so

# fall back to a small query that materializes the rows.

sql = (
    f"SELECT * FROM `{self.dataset_project}."
    f"{self.dataset_id}.{table_id}` LIMIT {int(n)}"
)
row_iter = self.client.query(sql).result()

This query selects all columns and limits results to the requested n rows, returning an iterator that the function converts to a list of dictionaries.

Error Handling and Return Format

After retrieving rows through either path, sample_rows transforms the results into a standardized format:

return [dict(r.items()) for r in row_iter]

If any step fails—such as when a view references unavailable tables or permission errors occur—the function returns None to enable graceful degradation.

Practical Usage Examples

Sampling from a Standard Table

When sampling from a native BigQuery table, sample_rows uses the efficient list_rows API:

from okf.src.reference_agent.sources.bigquery import BigQuerySource

src = BigQuerySource("my-project.my_dataset")
ref = src.list_concepts()[0]          # assume first concept is a table

rows = src.sample_rows(ref, n=5)      # → uses list_rows()

print(rows)

Sampling from a View

For views, the function automatically switches to the query fallback mechanism:

src = BigQuerySource("my-project.my_dataset")

# Find a view concept – its hint will have `wildcard: False` and `table_type` will be "VIEW"

view_ref = next(c for c in src.list_concepts()
                if c.type == "BigQuery Table"
                and src.client.get_table(src._dataset_ref.table(src._representative_table_id(c))).table_type == "VIEW")
rows = src.sample_rows(view_ref, n=3)   # → runs SELECT * … LIMIT 3

print(rows)

Summary

• sample_rows in okf/src/reference_agent/sources/bigquery.py provides unified sampling across all BigQuery object types.
• Table type detection via the table_type attribute determines the execution path.
• Native tables use the fast list_rows API for direct storage access.
• Non-table objects (views, materialized views, external tables, snapshots) trigger a SELECT * ... LIMIT n query fallback because the list_rows endpoint only supports base tables.
• Error handling returns None when metadata retrieval or query execution fails.

Frequently Asked Questions

Why can't sample_rows use list_rows for BigQuery views?

The underlying BigQuery tabledata.list REST endpoint only supports reading from base tables with physical storage. Views, materialized views, and external tables lack direct storage backing, so sample_rows must execute a query to materialize the data before sampling.

What happens when sample_rows encounters an external table?

External tables (identified by table_type == "EXTERNAL") follow the same fallback path as views. The function executes SELECT * ... LIMIT n against the external table, allowing BigQuery's query engine to handle the external data source connection and return sampled rows.

How does the function handle permission errors when sampling from a view?

If the view cannot be queried due to insufficient permissions or if the underlying tables are inaccessible, sample_rows catches the exception and returns None. This graceful failure handling prevents the agent from crashing while indicating that sampling is unavailable for that specific resource.

Is there a performance difference between the two sampling methods?

Yes. The list_rows API provides faster access for native tables because it streams directly from BigQuery storage. The query fallback requires the query engine to plan and execute a job, introducing additional latency for views and external tables, though the LIMIT clause minimizes data processing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}