How to Query Finelog for CPU and Memory Profile Captures

Use the finelog query CLI with standard SQL against the iris.task or iris.profile tables to retrieve memory_peak_mb and cpu_time_sec metrics.

Finelog is the telemetry and profiling storage system in the marin-community/marin repository. It stores per-task CPU and memory data in predefined SQL tables that you query through a command-line interface. Understanding how to extract these metrics is essential for debugging performance bottlenecks in distributed task execution.

Finelog Data Model for Profiling

Finelog organizes profiling data into two primary tables. Choosing the correct table determines whether you get aggregate summaries or time-series snapshots.

The iris.task Table

The iris.task table stores one row per task execution with cumulative metrics. This is the default source for high-level performance analysis.

Column	Type	Purpose
task_id	string	Unique task identifier
attempt_id	string	Execution attempt for retries
memory_peak_mb	float	Maximum memory observed (MiB)
cpu_time_sec	float	Cumulative CPU seconds consumed

The iris.profile Table

The iris.profile table contains periodic snapshots during task execution. Query this for fine-grained temporal analysis.

Column	Type	Purpose
profile_ts	timestamp	When the sample was taken
cpu_time_sec	float	Cumulative CPU time at sample
memory_peak_mb	float	Memory usage at sample

The schema definitions for both tables are located in lib/finelog/src/finelog/schema.py as implemented in marin-community/marin.

Basic Query Syntax

The Finelog CLI automatically handles authentication through cached Iris IAP credentials. The general invocation pattern is:

uv run finelog query <cluster> '<SQL>'

• <cluster> — Finelog deployment name (e.g., marin, cw-us-east-08a)
• <SQL> — Single-quoted SQL statement

For multiline queries, pass SQL via stdin:

uv run finelog query marin <<'EOF'
SELECT task_id, memory_peak_mb
FROM "iris.task"
LIMIT 5
EOF

The CLI implementation resides in lib/finelog/src/finelog/deploy/cli.py according to the marin-community/marin source code.

Querying Memory Peak Captures

To find the highest memory usage per task execution, aggregate memory_peak_mb from iris.task:

uv run finelog query marin '
SELECT task_id,
       attempt_id,
       max(memory_peak_mb) AS peak_mib
FROM   "iris.task"
GROUP BY task_id, attempt_id
ORDER BY peak_mib DESC
'

This query groups by both task_id and attempt_id to handle task retries correctly. The max() aggregation ensures you capture the worst-case memory consumption for each attempt.

Filtering by Task Prefix

Add a WHERE clause to narrow results to specific task families:

uv run finelog query cw-us-east-08a <<'SQL'
SELECT task_id,
       attempt_id,
       max(memory_peak_mb) AS peak_mib
FROM   "iris.task"
WHERE  task_id LIKE '/power/example/%'
GROUP BY task_id, attempt_id
ORDER BY peak_mib DESC
LIMIT 10
SQL

Always include time bounds or LIMIT clauses to prevent full-table scans that trigger timeouts.

Querying CPU Time Captures

Finelog does not store instantaneous CPU percentage. Instead, it tracks cumulative cpu_time_sec—the total processor seconds consumed by a task.

Retrieve total CPU time per execution:

uv run finelog query marin '
SELECT task_id,
       attempt_id,
       max(cpu_time_sec) AS cpu_seconds
FROM   "iris.task"
GROUP BY task_id, attempt_id
ORDER BY cpu_seconds DESC
'

The max() aggregation is technically redundant for iris.task (each row represents one attempt), but it defends against duplicate rows during data backfills.

Time-Series CPU Analysis

For per-sample CPU progression, query iris.profile:

uv run finelog query marin '
SELECT task_id,
       attempt_id,
       profile_ts,
       cpu_time_sec,
       memory_peak_mb
FROM   "iris.profile"
WHERE  task_id = '/power/example/run-42'
  AND  profile_ts >= now() - INTERVAL '5 minutes'
ORDER BY profile_ts DESC
LIMIT 20
'

This reveals how CPU time accumulates over the task lifetime. Subtract successive cpu_time_sec values to derive per-interval CPU usage.

Performance Best Practices

Efficient Finelog queries require attention to data volume and partitioning.

Always Bound Time Ranges

Unbounded queries scan entire namespaces and frequently timeout. See the "Diagnosing query latency" section in lib/finelog/OPS.md (lines 73-82) for troubleshooting guidance.

Bad:

uv run finelog query marin 'SELECT * FROM "iris.profile"'

Good:

uv run finelog query marin '
SELECT *
FROM   "iris.profile"
WHERE  profile_ts >= now() - INTERVAL '1 hour'
'

Discover Available Namespaces

List all queryable tables before writing queries:

uv run finelog namespaces marin

This command is documented in lib/finelog/OPS.md (lines 54-60) and helps verify table names against your target cluster's schema.

Choose Output Formats

• Default: JSONL (machine-parseable)
• Human-readable: --format table

uv run finelog query marin 'SELECT * FROM "iris.task" LIMIT 3' --format table

How Profiling Data Is Produced

Understanding data provenance helps interpret query results. Iris jobs emit profiling rows through lib/iris/scripts/job_profile_summary.py, which writes to Finelog during task execution. The Iris IAP authentication flow described in lib/finelog/OPS.md (lines 4-31) and lib/iris/OPS.md enables secure CLI access to these records.

Summary

• Query iris.task for per-execution aggregates of memory_peak_mb and cpu_time_sec
• Query iris.profile for time-series snapshots with profile_ts granularity
• Use uv run finelog query <cluster> '<SQL>' with stdin redirection for multiline statements
• Always filter by time to avoid timeouts on large namespaces
• Reference lib/finelog/OPS.md for authentication, query patterns, and performance diagnostics

Frequently Asked Questions

What is the difference between iris.task and iris.profile?

iris.task contains one row per task attempt with final cumulative metrics. iris.profile contains multiple rows per task—periodic samples captured during execution. Use iris.task for summary analysis and iris.profile for temporal debugging.

Why doesn't Finelog store CPU percentage?

CPU percentage is a derived metric requiring sampling intervals and host attribution. Finelog stores cpu_time_sec as a cumulative counter, which is monotonic, mergeable across distributed workers, and independent of clock skew. Calculate percentage by differencing consecutive samples and dividing by wall-clock time.

How do I handle authentication errors with finelog query?

The CLI retrieves Iris IAP credentials from your local cache. Run through the Iris authentication flow documented in lib/finelog/OPS.md (lines 4-31) and lib/iris/OPS.md before issuing queries. Cached credentials typically expire after 24 hours.

Can I join iris.task and iris.profile in a single query?

Yes—both tables share task_id and attempt_id keys. However, cross-table joins can be expensive. Prefer filtering each table independently, then correlating results in your analysis code to reduce query latency and resource consumption.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}