How RTK Compresses Test Output for pytest and Other Testing Frameworks

RTK (Rust Token Killer) compresses test output by executing commands with minimal verbosity flags, capturing raw stdout/stderr through runner::run_filtered, parsing results via framework-specific state machines or tiered parsers, and re-rendering concise summaries that preserve only statistics and critical failure excerpts.

The rtk-ai/rtk repository implements a sophisticated output compression pipeline that reduces token-heavy test logs into digestible summaries. By intercepting the execution of testing frameworks like pytest and Vitest, RTK applies framework-specific parsing strategies to extract essential statistics while discarding redundant output.

How RTK Compresses pytest Output

RTK handles Python testing through src/cmds/python/pytest_cmd.rs, implementing a state-machine filter that tracks four distinct parsing phases.

Command Preparation and Quiet Flags

Before execution, RTK prepares the pytest command to minimize initial verbosity. The code detects whether pytest exists as a standalone binary; if not, it falls back to python -m pytest. It automatically appends -q (quiet mode) and --tb=short (short traceback format) unless the user has already specified custom traceback flags.

// From src/cmds/python/pytest_cmd.rs lines 15-33
// Adds -q and --tb=short to reduce output volume

The prepared command is then passed to runner::run_filtered (located in src/core/runner.rs), which streams the child process, records stdout/stderr, and writes a raw copy to a tee file for debugging purposes.

State Machine Parsing

The filter_pytest_output function implements a four-state parser that processes output line-by-line:

• Header: Identifies the start of test output
• TestProgress: Captures test file progress lines (e.g., "test_module.py ...")
• Failures: Isolates failure sections and error tracebacks
• Summary: Captures the final statistics line

Located at lines 58-132 in src/cmds/python/pytest_cmd.rs, this loop tracks state transitions to separate noise from actionable data.

Summary Generation and Truncation

Once parsing completes, build_pytest_summary (lines 44-121) constructs the final output:

1. parse_summary_line extracts pass/fail/skip counts from the summary line
2. Formats a one-line header: Pytest: X passed, Y failed, Z skipped
3. Lists up to five failures, showing only the test name and relevant error lines
4. Applies utils::truncate to limit each error line to 100 characters

The result eliminates over 90% of original token volume while preserving diagnostic essentials.

How RTK Compresses Vitest Output

For JavaScript testing, src/cmds/js/vitest_cmd.rs implements a three-tier parsing strategy that gracefully handles varying output formats.

JSON and Regex Tiered Parsing

The VitestParser attempts extraction in three stages:

Tier 1 – Full JSON Parse: The parser deserializes stdout into VitestJsonOutput. If package manager prefixes (like pnpm or dotenv) appear before the JSON, extract_json_object isolates the valid JSON block first.

Tier 2 – Regex Fallback: If JSON parsing fails, extract_stats_regex scans for patterns like "Tests X passed, Y failed" along with duration. extract_failures_regex then identifies specific failure lines.

Tier 3 – Passthrough: When both structured methods fail, RTK retains the raw (truncated) output and emits a warning, ensuring the user always receives feedback.

Graceful Degradation

This tiered approach ensures RTK compresses output even when users override the default --reporter=json flag. The regex tier captures statistics from human-readable formats, while the passthrough tier guarantees visibility into unparsable edge cases.

Shared Infrastructure for Output Compression

Both frameworks rely on common utilities located in src/core/ and src/parser/:

• runner::run_filtered (src/core/runner.rs): Executes child processes, streams output, and applies filter callbacks
• utils::truncate (src/core/utils.rs): Shortens error messages exceeding length thresholds
• utils::strip_ansi (src/core/utils.rs): Removes color codes before regex parsing
• OutputParser trait (src/parser/mod.rs or src/parser/output_parser.rs): Defines the three-tier contract (Full, Degraded, Passthrough)
• TestResult and TestFailure (src/parser/test_result.rs): Normalized structures used by all formatters to ensure consistent output presentation

Practical Usage Examples

Execute compressed test runs using RTK's CLI:

# Python - runs pytest with -q/--tb=short automatically

rtk pytest
rtk pytest tests/unit/

# JavaScript - forces JSON output and parses comprehensively

rtk vitest
rtk vitest src/**/*.test.ts

To bypass compression and view full output while still recording the execution:

rtk proxy pytest -vv

Summary

• RTK compresses test output through a four-stage pipeline: command preparation, execution capture, framework-specific parsing, and concise re-formatting.
• pytest uses a state-machine parser (filter_pytest_output) with four states to isolate failures and summaries from noise.
• Vitest employs a three-tier parser (VitestParser) that attempts JSON extraction, falls back to regex, and finally uses passthrough if necessary.
• All implementations share common infrastructure including runner::run_filtered, utils::truncate, and the OutputParser trait.
• Compressed output preserves only statistics and up to five truncated failure excerpts, reducing token volume by over 90%.

Frequently Asked Questions

How does RTK handle frameworks without native JSON output?

RTK implements regex-based extraction tiers that scan human-readable output for statistics patterns. For pytest, the state machine parses plain text progress indicators and summary lines. For Vitest, regex patterns extract "X passed, Y failed" counts when JSON is unavailable. This ensures compression works regardless of the reporting format.

What happens when RTK cannot parse test output?

When parsers fail, RTK enters degraded mode. First, it attempts regex extraction to salvage statistics. If that fails, it falls back to passthrough, displaying the raw (truncated) output with a warning. All raw output remains available in the tee file for debugging, ensuring no data is lost even when compression fails.

Does RTK modify the test command arguments?

Yes, RTK prepends optimization flags automatically. For pytest, it adds -q and --tb=short unless conflicting traceback flags are detected. For Vitest, it injects --reporter=json and run (non-watch mode). These modifications occur in the command builders within src/cmds/python/pytest_cmd.rs and src/cmds/js/vitest_cmd.rs.

Where does RTK store the raw test output before compression?

Raw output is written to a tee file via runner::run_filtered (src/core/runner.rs). This occurs simultaneously with the filtering process, ensuring users can access the complete unprocessed logs for debugging even when the compressed summary is displayed in the terminal.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}