How to Run Tests in the Marin Project: A Complete Guide

To run tests in the Marin project, use uv run pytest for individual files or uv run --no-project infra/ci/run_tests.py for the full safe suite that automatically excludes slow, integration, and cluster-dependent tests.

The Marin project (marin-community/marin) employs a layered testing strategy spanning unit tests to full-cluster end-to-end (E2E) validations. Understanding how to properly invoke these tests ensures you can validate code changes without accidentally triggering resource-intensive or infrastructure-dependent suites.

Install the Development Toolchain

Marin uses uv as its dependency manager and task runner. Before executing any tests, synchronize your environment from the repository root:

uv sync

This command installs all runtime dependencies and test utilities specified in the project configuration. While uv typically provides pytest automatically, you can verify the installation with uv tool install pytest if needed.

Run Individual Tests or Test Files

For rapid iteration during development, execute specific test files or pattern-matched tests using the uv run wrapper. This ensures the proper isolated environment is activated:

uv run pytest path/to/test_file.py
uv run pytest -k "my_feature"

These commands appear in the repository-wide TESTING.md (lines 11–16) and the Iris-specific guide at lib/iris/TESTING.md (lines 98–101). Running individual files bypasses the default exclusions, so ensure your selected tests do not require external services unless explicitly intended.

Execute the Full Safe Test Suite

The recommended entry point for comprehensive validation is the CI helper script. This script is referenced in TESTING.md as the standard way to run the safe test suite:

uv run --no-project infra/ci/run_tests.py

This command in infra/ci/run_tests.py performs three critical functions:

1. Discovers all test files affected by the current branch
2. Applies default marker exclusions for slow, integration, cluster, Docker, and manual tests
3. Enforces the 60‑second per‑test timeout defined in TESTING.md (lines 25–29)

Using this script prevents accidental execution of GPU‑heavy workloads or tests requiring live infrastructure.

Understanding Test Markers and Filters

Marin annotates tests with pytest markers to control execution context. These markers are defined throughout the codebase and documented in TESTING.md.

Available Markers

• slow: Long‑running or GPU‑heavy computations
• integration: Requires external services (e.g., GCS, Weights & Biases)
• requires_cluster: Needs a live Iris/Zephyr cluster
• docker: Requires Docker daemon access
• manual: Explicit developer‑run tests not suitable for CI

To include specific marker categories, run:

uv run pytest -m requires_cluster
uv run pytest -m "requires_cluster and not docker"

Critical Warning About Marker Overrides

Do not override the default marker expression with -m "not slow". According to TESTING.md (line 21), this replaces the built‑in safety filter and risks selecting live‑cluster tests that require infrastructure credentials. Always use the infra/ci/run_tests.py script for default exclusions, or explicitly include required markers rather than excluding unwanted ones.

Module-Specific Testing (Iris Example)

The Iris module (lib/iris/) maintains its own testing conventions documented in lib/iris/TESTING.md. When working within specific modules, use package‑scoped commands:

# Run all Iris unit tests

uv run --package marin-iris --group test pytest lib/iris/tests/

# Run E2E smoke tests requiring cluster access

uv run pytest lib/iris/tests/e2e/test_smoke.py -m requires_cluster -o "addopts="

# Run E2E tests without Docker dependencies

uv run pytest lib/iris/tests/e2e/ -m "requires_cluster and not docker" -o "addopts="

These patterns (lines 99–130 in lib/iris/TESTING.md) demonstrate the typical invocation: uv run pytest <path> -m <marker> -o "addopts=".

Handle Test Timeouts

Every test inherits a 60‑second timeout enforced by the test runner (TESTING.md, line 25). For legitimately long‑running operations, annotate the test function with an explicit timeout marker:

import pytest

@pytest.mark.timeout(180)
def test_long_running():
    # Heavy computation requiring 3 minutes

    pass

This overrides the default limit for that specific test only.

Best Practices and Common Pitfalls

When contributing to marin-community/marin, follow these guidelines:

• Never rely on print statements for test output; use assert statements or pytest.raises() for validation
• Use descriptive test names following the pattern test_<subject>_<scenario>_<expected>, as enforced in the Iris testing guide (lines 88–95)
• Prefer fakes over mocks when testing external service dependencies (e.g., InMemoryGcpService instead of full mocks), as recommended in the root guidelines (lines 106–114)
• Do not commit tests that override default timeouts without justification

Summary

• Run uv sync once to install the testing toolchain
• Execute individual tests with uv run pytest path/to/file.py
• Use uv run --no-project infra/ci/run_tests.py for the full safe suite with automatic exclusions
• Avoid -m "not slow" to prevent bypassing safety filters
• Apply @pytest.mark.timeout() to extend the 60‑second default for specific tests
• Reference TESTING.md for repository‑wide rules and lib/iris/TESTING.md for module‑specific conventions

Frequently Asked Questions

What is the difference between pytest and infra/ci/run_tests.py?

Running uv run pytest directly executes all discovered tests in the current directory, respecting only the configuration in pyproject.toml or pytest.ini. The infra/ci/run_tests.py script adds branch‑aware discovery, applies critical marker exclusions (slow, integration, docker, cluster), and enforces the 60‑second timeout. Use the script for CI‑equivalent validation and direct pytest calls for targeted development debugging.

How do I run tests that require a live cluster?

Execute tests marked with requires_cluster explicitly: uv run pytest -m requires_cluster. Ensure you have active credentials for the Iris or Zephyr cluster and understand that these tests interact with real infrastructure. The Iris E2E examples in lib/iris/TESTING.md show the proper pattern: uv run pytest lib/iris/tests/e2e/ -m requires_cluster -o "addopts=".

Why should I avoid using -m "not slow" when running tests?

The -m flag replaces pytest's default marker expression rather than appending to it. According to TESTING.md (line 21), the project configures default exclusions internally; overriding them with -m "not slow" risks including cluster‑dependent or integration tests that require external services, causing failures or security risks in local environments.

How do I increase the timeout for a specific test?

Import pytest and apply the @pytest.mark.timeout(seconds) decorator to the test function. The default timeout is 60 seconds as defined in the global configuration. For example, @pytest.mark.timeout(180) allows three minutes for that specific test case. This is preferred over modifying global timeout settings.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}