How to Run Tests in Colibri: A Complete Guide Using Make and Python

Run Colibri tests using make test-c for C unit tests, make test-py for Python integration tests, or make check for the full CI validation suite.

The Colibri repository (JustVugg/colibri) ships with a lightweight, cross-platform test suite that validates both the pure-C inference engine and Python helper utilities. Whether you are modifying the core engine or adjusting the coli launcher scripts, the repository's top-level Makefile provides standardized targets to verify correctness on Linux, macOS, and Windows.

Repository Test Structure

Test files are organized under c/tests/ and split into two categories: compiled C unit tests that exercise the inference engine directly, and Python integration tests that validate the launcher, model conversion tools, and OpenAI-compatible server. The Makefile at the repository root orchestrates these through three primary targets: test-c, test-py, and check.

The helper script c/tools/run_tests.py serves as the execution engine for C tests, looping over compiled binaries and reporting non-zero exit statuses as failures. This Python-based wrapper ensures consistent behavior across POSIX and Windows environments without relying on shell-specific loops.

Running C Unit Tests

The C test suite consists of small, focused executables such as test_int3.c and test_fp8_repack.c that validate specific engine components. To compile and execute the entire C test suite, use the dedicated Make target.

First, build the engine for your target model family (e.g., GLM-5.2) to ensure test binaries link against the correct libraries:

make -C c glm

Then run all C tests:

make test-c

This command compiles each test binary and invokes c/tools/run_tests.py to execute them sequentially. Any non-zero exit code from a test binary halts the suite with a failure status.

Debugging Single C Tests

For isolated debugging, invoke run_tests.py directly against a specific compiled binary:

python ./c/tools/run_tests.py ./c/tests/test_fp8_repack

The script exits immediately upon the first failure, making it suitable for integration into custom CI pipelines or local debugging workflows.

Running Python Integration Tests

Python tests verify high-level functionality including the coli launcher, model conversion utilities, and server compatibility layers. These tests require Python 3.8 or newer and the pytest framework.

Execute the Python test suite via Make:

make test-py

This target invokes pytest on the c/tests/ directory, discovering and executing all Python test files. The tests in c/tests/test_fp8_repack.py demonstrate how Python wrappers validate FP8 repacking tools against the compiled engine.

Full CI Validation

For comprehensive validation matching the continuous integration pipeline, run the check target. This performs static analysis, linting, and executes both C and Python test suites in sequence.

make check

Use this command before submitting pull requests to ensure your changes meet repository quality standards across all supported platforms.

Platform-Specific Considerations

Colibri's test infrastructure is designed for portability. The run_tests.py script avoids POSIX-only shell constructs, ensuring that make test-c functions correctly in Windows Command Prompt, PowerShell, and Git Bash. On Unix-like systems, the same targets leverage standard Make and Python environments without additional configuration.

Summary

• Run C unit tests with make test-c after building the engine via make -C c glm
• Run Python tests with make test-py (requires Python ≥3.8 and pytest)
• Run full CI validation with make check to include linting and static analysis
• Debug individual tests using python ./c/tools/run_tests.py <path_to_binary>
• Cross-platform support is handled by c/tools/run_tests.py, eliminating shell dependency on Windows

Frequently Asked Questions

How do I run a single C test binary in Colibri?

Use the run_tests.py helper script directly with the path to the compiled binary. For example: python ./c/tools/run_tests.py ./c/tests/test_fp8_passthrough. The script handles execution and exit status reporting, making it ideal for debugging specific engine components without running the full suite.

What Python version is required for Colibri tests?

The Python integration tests require Python 3.8 or newer. This ensures compatibility with the pytest framework and the type hints used in the launcher (coli) and server components. The make test-py target assumes a valid Python environment with pytest installed.

How does Colibri handle test failures on Windows?

The repository uses c/tools/run_tests.py as a cross-platform test runner instead of shell loops. This allows make test-c to execute correctly in Windows Command Prompt, PowerShell, or Git Bash. The runner exits with a non-zero status on the first failure, consistent with POSIX behavior, ensuring CI pipelines behave identically across operating systems.

What is the difference between make test-c and make check?

make test-c compiles and executes only the C unit tests in c/tests/, validating the inference engine core. make check runs the complete continuous integration suite, which includes building the engine, running C tests, running Python tests via pytest, and performing static analysis and linting. Use check for pre-submission validation and test-c for rapid iteration during engine development.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}