Cocoindex Versioning and Update Strategy: How the Rust-Python Bridge Stays in Sync

Cocoindex uses a Git tag-driven automation pipeline to synchronize version numbers between its Rust core engine and Python client library, with runtime validation that prevents version mismatches at import time.

The cocoindex-io/cocoindex repository maintains a dual-language codebase shipping both a Rust core engine and a Python client library. To prevent drift between these components, the project implements a strict versioning protocol where a single Git tag propagates version identifiers across multiple manifest files and enforces compatibility at runtime.

How Version Propagation Works in Cocoindex

Cocoindex employs a centralized release workflow that treats the Git tag as the single source of truth for all version numbers. When maintainers push a tag like refs/tags/v1.2.3, an automated script distributes that version across the Rust workspace and Python package metadata.

Tag-Driven Release Workflow

The automation lives in .github/scripts/update_version.py, which executes four critical operations when triggered by a release tag:

1. Extract version from Git reference – The extract_version_from_github_ref function parses the GITHUB_REF environment variable to isolate the semantic version string.
2. Update Rust workspace – The update_cargo_version function rewrites the version = "..." field in the root Cargo.toml and rust/core/Cargo.toml.
3. Generate Python version constants – The write_python_version function creates python/cocoindex/_version.py, writing both the PyPI-compatible __version__ and the exact Rust CORE_VERSION.
4. Patch Python project metadata – The update_pyproject_version function replaces the dynamic version placeholder in pyproject.toml with the concrete PEP 440-compliant version.

If any step fails, the CI job terminates immediately, preventing inconsistent releases from reaching PyPI or Cargo.

Runtime Sanity Checks

When users import the cocoindex package, python/cocoindex/_version_check.py executes automatically to verify binary compatibility:

# python/cocoindex/_version_check.py

from ._internal import core as _core
from ._version import CORE_VERSION as _CORE_VERSION

if engine_version is not None and engine_version != _CORE_VERSION:
    problems.append(
        f"Version mismatch: Python package expects core version {_CORE_VERSION!r}, "
        f"but cocoindex._internal.core reports {engine_version!r}."
    )

This check compares the CORE_VERSION constant against the version reported by the compiled Rust extension (cocoindex._engine). If the values diverge, the module raises a RuntimeError with troubleshooting guidance for resolving path conflicts or stale binary artifacts.

Version Conversion Between Rust and Python

Rust and Python use incompatible pre-release naming conventions. Cocoindex bridges this gap through the rust_version_to_pypi function in .github/scripts/update_version.py, which converts Cargo pre-release suffixes to PEP 440 format:

• -alpha.1 becomes a1
• -beta.2 becomes b2
• -rc.1 becomes rc1

The script writes both formats to python/cocoindex/_version.py:

# python/cocoindex/_version.py (auto-generated)

__version__ = "1.3.0a1"          # PyPI-compatible (PEP 440)

CORE_VERSION = "1.3.0-alpha.1"   # Exact Rust version

This ensures that pip understands the version ordering while the runtime check validates exact binary compatibility with the Rust crate.

Practical Implementation Details

Checking Installed Versions

Users can inspect the current version pair at runtime:

import cocoindex

print("Python package version:", cocoindex.__version__)   # e.g., "1.2.3"

print("Embedded Rust core version:", cocoindex._version.CORE_VERSION)  # e.g., "1.2.3"

Debugging Version Mismatches

If the Rust engine and Python package drift, import errors provide actionable diagnostics:

try:
    import cocoindex
except RuntimeError as exc:
    print("Version sanity error:", exc)
    # Output suggests checking for old .pyd files or conflicting sys.path entries

Local Pre-Release Testing

Developers can simulate release tagging locally to verify the version propagation logic:

# Simulate a GitHub release tag locally

export GITHUB_REF=refs/tags/v1.3.0-alpha.1

# Execute the version update script

python .github/scripts/update_version.py

This updates Cargo.toml to version 1.3.0-alpha.1, generates _version.py with __version__ = "1.3.0a1" and CORE_VERSION = "1.3.0-alpha.1", and patches pyproject.toml accordingly. After running maturin develop, the local build contains a synchronized Rust-Python pair ready for testing.

Key Files in the Versioning System

• rust/core/Cargo.toml – Workspace manifest defining the Rust engine version.
• Cargo.toml (root) – Workspace-level configuration updated by the release script.
• python/cocoindex/_version.py – Auto-generated Python file containing __version__ and CORE_VERSION constants.
• python/cocoindex/_version_check.py – Runtime validation module that raises RuntimeError on version mismatch.
• .github/scripts/update_version.py – CI automation script that propagates Git tags across all manifest files.
• pyproject.toml – Python package metadata file receiving the concrete version string during releases.

Summary

• Git tags drive everything – The update_version.py script propagates tag values to Cargo.toml, pyproject.toml, and _version.py.
• Automatic conversion – Pre-release identifiers translate from Cargo syntax (-alpha.1) to PEP 440 (a1) for PyPI compatibility.
• Runtime enforcement – The _version_check.py module guarantees that imported Python packages match their embedded Rust binaries.
• Fail-fast CI – Release workflows halt if any version file update fails, preventing partial or inconsistent releases.

Frequently Asked Questions

How does Cocoindex ensure the Rust engine and Python package versions match?

Cocoindex implements a runtime validation check in python/cocoindex/_version_check.py that executes immediately upon importing the package. This script compares the CORE_VERSION constant (derived from the Git tag) against the version reported by the compiled Rust extension. If the values differ, it raises a RuntimeError before any functionality executes, preventing cryptic errors from incompatible binary interfaces.

What happens if the Cocoindex Rust core version doesn't match the Python client?

Users see a clear RuntimeError message indicating the expected and actual version strings, along with common remediation steps. Typically, version mismatches occur when stale .pyd or .so files persist in the Python path, when multiple Cocoindex installations exist in sys.path, or when developers modify one codebase component without updating the other. The error message directs users to clean their build artifacts and reinstall.

How are pre-release versions like alpha and beta handled in Cocoindex?

The rust_version_to_pypi function in .github/scripts/update_version.py converts Rust pre-release syntax (e.g., -alpha.1, -beta.2, -rc.1) into Python's PEP 440 format (a1, b2, rc1). The script writes the PyPI-compatible version to __version__ while preserving the exact Rust notation in CORE_VERSION. This allows pip to properly sort pre-releases while maintaining exact binary compatibility checks at runtime.

Where is the single source of truth for Cocoindex version numbers?

The Git tag serves as the sole source of truth. When maintainers push a tag formatted as refs/tags/vX.Y.Z, the release workflow triggers update_version.py, which extracts the version and writes it to all relevant manifests including rust/core/Cargo.toml, python/cocoindex/_version.py, and pyproject.toml. This tag-driven approach guarantees that published releases on both Crates.io and PyPI originate from identical version identifiers.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}