How to Contribute to CocoIndex: A Complete Guide for Python and Rust Developers

To contribute to cocoindex.io, fork the repository, set up the hybrid Python/Rust environment using uv and cargo, make your changes in the relevant core files like python/cocoindex/_internal/api.py or rust/ops_text/src/split/recursive.rs, and submit a PR after running the full test suite.

CocoIndex is an open-source data indexing engine maintained at cocoindex-io/cocoindex that combines a Python API with a high-performance Rust core. Whether you are fixing documentation typos or implementing new connectors, understanding the dual-language architecture is essential for a successful contribution. This guide walks you through the complete workflow from environment setup to merged pull request.

Prerequisites and Repository Setup

Before writing code, ensure you have the required tools installed on your system. The project uses uv for Python dependency management and cargo for the Rust toolchain.

1. Fork and clone the repository:

   git clone https://github.com/your-username/cocoindex.git
   cd cocoindex

2. Install uv (if not already present):

   curl -LsSf https://astral.sh/uv/install.sh | sh

3. Review the contributing policy at cocoindex.io/docs/contributing/guide and browse issues labeled good first issue on GitHub to find beginner-friendly tasks.

Setting Up the Development Environment

The CocoIndex codebase requires building the Rust extension and installing Python dependencies in a specific order. All build commands are documented in the repository’s CLAUDE.md file.

Create and activate the virtual environment, then install dependencies:

uv venv
source .venv/bin/activate
uv sync

Build the Rust core and install the Python package (required after any Rust change):

uv run maturin develop

This command compiles the Rust source and makes the native extension available to the Python runtime.

Understanding the Codebase Architecture

CocoIndex is organized into distinct Python and Rust layers. Knowing which layer to modify saves time and ensures your changes align with the project’s design.

Python API Layer

The public Python interface lives in python/cocoindex/_internal/api.py, which contains core entry points like mount, mount_each, and mount_target. When adding new connectors or decorators, follow the private-symbol convention described in CLAUDE.md by prefixing non-public imports with an underscore.

Rust Engine Layer

Performance-critical operations reside in the rust/ directory. For example, text processing logic is implemented in rust/ops_text/src/split/recursive.rs, while PyO3 bridge helpers for error conversion and futures live in rust/py_utils/src/lib.rs. Modify these files when optimizing core algorithms or adding low-level functionality.

Making Your First Contribution

Once your environment is ready, follow this workflow to submit changes:

1. Create a feature branch:

   git checkout -b my-feature

2. Implement your changes in the appropriate layer. For Python connectors, reference existing implementations like python/cocoindex/connectors/postgres/_target.py to understand the pattern for mounting table targets.

3. Run linting and formatting using ruff:

   uv run ruff format .
   uv run ruff check .

4. Add or update tests. New Python functionality requires unit tests in python/tests/, while Rust changes need tests in the corresponding rust/*/src/tests modules.

5. Commit and push:

   git add .
   git commit -m "feat: short description of change"
   git push origin my-feature

6. Open a Pull Request using the "contributions" template. The CI workflow automatically runs the full test matrix including Rust builds, Python tests, and type checks.

Testing and Quality Assurance

Verify your changes do not break existing functionality by running the complete test suite:

Rust tests:

cargo test

Python type-checking:

uv run mypy

Python tests:

uv run pytest python/

All checks must pass before maintainers can merge your PR. If you modify the Rust core, remember to run uv run maturin develop again to rebuild the extension before testing Python changes.

Code Example: Adding a New Connector

Below is a minimal implementation of a "dummy" connector that declares a table target in PostgreSQL. This example reuses existing infrastructure from python/cocoindex/connectors/postgres/_target.py:

# file: python/cocoindex/connectors/dummy/_target.py

from cocoindex import mount
from cocoindex.connectors.postgres._target import mount_table_target

# Public API – expose the mount helper

__all__ = ["mount_dummy_table_target"]

async def mount_dummy_table_target(env, *, table_name: str):
    """
    Example connector that creates a dummy PostgreSQL table.
    """
    # Re‑use the existing Postgres target builder

    return await mount_table_target(env, table_name=table_name)

Key implementation details demonstrated here include using async functions for I/O-bound operations, reusing existing target-mount helpers like mount_table_target, and explicitly exporting public functions via __all__ to maintain clean namespace boundaries.

Summary

• Fork and clone the cocoindex-io/cocoindex repository, then install uv and cargo to manage the hybrid Python/Rust environment.
• Build the project using uv sync followed by uv run maturin develop to compile the Rust extension.
• Modify the correct layer: Python API changes belong in python/cocoindex/_internal/api.py, while engine optimizations target files like rust/ops_text/src/split/recursive.rs.
• Maintain code quality by running ruff for formatting, mypy for type checking, and the full pytest and cargo test suites before submitting.
• Submit PRs through the "contributions" template and ensure CI passes for both language runtimes.

Frequently Asked Questions

What programming languages do I need to know to contribute to cocoindex.io?

You should be comfortable with Python for API and connector development and Rust for core engine contributions. Many tasks, such as adding new data connectors, require only Python knowledge, while performance optimizations or text processing improvements require Rust expertise.

How do I rebuild the Rust core after making changes?

Run uv run maturin develop from the repository root. This command recompiles the Rust source code and reinstalls the Python extension module, making your changes available to the Python runtime for testing.

Where should I add tests for my contribution?

Add Python unit tests in the python/tests/ directory, following the existing test structure. For Rust changes, add tests within the corresponding rust/*/src/tests modules or as inline #[cfg(test)] blocks in the source files.

What is the difference between the Python API and Rust engine in CocoIndex?

The Python API in python/cocoindex/_internal/api.py provides the user-facing interface for defining flows, mounts, and connectors. The Rust engine handles the actual execution, parallel processing, and low-level operations like text splitting in rust/ops_text/src/split/recursive.rs, offering performance-critical functionality exposed to Python via PyO3 bindings.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}