how-to-guide

CocoIndex Connector Configuration: Practical Examples for LocalFS, PostgreSQL, and SQLite

May 5, 2026 cocoindex-io/cocoindex ↗

CocoIndex connectors use a two-level handler architecture where register_root_target_states_provider registers a root handler that reconciles external systems, exposes thin wrapper functions like declare_dir_target or declare_table_target, and automatically manages schema creation, updates, and deletions.

CocoIndex treats external systems—filesystems, relational databases, and vector stores—as target states that your pipeline reconciles automatically. Understanding cocoindex connector configuration requires knowing how each connector registers handlers, manages schemas, and exposes high-level APIs. This guide walks through production-ready configuration examples using the LocalFS, PostgreSQL, and SQLite connectors as implemented in the cocoindex-io/cocoindex repository.

Local Filesystem Connector Configuration

The LocalFS connector manages directory structures and file contents on disk through a hierarchical handler pattern.

Core Architecture and Registration

At the bottom of cocoindex/connectors/localfs/_target.py, the module registers a root provider that manages top-level directories:

_root_provider = coco.register_root_target_states_provider(
    "cocoindex/localfs", _RootHandler()
)  # Lines 72-74

The _RootHandler reconciles root-level entries and yields child DirTarget instances, each owning an _EntryHandler that tracks file fingerprints via _EntryTrackingRecord and executes _EntryAction objects (write, delete, create-parents).

Public API Methods

The connector exposes three primary configuration functions in cocoindex/connectors/localfs/__init__.py:

declare_dir_target(path, create_parent_dirs=True) → DirTarget: Returns a DirTarget instance for declaring nested files and subdirectories.
declare_file(path, content, create_parent_dirs=False) → None: One-off helper for writing a single file without managing a directory context.
mount_dir_target(path, create_parent_dirs=True) → DirTarget: Async convenience that mounts the target and returns a ready-to-use DirTarget.

Configuration Example: Static Site Generation

Declare a directory target and render Markdown files to HTML:

import pathlib
import cocoindex as coco
from cocoindex.connectors import localfs
from cocoindex.flow import PatternFilePathMatcher

@coco.fn
async def render_page(src: coco.FilePath, out: localfs.DirTarget) -> None:
    html = f"<html><body>{await src.read_text()}</body></html>"
    out.declare_file(f"{src.file_path.stem}.html", html)

async def build_site():
    out_dir = await coco.use_mount(
        localfs.declare_dir_target, pathlib.Path("./site_out")
    )
    files = localfs.walk_dir(
        pathlib.Path("./markdown"),
        path_matcher=PatternFilePathMatcher(included_patterns=["**/*.md"]),
    )
    await coco.mount_each(render_page, files.items(), out_dir)

coco.App(
    coco.AppConfig(name="StaticSite"),
    build_site,
).update_blocking(report_to_stdout=True)

The DirTarget.declare_file method (lines 300-326 in _target.py) handles content hashing and parent directory creation atomically.

PostgreSQL Connector Configuration

The PostgreSQL connector implements a two-level model separating table schema management from row-level operations.

Two-Level Handler Architecture

In cocoindex/connectors/postgres/_target.py, the connector registers a table-level provider:

_table_provider = coco.register_root_target_states_provider(
    "cocoindex/postgres/table", _TableHandler()
)  # Lines 335-337

The _TableHandler manages table creation and schema migrations, yielding _RowHandler instances that perform upserts and deletes. Table-level actions (_TableAction) track primary-key signatures and column definitions, while row-level actions (_RowAction) track row fingerprints for idempotency.

Schema Definition and Mapping

Use TableSchema.from_class to infer column types from dataclasses, with optional overrides via PgType annotations:

schema = await postgres.TableSchema.from_class(
    User,
    primary_key=["id"],
    column_overrides={"age": postgres.PgType("smallint")}
)  # Lines 90-120

Configuration Example: Dataclass Synchronization

Sync a list of dataclasses into a managed Postgres table:

import cocoindex as coco
from cocoindex.connectors import postgres
from dataclasses import dataclass

@dataclass
class User:
    id: int
    name: str
    age: int

async def main():
    schema = await postgres.TableSchema.from_class(
        User, primary_key=["id"]
    )
    pool = await postgres.create_pool(dsn="postgresql://user:pw@localhost/db")
    
    tbl = await coco.use_mount(
        postgres.declare_table_target,
        pool,
        "users",
        schema,
    )
    
    for user in [User(1, "Alice", 30), User(2, "Bob", 25)]:
        tbl.declare_row(row=user)  # Lines 664-676

coco.App(
    coco.AppConfig(name="UserSync"),
    main,
).update_blocking(report_to_stdout=True)

SQLite Connector Configuration

The SQLite connector extends the two-level pattern with thread-safe connection management and native vector extension support.

Connection Management and Vector Extensions

The connect() function returns a ManagedConnection wrapping sqlite3.Connection with an RWLock for thread safety and automatic extension loading:

conn = sqlite.connect(":memory:", load_vec=True)  # Lines 271-279

In cocoindex/connectors/sqlite/_target.py, the root registration follows the standard pattern:

_table_provider = coco.register_root_target_states_provider(
    "cocoindex/sqlite/table", _TableHandler()
)  # Lines 106-108

Virtual Table Support

For vector storage, define a Vec0TableDef to configure sqlite-vec virtual tables with partition keys and auxiliary columns. The connector automatically maps NumPy ndarray columns to float[N] types when the vector extension is loaded (see _get_type_mapping, lines 11-24).

Configuration Example: Vector Storage with vec0

Store embeddings in a managed SQLite virtual table:

import numpy as np
import cocoindex as coco
from cocoindex.connectors import sqlite
from dataclasses import dataclass

@dataclass
class Document:
    id: int
    text: str
    embedding: np.ndarray

async def main():
    schema = await sqlite.TableSchema.from_class(Document, primary_key=["id"])
    
    vec0_def = sqlite.Vec0TableDef(
        partition_key_columns=["year"],
        auxiliary_columns=["metadata"]
    )
    
    conn = sqlite.connect("./vectors.db", load_vec=True)
    
    tbl = await coco.use_mount(
        sqlite.declare_table_target,
        conn,
        "documents",
        schema,
        virtual_table_def=vec0_def,
    )
    
    for doc in [
        Document(1, "hello", np.random.rand(384).astype(np.float32)),
        Document(2, "world", np.random.rand(384).astype(np.float32)),
    ]:
        tbl.declare_row(row=doc)  # Lines 635-648

coco.App(
    coco.AppConfig(name="EmbeddingStore"),
    main,
).update_blocking(report_to_stdout=True)

Universal Connector Architecture

All CocoIndex connectors follow a consistent five-step configuration pattern:

Register root provider: Call coco.register_root_target_states_provider with a handler implementing the reconcile method.
Implement TargetHandler: Create handlers that compute diffs between desired and actual state.
Provide child handlers: Return coco.ChildTargetDef wrappers for hierarchical resources (tables → rows, directories → files).
Expose thin wrappers: Export declare_* and mount_* functions in cocoindex/connectors/<name>/__init__.py.
Use in pipeline: Invoke coco.use_mount inside @coco.fn decorated functions to obtain target wrappers and declare objects.

This uniformity allows swapping connectors with minimal code changes—replace the import and target type while keeping the declaration logic intact.

Quick Reference: All CocoIndex Connectors

Connector	Target Class	Key Configuration Functions
LocalFS	`DirTarget`	`declare_dir_target`, `declare_file`, `mount_dir_target`
PostgreSQL	`TableTarget[RowT]`	`declare_table_target`, `mount_table_target`, `create_pool`
SQLite	`TableTarget[RowT]`	`declare_table_target`, `mount_table_target`, `connect`
Qdrant	`CollectionTarget`	`declare_collection_target`, `mount_collection_target`
Kafka	`TopicTarget`	`declare_topic_target`, `mount_topic_target`
Neo4j	`GraphTarget`	`declare_graph_target`, `mount_graph_target`
LanceDB	`DatasetTarget`	`declare_dataset_target`
Amazon S3	`BucketTarget`	`declare_bucket_target` (source)
Google Drive	`DriveFolderTarget`	`declare_folder_target` (source)

All connectors reside under python/cocoindex/connectors/ and implement the registration and wrapper conventions described above.

Summary

CocoIndex connector configuration relies on registering a root target-state provider that reconciles external systems declaratively.
LocalFS uses DirTarget and declare_dir_target to manage file hierarchies with automatic content fingerprinting.
PostgreSQL and SQLite use TableTarget with TableSchema.from_class to map dataclasses to tables, supporting primary keys, column overrides, and vector extensions.
Universal pattern: Register provider → Implement handlers → Expose wrappers → Mount in pipeline → Declare objects.
All connectors support both synchronous declare_* functions and asynchronous mount_* variants for use with coco.use_mount.

Frequently Asked Questions

How do I choose between `declare_table_target` and `mount_table_target`?

Use declare_table_target inside standard Python functions when you need a synchronous interface for configuration setup. Use mount_table_target (or coco.use_mount with declare_table_target) inside async @coco.fn decorated pipeline functions when CocoIndex should manage the target lifecycle, compute diffs, and reconcile state automatically. The mount variants integrate with CocoIndex's action engine for idempotent updates.

Can I configure custom PostgreSQL column types when using `TableSchema.from_class`?

Yes, pass the column_overrides dictionary mapping field names to postgres.PgType instances. For example, override Python int to PostgreSQL smallint or jsonb for complex objects. The PgType annotation accepts any valid PostgreSQL type string, applied during table creation in _TableHandler (lines 77-100 in postgres/_target.py).

How does the SQLite connector handle vector embeddings?

The connector auto-detects NumPy ndarray fields and maps them to float[N] columns when load_vec=True is passed to sqlite.connect(). For advanced vector similarity search, pass a Vec0TableDef to declare_table_target to create sqlite-vec virtual tables with partition keys and auxiliary metadata columns, enabling efficient approximate nearest neighbor queries alongside standard relational data.

What is the difference between a target handler and a child handler in CocoIndex?

The root target handler manages top-level resources (tables, directories) and decides whether to create, alter, or drop them. Child handlers manage contained objects (rows, files) and are instantiated via coco.ChildTargetDef returned by the parent's reconcile method. This two-level architecture allows CocoIndex to batch operations efficiently—schema changes happen once at the table level, while row-level changes occur per record during the reconciliation phase.

Have a question about this repo?

These articles cover the highlights, but your codebase questions are specific. Give your agent direct access to the source. Share this with your agent to get started:

Share the following with your agent to get started:

curl -s "https://instagit.com/install.md"

Add to your MCP client configuration:

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}

Ask your agent:

"Use Instagit MCP to understand how cocoindex-io/cocoindex works."

Works with

Claude Codex Cursor VS Code OpenClaw Any MCP Client

Maintain an open-source project? Get it listed too →