# How to Define Data Schemas for CocoIndex: A Complete Guide to Vector and Table Structures

> Define CocoIndex data schemas for vector and table structures. Declare VectorSchema, implement VectorSchemaProvider, and compose into TableSchema or CollectionSchema for SQLite or Qdrant.

- Repository: [CocoIndex/cocoindex](https://github.com/cocoindex-io/cocoindex)
- Tags: how-to-guide
- Published: 2026-05-05

---

**Define data schemas for CocoIndex by declaring `VectorSchema` objects for embeddings, implementing the `VectorSchemaProvider` protocol for dynamic dimensions, and composing them into `TableSchema` or `CollectionSchema` objects that you mount to connectors like SQLite or Qdrant.**

CocoIndex separates **schema declaration** from data-processing logic, allowing you to pre-define how external storage structures should look before any data flows. In the cocoindex-io/cocoindex repository, schemas tell the engine the exact shape of your data—including embedding dimensions, nullability constraints, and primary keys—so it can automatically create and reconcile tables and vector collections. Understanding how to define data schemas for CocoIndex is essential for building declarative data pipelines that sync seamlessly with downstream storage systems.

## Core Schema Building Blocks

### VectorSchema and MultiVectorSchema

The foundation of any embedding-related schema starts in [`python/cocoindex/resources/schema.py`](https://github.com/cocoindex-io/cocoindex/blob/main/python/cocoindex/resources/schema.py). The **`VectorSchema`** class (lines 23-28) defines the data type and dimensionality of a single embedding vector using `np.dtype` and a size parameter. For advanced use cases, **`MultiVectorSchema`** (lines 49-55) wraps a `VectorSchema` with additional multi-vector metadata.

```python
import numpy as np
from cocoindex.resources import schema as cs

# Define a 384-dimensional float32 embedding column

embed_schema = cs.VectorSchema(dtype=np.dtype(np.float32), size=384)

```

### VectorSchemaProvider Protocol

For dynamic schema resolution at runtime, implement the **`VectorSchemaProvider`** protocol defined at lines 17-21 in [`python/cocoindex/resources/schema.py`](https://github.com/cocoindex-io/cocoindex/blob/main/python/cocoindex/resources/schema.py). This protocol requires a `__coco_vector_schema__` method that returns a `VectorSchema` object. According to the cocoindex source code, embedding operators like `SentenceTransformerEmbedder` in [`python/cocoindex/ops/sentence_transformers.py`](https://github.com/cocoindex-io/cocoindex/blob/main/python/cocoindex/ops/sentence_transformers.py) implement this protocol to automatically expose their output dimensions.

```python
from cocoindex.ops import sentence_transformers as st

class MyEmbedder(st.SentenceTransformerEmbedder):
    """Wraps a SentenceTransformer model and provides its VectorSchema."""
    ...
    

# The engine calls `await embedder.__coco_vector_schema__()` at runtime

# to resolve dimensions dynamically.

```

### Column and Table Definitions

For relational databases, **`ColumnDef`** in [`python/cocoindex/connectors/sqlite/_target.py`](https://github.com/cocoindex-io/cocoindex/blob/main/python/cocoindex/connectors/sqlite/_target.py) (lines 68-77) describes individual columns including SQL type, nullability, optional encoder, and whether the column stores vectors. The **`TableSchema`** class (lines 83-94) aggregates these into a full table definition with primary keys and optional row-type metadata.

PostgreSQL connectors follow an identical pattern in [`python/cocoindex/connectors/postgres/_target.py`](https://github.com/cocoindex-io/cocoindex/blob/main/python/cocoindex/connectors/postgres/_target.py), ensuring consistent schema definition across SQL backends.

### Collection Schemas for Vector Databases

When targeting Qdrant, use **`CollectionSchema`** defined in [`python/cocoindex/connectors/qdrant/_target.py`](https://github.com/cocoindex-io/cocoindex/blob/main/python/cocoindex/connectors/qdrant/_target.py) (lines 8-33). This bundles one or more **`QdrantVectorDef`** objects that specify both the vector schema and distance metric (cosine, Euclidean, etc.).

```python
from cocoindex.connectors.qdrant import _target as qdrant_target
from cocoindex.resources import schema as cs
import numpy as np

collection_schema = await qdrant_target.CollectionSchema.create(
    vectors={
        "embedding": qdrant_target.QdrantVectorDef(
            schema=cs.VectorSchema(dtype=np.dtype(np.float32), size=384),
            distance="cosine",
        )
    }
)

```

## The Schema Declaration Workflow

CocoIndex follows a four-phase pattern for schema handling. As implemented in the cocoindex source code, the workflow proceeds as follows:

1. **Declare vector shapes** – Instantiate `VectorSchema` directly or create a class implementing `VectorSchemaProvider`.
2. **Plug into column definitions** – Mark columns with `is_vector=True` and let the engine pull the `VectorSchema` via `get_vector_schema`.
3. **Build table/collection schemas** – Pass dictionaries of `ColumnDef` or `QdrantVectorDef` objects, or use auto-inspection helpers like `TableSchema.from_class` and `CollectionSchema.create`.
4. **Mount the schema** – Pass the schema object to connector-specific `declare_*_target` functions, which sync the external store and reconcile future changes.

## Practical Implementation Examples

### Declaring a Simple Vector Schema

For fixed-dimension embeddings, instantiate `VectorSchema` directly with the appropriate NumPy dtype and size.

```python
import numpy as np
from cocoindex.resources import schema as cs

# 384-dimensional float32 embeddings

vector_schema = cs.VectorSchema(dtype=np.dtype(np.float32), size=384)

```

### Building a SQLite Table Schema with Vector Columns

Use `TableSchema.from_class` to auto-inspect a Python dataclass, then override specific columns with your vector schema. In [`python/cocoindex/connectors/sqlite/_target.py`](https://github.com/cocoindex-io/cocoindex/blob/main/python/cocoindex/connectors/sqlite/_target.py), the `from_class` method accepts a `column_overrides` dictionary to specify vector columns.

```python
from dataclasses import dataclass
import numpy as np
from cocoindex.resources import schema as cs
from cocoindex.connectors.sqlite import _target as sqlite_target

@dataclass
class Document:
    id: int
    title: str
    content: str
    embedding: np.ndarray  # vector column

# Override the inferred type for `embedding`

vector_schema = cs.VectorSchema(dtype=np.dtype(np.float32), size=384)

table_schema = await sqlite_target.TableSchema.from_class(
    record_type=Document,
    primary_key=["id"],
    column_overrides={"embedding": vector_schema},
)

```

### Creating a Qdrant Collection Schema

For vector databases, define named vector fields with specific distance metrics using the Qdrant connector.

```python
from cocoindex.connectors.qdrant import _target as qdrant_target
from cocoindex.resources import schema as cs
import numpy as np

collection_schema = await qdrant_target.CollectionSchema.create(
    vectors={
        "embedding": qdrant_target.QdrantVectorDef(
            schema=cs.VectorSchema(dtype=np.dtype(np.float32), size=384),
            distance="cosine",
        ),
        "sparse_embedding": qdrant_target.QdrantVectorDef(
            schema=cs.VectorSchema(dtype=np.dtype(np.float32), size=768),
            distance="dot",
        )
    }
)

```

### Mounting and Reconciling Schemas

Once defined, mount the schema using `coco.use_mount` with the appropriate connector function. The target schema persists as declarative state, and subsequent data operations reconcile against it.

```python
import pathlib
from cocoindex import coco
from cocoindex.connectors.sqlite import declare_table_target

# Mount the schema from the previous example

db_path = pathlib.Path("./my.db")
target = await coco.use_mount(
    declare_table_target,
    db_path,
    table_schema,            # The TableSchema we built

    table_name="documents",
)

```

## Summary

- **`VectorSchema`** in [`python/cocoindex/resources/schema.py`](https://github.com/cocoindex-io/cocoindex/blob/main/python/cocoindex/resources/schema.py) defines embedding dimensions and data types using NumPy dtypes.
- Implement **`VectorSchemaProvider`** when you need runtime schema resolution from models like Sentence Transformers or LiteLLM.
- Use **`TableSchema.from_class`** for SQL connectors and **`CollectionSchema.create`** for Qdrant to auto-inspect Python record types.
- Override auto-inferred types using `column_overrides` or explicit vector definitions when mixing standard and vector columns.
- Mount schemas using connector-specific functions like `declare_table_target` to enable automatic table creation and schema reconciliation.

## Frequently Asked Questions

### What is the difference between VectorSchema and VectorSchemaProvider?

**`VectorSchema`** is a static struct that hardcodes the dtype and dimensionality of a vector. **`VectorSchemaProvider`** is a protocol that dynamic components like embedding models implement to expose their output dimensions at runtime. When you define data schemas for CocoIndex, use `VectorSchema` for fixed embeddings and `VectorSchemaProvider` when the model determines the shape.

### How do I auto-generate schemas from Python dataclasses?

Use the **`TableSchema.from_class`** method (available in [`sqlite/_target.py`](https://github.com/cocoindex-io/cocoindex/blob/main/sqlite/_target.py) and [`postgres/_target.py`](https://github.com/cocoindex-io/cocoindex/blob/main/postgres/_target.py)) or **`CollectionSchema.create`** for Qdrant. These class methods inspect your dataclass, NamedTuple, or Pydantic model fields and generate appropriate column definitions. Pass `column_overrides` to specify vector schemas for fields that should store embeddings.

### Can I define multiple vector fields in a single collection?

Yes. When creating a **`CollectionSchema`** for Qdrant, provide a dictionary mapping field names to **`QdrantVectorDef`** objects. Each definition can specify different dimensions, data types, and distance metrics, allowing you to store dense and sparse vectors in the same collection.

### Where are the core schema definitions located in the codebase?

The base schema types reside in [`python/cocoindex/resources/schema.py`](https://github.com/cocoindex-io/cocoindex/blob/main/python/cocoindex/resources/schema.py), including `VectorSchema`, `MultiVectorSchema`, and the `VectorSchemaProvider` protocol. Connector-specific implementations live in [`python/cocoindex/connectors/sqlite/_target.py`](https://github.com/cocoindex-io/cocoindex/blob/main/python/cocoindex/connectors/sqlite/_target.py) for SQLite tables, [`python/cocoindex/connectors/postgres/_target.py`](https://github.com/cocoindex-io/cocoindex/blob/main/python/cocoindex/connectors/postgres/_target.py) for PostgreSQL, and [`python/cocoindex/connectors/qdrant/_target.py`](https://github.com/cocoindex-io/cocoindex/blob/main/python/cocoindex/connectors/qdrant/_target.py) for Qdrant collections.