How to Implement Semantic Search Using FTS5 in agent-knowledge

agent-knowledge enables semantic search by combining SQLite FTS5 with vector embeddings, storing LLM-generated vectors in a BLOB column and querying them via the vector extension's similarity functions alongside traditional BM25 scoring.

The agent-knowledge system, as implemented in the anthropics/claude-plugins-community repository, extends SQLite's Full-Text Search capabilities with semantic vector retrieval. This architecture allows Claude plugins to retrieve documents based on meaning rather than exact keyword matches, using a hybrid approach that ranks results by both textual relevance and embedding similarity.

Core Architecture: FTS5 and Vector Embeddings

The Extended Virtual Table Schema

At the foundation of the implementation lies a specialized FTS5 virtual table defined in src/schema.sql. Unlike standard FTS5 tables that only index text, this schema includes a binary column for vector storage:

CREATE VIRTUAL TABLE knowledge_fts USING fts5(
  content, 
  embed, 
  tokenize="porter"
);

The embed column stores BLOB data representing floating-point vectors generated by embedding models. This design allows the same virtual table to support both full-text queries and vector similarity searches. According to the source code, SQLite must be compiled with the vector extension and loaded at runtime using sqlite3_load_extension('vector0') to enable functions like vector_cosine and vector_l2.

Embedding Provider Abstraction

The embedding generation logic is decoupled from storage in src/embeddings.ts. The system supports any OpenAI-compatible text-embedding model, defaulting to text-embedding-ada-002. The OpenAIEmbeddingProvider class standardizes the interface, ensuring that documents and queries are encoded into the same high-dimensional space before insertion or retrieval.

Step-by-Step Implementation

Initialize the Knowledge Base

First, instantiate the AgentKnowledge class with a database path and embedding provider. This configuration establishes the connection to SQLite and prepares the embedding pipeline:

import { AgentKnowledge } from 'agent-knowledge';
import { OpenAIEmbeddingProvider } from 'agent-knowledge/embeddings';

const knowledge = new AgentKnowledge({
  dbPath: './knowledge.db',
  embeddingProvider: new OpenAIEmbeddingProvider({
    apiKey: process.env.OPENAI_API_KEY,
    model: 'text-embedding-ada-002',
  }),
});

Configure the FTS5 Index

Before performing semantic searches, initialize the FTSS (FTS5 + vector-search) index. The initFTS() method in src/search.ts handles the creation of the virtual table and loads the required vector extension:

await knowledge.initFTS();   // creates the virtual table & loads vector extension

This setup step is required only once per database, as it defines the schema and registers the SQLite vector extension that powers similarity calculations.

Perform Semantic Queries

Execute semantic searches by specifying the search type and optional hybrid weighting. The search() method encodes the query, computes similarity scores, and returns ranked results:

const results = await knowledge.search(
  'How do I configure OAuth in Claude Code?',
  {
    type: 'semantic',   // triggers FTSS path
    topK: 5,
    hybridWeight: 0.3,  // 30% vector, 70% BM25
  },
);

for (const hit of results) {
  console.log(`
    Document: ${hit.path}
    Score:    ${hit.score.toFixed(3)}
    Snippet:  ${hit.snippet}
  `);
}

Hybrid Search and Ranking

Combining BM25 with Cosine Similarity

The implementation in src/search.ts supports hybrid scoring that merges traditional keyword relevance with semantic similarity. The final ranking score uses a weighted sum:

score = 0.7 * bm25 + 0.3 * cosine_similarity

You can adjust the balance via the hybridWeight parameter (0.0 to 1.0), where higher values prioritize vector similarity over keyword matching. This approach ensures that exact keyword matches remain relevant while capturing conceptually related documents that might not share specific terminology.

Handling Result Decay

Each document carries a decay factor based on its age, which multiplies into the final score. This mechanism ensures that newer information ranks higher than stale data, even if both match the query semantically. The decay calculation is applied during the ranking phase in src/search.ts.

Automatic Index Updates

When adding new documents, the system automatically generates embeddings and updates the FTSS index in a single transaction. The addDocument() method handles both text insertion and vector computation:

await knowledge.addDocument({
  path: 'docs/oauth.md',
  content: await readFile('docs/oauth.md', 'utf8'),
});

This transactional approach ensures consistency between the plain-text content and its vector representation, eliminating the need for manual reindexing when the knowledge base grows.

Low-Level SQLite Vector Queries

For advanced use cases, you can query the FTSS index directly using raw SQL. This approach provides full control over similarity metrics and ranking algorithms:

import sqlite3 from 'sqlite3';
const db = new sqlite3.Database('./knowledge.db');

const sql = `
  SELECT rowid, content, bm25(knowledge_fts) AS bm25,
         vector_cosine(embedding, ?) AS sim
  FROM knowledge_fts
  ORDER BY (0.7 * bm25) + (0.3 * sim) DESC
  LIMIT 5;
`;
const queryVector = await knowledge.embeddingProvider.encode('OAuth setup');
db.all(sql, queryVector, (err, rows) => {
  console.table(rows);
});

This query demonstrates the direct use of vector_cosine() against the embedded vectors, reproducing the hybrid scoring logic at the SQL level.

Summary

• FTS5 virtual tables in agent-knowledge store both text content and vector embeddings in a unified schema (src/schema.sql).
• Vector extension loading is required via sqlite3_load_extension('vector0') to enable similarity functions.
• Hybrid scoring combines BM25 keyword relevance with cosine similarity, configurable through the hybridWeight parameter.
• Automatic indexing occurs through addDocument(), which generates embeddings and updates the virtual table transactionally.
• OpenAI-compatible providers are supported via the abstraction layer in src/embeddings.ts, with text-embedding-ada-002 as the default.

Frequently Asked Questions

What SQLite extensions are required to enable semantic search in agent-knowledge?

SQLite must be compiled with the vector extension (vector0) and loaded at runtime using sqlite3_load_extension('vector0'). This extension provides the vector_cosine and vector_l2 functions necessary for computing similarity between stored embeddings and query vectors.

How does agent-knowledge handle updates to the semantic index when documents change?

The system updates the index automatically through the addDocument() method. When a new document is added, agent-knowledge generates its embedding via the configured provider and inserts both the text and vector into the FTS5 virtual table within a single SQLite transaction, ensuring atomic updates.

Can I use a different embedding model with agent-knowledge?

Yes, any OpenAI-compatible text-embedding model can be used by implementing the provider interface in src/embeddings.ts. While the default configuration uses text-embedding-ada-002, you can substitute alternative models by configuring the embeddingProvider parameter during AgentKnowledge initialization.

How do I tune the balance between keyword matching and semantic similarity?

Adjust the hybridWeight parameter in the search options, which accepts values between 0.0 and 1.0. A value of 0.3 assigns 30% weight to vector similarity and 70% to BM25 keyword scoring, while higher values prioritize semantic meaning over exact keyword matches.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}