Understanding Hister Architecture Components: A Deep Dive into the Self-Hosted Search Engine

Hister's architecture consists of a modular Go-based system comprising a CLI entry point, Cobra command layer, HTTP server with MCP integration, SQLite persistence layer, full-text search engine, optional semantic search module, and dual web/terminal interfaces.

The asciimoo/hister repository implements a privacy-first, self-hosted search engine designed for local document indexing and retrieval. Understanding the core components of Hister's architecture reveals how the system maintains a single source of truth while supporting multiple access patterns—from command-line operations to web browsers and AI assistants via the Message Control Protocol (MCP).

CLI Entry Point and Command Layer

The application bootstrap process begins in [hister.go](https://github.com/asciimoo/hister/blob/master/hister.go), which serves as the program's entry point. This file initializes the command-line interface by invoking cmd.Execute(), delegating all user interaction handling to the dedicated command package.

Cobra Command Definitions

The cmd package implements all user-facing commands using the Cobra framework. Located in files such as [cmd/version.go](https://github.com/asciimoo/hister/blob/master/cmd/version.go), [cmd/users.go](https://github.com/asciimoo/hister/blob/master/cmd/users.go), and [cmd/update.go](https://github.com/asciimoo/hister/blob/master/cmd/update.go), this layer handles flag parsing, help text generation, and sub-command dispatching for operations including listen, update, check-update, and user management. The command layer abstracts the underlying server complexity behind a familiar CLI interface.

// hister.go – entry point
func main() {
    if err := cmd.Execute(); err != nil {
        os.Exit(1)
    }
}

HTTP Server and API Layer

The server package provides the central nervous system of Hister, exposing REST APIs and handling external integrations. This component manages routing, middleware, debugging capabilities, and graceful shutdown procedures.

Server Bootstrap

The [server/server.go](https://github.com/asciimoo/hister/blob/master/server/server.go) file contains the server initialization logic, configuring the HTTP listener and wiring together the various middleware components. This file orchestrates the startup sequence that binds the application to its configured port (default 4433) and prepares the request handling pipeline.

MCP Integration for AI Assistants

A distinctive feature of Hister's architecture is the Message Control Protocol (MCP) endpoint implemented in [server/mcp.go](https://github.com/asciimoo/hister/blob/master/server/mcp.go). This handler enables AI assistants and browser extensions to communicate directly with the search engine, allowing programmatic document ingestion and retrieval. The MCP endpoint shares the same authentication and authorization mechanisms as the standard REST API.

REST Endpoints

API route definitions reside in [server/endpoints.go](https://github.com/asciimoo/hister/blob/master/server/endpoints.go), while specific endpoint implementations appear in [server/api.go](https://github.com/asciimoo/hister/blob/master/server/api.go). These files expose functionality for document CRUD operations, full-text search, and user session management. The search endpoint in particular handles complex query parsing including field filters, wildcards, negation, and result ranking.

// POST /api/v1/documents  (server/api.go)
type Document struct {
    ID      string `json:"id,omitempty"`
    Title   string `json:"title"`
    Content string `json:"content"`
    UserID  string `json:"user_id"`
}

Data Persistence and Models

Hister utilizes SQLite as its primary persistence mechanism, with the data model organized under the server/model package. This layer handles database migrations, entity relationships, and repository patterns for all persistent data.

Entity Definitions

The model layer defines core entities including users, sessions, history entries, crawls, and embeddings. Key files include [model/user.go](https://github.com/asciimoo/hister/blob/master/server/model/user.go) for authentication data, [model/history.go](https://github.com/asciimoo/hister/blob/master/server/model/history.go) for query logging, and [model/crawl.go](https://github.com/asciimoo/hister/blob/master/server/model/crawl.go) for web crawling metadata. These structures support the full-text indexing system by maintaining references to indexed content and metadata.

Search Engine Core Components

The full-text search functionality resides within the server package, implementing an inverted index structure for efficient content retrieval. Unlike external search dependencies, Hister embeds its indexing engine directly within the binary, ensuring offline functionality and data privacy.

Full-Text Indexing

The search implementation in [server/api.go](https://github.com/asciimoo/hister/blob/master/server/api.go) processes queries against the inverted index, supporting boolean logic, phrase matching, and relevance scoring. The engine operates on content extracted from web pages and user-uploaded documents, storing index data alongside the SQLite database.

Web Crawler Integration

The crawler module, defined in [model/crawl.go](https://github.com/asciimoo/hister/blob/master/server/model/crawl.go), fetches remote web pages and extracts textual content for indexing. This component handles bulk website imports, respecting robots.txt and implementing rate limiting to prevent server overload during large ingestion operations.

Semantic Search Capabilities

Hister extends beyond traditional keyword matching through its optional semantic search module. This component enables vector similarity searches by integrating with external embedding services.

Embedding Service Integration

The [model/embedding.go](https://github.com/asciimoo/hister/blob/master/server/model/embedding.go) file defines the interface to configurable embeddings endpoints. When enabled, the system sends document text to the configured service and stores the resulting vector representations in the database, allowing for similarity-based retrieval in addition to lexical matching.

// embedding.go – registers an external embeddings endpoint
cfg := config.Load()
cfg.SemanticSearch.Endpoint = "http://localhost:8080/embeddings"
model.SetEmbeddingConfig(cfg.SemanticSearch) // model/embedding.go

User Interface Implementations

Hister provides multiple interface options to accommodate different user workflows, all consuming the same underlying HTTP API.

Svelte Web Interface

The browser-based interface resides in webui/website, implemented as a Svelte + Vite application. This component provides visual search interfaces, settings management, and result visualization through modern web technologies. The server embeds these static assets and serves them at the root endpoint.

Terminal UI with Bubble Tea

For console-based interactions, the client package implements a Terminal User Interface (TUI) using the Bubble Tea framework. Files such as [client/search.go](https://github.com/asciimoo/hister/blob/master/client/search.go) and [client/history.go](https://github.com/asciimoo/hister/blob/master/client/history.go) provide interactive search capabilities, document snippet viewing, and history management without leaving the command line.

import "github.com/asciimoo/hister/client"

func doSearch(query string) {
    c := client.New("http://127.0.0.1:4433")
    res, err := c.Search(query)          // client/search.go
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(res.Hits)                // print matching documents
}

Summary

• Modular Go architecture separates concerns between CLI commands, HTTP server, data models, and client interfaces
• Cobra-powered CLI in cmd/ handles application lifecycle and user commands
• HTTP server layer in server/ provides REST APIs and MCP endpoints for AI integration
• SQLite persistence via server/model/ manages users, sessions, crawls, and document metadata
• Embedded full-text engine enables offline search without external dependencies
• Optional semantic search through configurable embedding services in model/embedding.go
• Dual interface approach serves both web users (Svelte) and terminal users (Bubble Tea TUI) through the same API

Frequently Asked Questions

What database does Hister use for data storage?

Hister uses SQLite as its primary persistence mechanism, as implemented in the server/model package. This choice enables true self-hosting without requiring separate database server infrastructure, keeping all indexed content and metadata within the user's control under single-file database storage.

How does Hister integrate with AI assistants?

The system exposes a Message Control Protocol (MCP) endpoint defined in [server/mcp.go](https://github.com/asciimoo/hister/blob/master/server/mcp.go). This specialized handler allows AI assistants and browser extensions to programmatically add documents to the index and execute searches, effectively turning Hister into a personal knowledge base for AI tools while maintaining local data sovereignty.

Can Hister perform semantic search or only keyword matching?

Hister supports both lexical and semantic search. The core engine provides traditional full-text indexing with boolean operators and ranking, while the optional embedding module in [model/embedding.go](https://github.com/asciimoo/hister/blob/master/server/model/embedding.go) enables vector similarity search by integrating with external embedding services via configurable HTTP endpoints.

What frontend technologies power Hister's web interface?

The web interface uses Svelte compiled with Vite, located in the webui/website directory. This provides a modern, reactive single-page application that communicates with the Go backend via REST API calls. For terminal users, a separate Bubble Tea TUI implementation in the client package offers equivalent functionality within console environments.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}