Hister Custom Query Language DSL: Syntax Reference and Usage Guide

Hister implements a purpose-built domain-specific language (DSL) that translates human-readable search strings into Bleve query objects, enabling complex full-text search without manual query construction.

Hister is an open-source search engine built on Bleve that provides an intuitive query syntax for indexing and retrieving documents. The Hister custom query language DSL bridges user input and the underlying search index through the querybuilder package. This system parses search strings into structured Bleve queries, supporting everything from simple keyword matching to advanced boolean logic, field-specific filtering, and regular expression searches.

Core DSL Syntax and Constructs

The DSL defines several fundamental query patterns in server/indexer/querybuilder/parser.go. Each construct maps to specific Bleve query types assembled by the Build function in server/indexer/querybuilder/search.go.

Plain Keywords and Boolean Modifiers

Unquoted terms perform standard keyword matching against the default field, with each word becoming an independent MatchQuery combined in a boolean structure. The syntax supports explicit boolean control through prefix operators:

• +keyword — Forces the term into the must clause (required match)
• -keyword — Forces the term into the must-not clause (explicit exclusion)
• * or empty input — Treated as MatchAllQuery returning every document

// Simple keyword search creates MatchQuery clauses
q, _ := querybuilder.Build("golang concurrency")

// Boolean modifiers for required/excluded terms
q, _ := querybuilder.Build("+required_term -excluded_term optional_term")

Exact Phrase Matching

Double-quoted strings trigger phrase queries that preserve word order and proximity. The lexer identifies these as TokenQuoted tokens, which the builder converts into MatchPhraseQuery objects on the default field.

// Exact phrase search
q, _ := querybuilder.Build(`"memory leak detection"`)

Alternation and OR Logic

Parentheses with pipe-separated alternatives create disjunction queries. The lexer generates TokenAlternation tokens when encountering (a|b|c) syntax, which the Build function translates into Bleve DisjunctionQuery structures (any alternative may match).

// Match any alternative: error OR failure OR bug
q, _ := querybuilder.Build(`(error|failure|bug)`)

Field-Specific Filters

Tokens containing colons indicate field-scoped searches using field:value syntax. The parser extracts the field name before the colon and applies the query value specifically to that Bleve field, enabling targeted searches across specific document attributes.

// Search specific field for exact phrase
q, _ := querybuilder.Build(`title:"concurrency patterns"`)

Advanced Query Directives

Beyond basic matching, the DSL provides specialized directives for regex filtering, sorting, and pagination control.

URL Regular Expression Filtering

The url_re: prefix enables regular expression matching against URL fields while preserving backslash escape sequences. Prefixing with - negates the filter, placing it in the must-not clause.

// Include only URLs matching pattern
q, _ := querybuilder.Build(`url_re:.*\.github\.com.*`)

// Exclude URLs matching pattern (negation)
q, _ := querybuilder.Build(`-url_re:.*example\.com.*`)

Sorting and Result Limiting

Directives starting with specific prefixes modify result presentation rather than matching logic:

• sort:field — Specifies result ordering (e.g., sort:date)
• limit:n — Constrains result set size (e.g., limit:20)

These directives are parsed separately from match clauses but included in the final query configuration returned by the builder.

// Combined query with sorting and limiting
q, _ := querybuilder.Build(`golang sort:date limit:50`)

Query Parsing Architecture

The translation from DSL strings to Bleve queries occurs in two phases: tokenization and query assembly.

Lexer and Token Types

Located in server/indexer/querybuilder/parser.go, the Lexer tokenizes raw input into three distinct token types:

• TokenWord — Unquoted words and field filters
• TokenQuoted — Double-quoted phrases
• TokenAlternation — Parenthesized alternatives split on unescaped |

This token stream feeds into the query construction logic, preserving the semantic structure of the original query string.

Build Function and Query Assembly

The Build function in server/indexer/querybuilder/search.go orchestrates query construction according to the asciimoo/hister source code. It assembles tokens into a Bleve query.Query tree, organizing clauses into must, must-not, and should boolean groups based on prefix modifiers and alternation structures.

The resulting query object supports complex nested logic:

// Complex query combining field filters, alternation, negation, and directives
q, _ := querybuilder.Build(`title:"api design" (rest|graphql) -url_re:.*deprecated.* sort:date limit:10`)

Implementation Details and Source Files

The query language implementation spans several files within the server/indexer/querybuilder directory:

Component	File Path	Responsibility
Lexer and token definitions	server/indexer/querybuilder/parser.go	Tokenizes DSL input into typed tokens (TokenWord, TokenQuoted, TokenAlternation)
Query builder	server/indexer/querybuilder/search.go	Maps tokens to Bleve query structures via the Build function
Usage examples	server/indexer/querybuilder/search_test.go	Unit tests demonstrating DSL capabilities and expected query structures
Boolean and filter tests	server/indexer/querybuilder/builder_test.go	Tests for alternation logic, field filters, and boolean modifier handling

Summary

• Hister's DSL translates human-readable strings into Bleve query objects through the querybuilder package.
• Core syntax includes plain keywords (MatchQuery), quoted phrases (MatchPhraseQuery), and parenthetical alternation (DisjunctionQuery).
• Field filters use field:value syntax, while url_re: enables regex filtering on URL fields with optional negation via -url_re:.
• Control directives sort: and limit: handle result ordering and pagination without affecting match logic.
• Boolean prefixes + and - explicitly control term inclusion in must or must-not clauses, while * triggers a match-all query.
• Source implementation resides in server/indexer/querybuilder/parser.go (lexer) and server/indexer/querybuilder/search.go (builder).

Frequently Asked Questions

What is the Hister custom query language DSL?

The Hister custom query language DSL is a specialized syntax for constructing search queries that compile into Bleve query objects. It allows users to perform complex searches using plain keywords, exact phrases, boolean logic, and field-specific filters without writing low-level Bleve query code in Go.

How does Hister handle boolean operators in search queries?

Hister uses prefix modifiers rather than traditional AND/OR keywords. The + prefix forces a term into the must clause (required match), the - prefix forces exclusion into the must-not clause, and parenthetical syntax (a|b|c) creates OR logic through a DisjunctionQuery. Unprefixed terms default to optional matching in the should clause.

Which source files implement the query parsing in Hister?

The lexer and token definitions reside in server/indexer/querybuilder/parser.go, while the high-level query builder that maps tokens to Bleve structures is implemented in server/indexer/querybuilder/search.go. Supporting tests illustrating DSL usage appear in server/indexer/querybuilder/search_test.go and server/indexer/querybuilder/builder_test.go.

How do I perform regular expression searches on URLs in Hister?

Use the url_re: prefix followed by your regex pattern, such as url_re:.*\.pdf$ to match PDF files. To exclude URLs matching a pattern, prefix with a minus sign: -url_re:.*spamdomain\.com.*. The parser preserves backslashes in these expressions for proper regex compilation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}