# Hister Custom Query Language DSL: Syntax Reference and Usage Guide

> Learn Hister's custom query language DSL. Translate human-readable search strings into Bleve queries for powerful full-text search without manual construction. Explore the syntax and usage guide.

- Repository: [Adam Tauber/hister](https://github.com/asciimoo/hister)
- Tags: api-reference
- Published: 2026-08-27

---

**Hister implements a purpose-built domain-specific language (DSL) that translates human-readable search strings into Bleve query objects, enabling complex full-text search without manual query construction.**

Hister is an open-source search engine built on Bleve that provides an intuitive query syntax for indexing and retrieving documents. The **Hister custom query language DSL** bridges user input and the underlying search index through the `querybuilder` package. This system parses search strings into structured Bleve queries, supporting everything from simple keyword matching to advanced boolean logic, field-specific filtering, and regular expression searches.

## Core DSL Syntax and Constructs

The DSL defines several fundamental query patterns in [`server/indexer/querybuilder/parser.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/querybuilder/parser.go). Each construct maps to specific Bleve query types assembled by the `Build` function in [`server/indexer/querybuilder/search.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/querybuilder/search.go).

### Plain Keywords and Boolean Modifiers

Unquoted terms perform standard keyword matching against the default field, with each word becoming an independent `MatchQuery` combined in a boolean structure. The syntax supports explicit boolean control through prefix operators:

- **`+keyword`** — Forces the term into the **must** clause (required match)
- **`-keyword`** — Forces the term into the **must-not** clause (explicit exclusion)
- **`*`** or empty input — Treated as `MatchAllQuery` returning every document

```go
// Simple keyword search creates MatchQuery clauses
q, _ := querybuilder.Build("golang concurrency")

// Boolean modifiers for required/excluded terms
q, _ := querybuilder.Build("+required_term -excluded_term optional_term")

```

### Exact Phrase Matching

Double-quoted strings trigger phrase queries that preserve word order and proximity. The lexer identifies these as `TokenQuoted` tokens, which the builder converts into `MatchPhraseQuery` objects on the default field.

```go
// Exact phrase search
q, _ := querybuilder.Build(`"memory leak detection"`)

```

### Alternation and OR Logic

Parentheses with pipe-separated alternatives create disjunction queries. The lexer generates `TokenAlternation` tokens when encountering `(a|b|c)` syntax, which the `Build` function translates into Bleve `DisjunctionQuery` structures (any alternative may match).

```go
// Match any alternative: error OR failure OR bug
q, _ := querybuilder.Build(`(error|failure|bug)`)

```

### Field-Specific Filters

Tokens containing colons indicate field-scoped searches using `field:value` syntax. The parser extracts the field name before the colon and applies the query value specifically to that Bleve field, enabling targeted searches across specific document attributes.

```go
// Search specific field for exact phrase
q, _ := querybuilder.Build(`title:"concurrency patterns"`)

```

## Advanced Query Directives

Beyond basic matching, the DSL provides specialized directives for regex filtering, sorting, and pagination control.

### URL Regular Expression Filtering

The `url_re:` prefix enables regular expression matching against URL fields while preserving backslash escape sequences. Prefixing with `-` negates the filter, placing it in the must-not clause.

```go
// Include only URLs matching pattern
q, _ := querybuilder.Build(`url_re:.*\.github\.com.*`)

// Exclude URLs matching pattern (negation)
q, _ := querybuilder.Build(`-url_re:.*example\.com.*`)

```

### Sorting and Result Limiting

Directives starting with specific prefixes modify result presentation rather than matching logic:

- **`sort:field`** — Specifies result ordering (e.g., `sort:date`)
- **`limit:n`** — Constrains result set size (e.g., `limit:20`)

These directives are parsed separately from match clauses but included in the final query configuration returned by the builder.

```go
// Combined query with sorting and limiting
q, _ := querybuilder.Build(`golang sort:date limit:50`)

```

## Query Parsing Architecture

The translation from DSL strings to Bleve queries occurs in two phases: tokenization and query assembly.

### Lexer and Token Types

Located in [`server/indexer/querybuilder/parser.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/querybuilder/parser.go), the `Lexer` tokenizes raw input into three distinct token types:

- **`TokenWord`** — Unquoted words and field filters
- **`TokenQuoted`** — Double-quoted phrases
- **`TokenAlternation`** — Parenthesized alternatives split on unescaped `|`

This token stream feeds into the query construction logic, preserving the semantic structure of the original query string.

### Build Function and Query Assembly

The `Build` function in [`server/indexer/querybuilder/search.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/querybuilder/search.go) orchestrates query construction according to the `asciimoo/hister` source code. It assembles tokens into a Bleve `query.Query` tree, organizing clauses into must, must-not, and should boolean groups based on prefix modifiers and alternation structures.

The resulting query object supports complex nested logic:

```go
// Complex query combining field filters, alternation, negation, and directives
q, _ := querybuilder.Build(`title:"api design" (rest|graphql) -url_re:.*deprecated.* sort:date limit:10`)

```

## Implementation Details and Source Files

The query language implementation spans several files within the `server/indexer/querybuilder` directory:

| Component | File Path | Responsibility |
|-----------|-----------|----------------|
| Lexer and token definitions | [`server/indexer/querybuilder/parser.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/querybuilder/parser.go) | Tokenizes DSL input into typed tokens (`TokenWord`, `TokenQuoted`, `TokenAlternation`) |
| Query builder | [`server/indexer/querybuilder/search.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/querybuilder/search.go) | Maps tokens to Bleve query structures via the `Build` function |
| Usage examples | [`server/indexer/querybuilder/search_test.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/querybuilder/search_test.go) | Unit tests demonstrating DSL capabilities and expected query structures |
| Boolean and filter tests | [`server/indexer/querybuilder/builder_test.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/querybuilder/builder_test.go) | Tests for alternation logic, field filters, and boolean modifier handling |

## Summary

- **Hister's DSL** translates human-readable strings into Bleve query objects through the `querybuilder` package.
- **Core syntax** includes plain keywords (`MatchQuery`), quoted phrases (`MatchPhraseQuery`), and parenthetical alternation (`DisjunctionQuery`).
- **Field filters** use `field:value` syntax, while `url_re:` enables regex filtering on URL fields with optional negation via `-url_re:`.
- **Control directives** `sort:` and `limit:` handle result ordering and pagination without affecting match logic.
- **Boolean prefixes** `+` and `-` explicitly control term inclusion in must or must-not clauses, while `*` triggers a match-all query.
- **Source implementation** resides in [`server/indexer/querybuilder/parser.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/querybuilder/parser.go) (lexer) and [`server/indexer/querybuilder/search.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/querybuilder/search.go) (builder).

## Frequently Asked Questions

### What is the Hister custom query language DSL?

The Hister custom query language DSL is a specialized syntax for constructing search queries that compile into Bleve query objects. It allows users to perform complex searches using plain keywords, exact phrases, boolean logic, and field-specific filters without writing low-level Bleve query code in Go.

### How does Hister handle boolean operators in search queries?

Hister uses prefix modifiers rather than traditional AND/OR keywords. The `+` prefix forces a term into the must clause (required match), the `-` prefix forces exclusion into the must-not clause, and parenthetical syntax `(a|b|c)` creates OR logic through a `DisjunctionQuery`. Unprefixed terms default to optional matching in the should clause.

### Which source files implement the query parsing in Hister?

The lexer and token definitions reside in [`server/indexer/querybuilder/parser.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/querybuilder/parser.go), while the high-level query builder that maps tokens to Bleve structures is implemented in [`server/indexer/querybuilder/search.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/querybuilder/search.go). Supporting tests illustrating DSL usage appear in [`server/indexer/querybuilder/search_test.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/querybuilder/search_test.go) and [`server/indexer/querybuilder/builder_test.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/querybuilder/builder_test.go).

### How do I perform regular expression searches on URLs in Hister?

Use the `url_re:` prefix followed by your regex pattern, such as `url_re:.*\.pdf$` to match PDF files. To exclude URLs matching a pattern, prefix with a minus sign: `-url_re:.*spamdomain\.com.*`. The parser preserves backslashes in these expressions for proper regex compilation.