How to Define Exclusion Patterns for Local File Indexing in Hister

Hister supports Unix-style glob patterns for excluding directories from local indexing through both global and per-directory excludes lists, applying filepath.Match to directory names during the filesystem walk.

The open-source search indexer Hister (asciimoo/hister) builds its database by recursively walking configured filesystem paths. To prevent dependency caches, hidden folders, or temporary directories from polluting the index, you can define exclusion patterns directly in the YAML configuration. These patterns use standard glob syntax and are evaluated during the initial crawl and subsequent file-watching operations.

Where Exclusion Logic Lives in the Source Code

The filtering mechanism resides in files/files.go. When Hister indexes a path, it invokes DirectoryMatchesPath (approximately lines 99-107), which decomposes the file’s relative path into parent components. For each directory name, it calls shouldSkipDir (approximately lines 141-158) to determine if that branch of the tree should be skipped.

The shouldSkipDir function implements a hierarchical check:

1. Hidden directories – If include_hidden is false, any name beginning with a dot is rejected.
2. Well-known cache directories – Hard-coded names like node_modules, __pycache__, and vendor in the internal skipDirs map are always excluded.
3. User-defined patterns – The function iterates over the excludes slice and applies filepath.Match to the directory name.

func shouldSkipDir(name string, excludes []string, includeHidden bool) bool {
    if !includeHidden {
        if strings.HasPrefix(name, ".") {
            return true // hidden dir
        }
        if _, ok := skipDirs[name]; ok {
            return true // well-known cache dir
        }
    }
    for _, pattern := range excludes {
        if matched, _ := filepath.Match(pattern, name); matched {
            return true // user-defined exclude
        }
    }
    return false
}

Configuration Scopes for Exclusion Patterns

Hister reads exclusion settings from the Config struct defined in config/config.go (around line 112). You can define patterns at two levels of granularity:

Global Exclusion Patterns

The top-level excludes field applies to every directory configured for indexing. Place this at the root of your config.yaml:

excludes:
  - "tmp*"
  - "vendor"
  - ".*"

These patterns are stored in Config.Excludes []string and passed to the walker for every scanned path.

Per-Directory Exclusion Patterns

Within the indexer.directories list, each entry is a Directory struct that can override behavior with its own excludes slice. These patterns only affect the specific root path:

indexer:
  directories:
    - path: ~/projects/webapp
      includes_hidden: false
      excludes:
        - "node_modules"
        - "dist"
        - "cache?"

The Directory.Excludes field takes precedence alongside global excludes—both lists are checked during the walk.

Practical Configuration Examples

Use these YAML snippets to control what Hister ignores during indexing.

Example 1: Exclude all temporary directories globally

# ~/.config/hister/config.yaml

excludes:
  - "tmp*"
  - "temp"
  - "*.backup"

Example 2: Exclude build artifacts in a specific project while keeping them elsewhere

indexer:
  directories:
    - path: ~/work/frontend-app
      includes_hidden: false
      excludes:
        - "build"
        - "coverage"
        - "node_modules"

Example 3: Index hidden folders but exclude a specific cache directory

indexer:
  directories:
    - path: ~/documents
      includes_hidden: true
      excludes:
        - ".snapshot"
        - "Thumbs.db"

Understanding Glob Pattern Matching

Hister uses Go’s filepath.Match syntax, which supports:

• * – Matches any sequence of characters within the directory name.
• ? – Matches exactly one character.
• [abc] – Matches any single character inside the brackets.
• [a-z] – Matches any character in the range.

Critical limitation: Patterns match directory names only, not full filesystem paths. For example, the pattern logs excludes every directory named logs regardless of depth, but project/logs will not match because the slash is treated as a path separator, not part of the name being tested.

Summary

• Exclusion patterns in Hister are defined using glob syntax in config.yaml.
• The shouldSkipDir function in files/files.go evaluates patterns against directory names using filepath.Match.
• Define global exclusions via the top-level excludes field in the Config struct.
• Define granular exclusions using the excludes field inside individual indexer.directories entries.
• Hidden directories (dotfiles) and well-known cache folders are excluded by default unless includes_hidden is enabled.

Frequently Asked Questions

What glob syntax does Hister support for exclusions?

Hister uses Go’s filepath.Match implementation. You can use * for wildcards, ? for single characters, and character classes like [0-9]. However, recursive globs (**) are not supported; patterns apply only to individual directory names, not full paths.

Can I exclude specific files or only directories?

The current implementation in files/files.go only evaluates exclusion patterns against directory names in shouldSkipDir. Individual files cannot be excluded by name using the excludes configuration; the filter operates at the directory level to prune entire branches of the filesystem tree.

How do I exclude hidden directories like .git or .svn?

By default, Hister excludes all hidden directories (those starting with a dot) unless you set includes_hidden: true in the directory configuration. If you enable hidden directory indexing but want to exclude specific ones (e.g., .git), add them to the excludes list: - ".git".

Where does Hister look for the configuration file?

Hister typically reads config.yaml from the system configuration directory (commonly ~/.config/hister/ on Linux or the equivalent OS-specific path). The Config struct definition in config/config.go parses this file at startup to populate both global and per-directory exclusion lists.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}