How to Exclude Specific Directories or Files from Litho's Analysis in deepwiki-rs

You can exclude specific directories or files from Litho's analysis by configuring the excluded_dirs, excluded_files, and excluded_extensions fields in the Config struct, either through a litho.toml configuration file, CLI flags, or programmatically in Rust.

The deepwiki-rs repository implements Litho, a documentation engine that analyzes codebases to generate structured documentation. When processing large repositories, you often need to exclude build artifacts, dependencies, or generated files to reduce token usage and focus analysis on relevant source code. Litho provides three distinct filter mechanisms to control exactly which paths get analyzed.

Understanding Litho's Exclusion Filters

Litho's filtering system centers on the Config struct defined in src/config.rs. This struct maintains three collections that determine path eligibility during the file system traversal:

• excluded_dirs: Skips any path containing these directory names. By default, this includes [".litho", "litho.docs", "target", "node_modules", ".git", "build", "dist", "venv", ".svelte-kit", "__pycache__", "__tests__", "__mocks__", "__fixtures__"]【https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs#L580-L594】.

• excluded_files: Skips files matching exact names or simple * wildcard patterns. Defaults include ["litho.toml", "*.litho", "*.log", "*.tmp", "*.cache", "bun.lock", "package-lock.json", "yarn.lock", "pnpm-lock.yaml", "Cargo.lock", ".gitignore", "*.tpl", "*.md", "*.txt", ".env"]【https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs#L595-L607】.

• excluded_extensions: Skips files based solely on their extension. Default exclusions cover ["jpg", "jpeg", "png", "gif", "bmp", "ico", "mp3", "mp4", "avi", "pdf", "zip", "tar", "exe", "dll", "so", "archive"]【https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs#L612-L628】.

During the analysis phase, the FileExplorer tool in src/llm/tools/file_explorer.rs evaluates each path against these collections through its is_ignored method (lines 59-73)【https://github.com/sopaco/deepwiki-rs/blob/main/src/llm/tools/file_explorer.rs#L59-L73】.

Method 1: Configure Exclusions via litho.toml

The most persistent way to exclude specific directories or files from Litho's analysis is through the TOML configuration file. The CLI (src/cli.rs) automatically reads this file at startup and populates the Config struct.

Create or modify litho.toml in your project root:

# litho.toml

excluded_dirs = [
    ".litho",
    "target",
    "node_modules",
    "generated",      # Custom generated code directory

    "tests"           # Exclude test directories

]

excluded_files = [
    "*.log",
    "secret.env",     # Sensitive configuration

    "README.md",
    "*.tmp"
]

excluded_extensions = [
    "png",
    "mp4",
    "sqlite"          # Database files

]

This approach ensures that every run of Litho respects these exclusions without requiring additional command-line arguments.

Method 2: Override with CLI Flags

For temporary or one-off exclusions, you can pass overrides directly via command-line arguments. The CLI parser merges these flags into the Config instance before analysis begins, allowing you to exclude specific directories or files without modifying litho.toml.

# Exclude the docs/ folder and all backup files for this run only

deepwiki-rs -p . --exclude-dir docs --exclude-file *.bak --exclude-file *.old

This method is particularly useful when:

• Testing Litho against a subset of your codebase
• Running CI pipelines that need different exclusion rules than local development
• Temporarily ignoring directories that would otherwise cause excessive token consumption

Method 3: Programmatic Configuration

When embedding Litho as a library in your own Rust application, you can construct a Config instance manually and populate the exclusion vectors directly. This provides maximum flexibility for dynamic exclusion logic.

use deepwiki_rs::config::Config;
use deepwiki_rs::generator::run;

fn main() {
    let mut cfg = Config::default();
    
    // Exclude specific directories
    cfg.excluded_dirs.push("vendor".to_string());
    cfg.excluded_dirs.push("third_party".to_string());
    
    // Exclude specific files or patterns
    cfg.excluded_files.push("debug.log".to_string());
    cfg.excluded_files.push("local.env".to_string());
    
    // Exclude by extension
    cfg.excluded_extensions.push("sqlite".to_string());
    cfg.excluded_extensions.push("db".to_string());

    // Run Litho with custom configuration
    run(cfg).expect("Litho analysis failed");
}

This approach allows you to build exclusion rules based on runtime conditions, user preferences, or external configuration sources.

How Exclusion Filters Work Internally

Understanding the internal mechanics helps optimize your exclusion strategy. When Litho initiates analysis, the FileExplorer tool traverses the project tree and evaluates each path against the exclusion criteria in src/llm/tools/file_explorer.rs.

The evaluation logic follows this priority order:

1. Directory name check: If the path contains any string in excluded_dirs, the entire subtree is skipped immediately.
2. File name pattern matching: The file name is compared against excluded_files entries, supporting simple * wildcards.
3. Extension filtering: The file extension is checked against excluded_extensions.
4. Runtime flags: Additional checks for test files (if include_tests is false), hidden files (if include_hidden is false), and file size limits.

This short-circuit evaluation ensures that directory-level exclusions prevent unnecessary deep traversal of large ignored trees like node_modules or target, significantly improving performance.

Summary

• Litho uses three filter collections in the Config struct: excluded_dirs, excluded_files, and excluded_extensions, defined in src/config.rs.
• Default exclusions cover common build artifacts, dependency folders, and binary files, but you can customize them via litho.toml, CLI flags, or programmatic configuration.
• Performance optimization occurs through early short-circuiting in src/llm/tools/file_explorer.rs, where directory matches prevent traversal of entire subtrees.
• Flexible configuration allows temporary exclusions via command-line arguments (--exclude-dir, --exclude-file) or permanent rules via TOML configuration.

Frequently Asked Questions

How do I exclude a directory that isn't in the default exclusion list?

Add the directory name to the excluded_dirs array in your litho.toml file. For example, to exclude a generated folder: excluded_dirs = ["target", "generated", "node_modules"]. You can also use the --exclude-dir CLI flag for temporary exclusions without modifying the configuration file.

Can I use glob patterns or regex in the exclusion filters?

Litho supports simple wildcard patterns using * in the excluded_files field (e.g., *.log or debug.*), but it does not support full regular expressions or complex glob patterns like **. For directory exclusions, you must specify exact directory names that match any segment of the path.

Why are my exclusion rules not being applied during analysis?

Ensure your litho.toml is in the project root or the directory specified by the -p flag. Verify that the configuration file uses the correct field names (excluded_dirs, excluded_files, excluded_extensions) and TOML syntax. If using CLI flags, check that they are passed before any positional arguments. The FileExplorer logic in src/llm/tools/file_explorer.rs applies these filters at runtime, so syntax errors in the config may cause defaults to be used instead.

Does excluding directories improve performance significantly?

Yes. Litho's FileExplorer short-circuits directory traversal when it encounters an excluded_dirs match, preventing the engine from scanning large subtrees like node_modules or target directories. This reduces both file system I/O and the number of paths passed to the LLM, significantly decreasing analysis time and token consumption.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}