How the `ai-memory bootstrap` Command Collects Git Log, README, and Docs into Seed Wiki Pages

The bootstrap command runs a two-stage pipeline: first, the CLI harvests git history, README files, docs, and code comments locally into a JSON bundle; then the server processes that bundle through an LLM to generate structured wiki pages with a bootstrap.md manifest.

The ai-memory bootstrap command in the akitaonrails/ai-memory repository automates the creation of initial wiki pages from a project's existing artifacts. This process transforms scattered documentation—git commits, README files, markdown documentation, Rust module comments, and project-rule files—into a searchable, consolidated knowledge base. Understanding how this collection and processing pipeline works helps you optimize what gets ingested and how the LLM structures the output.

Local Source Collection in the CLI

The bootstrap process begins on the user's machine in crates/ai-memory-cli/src/commands/bootstrap.rs. The run() function orchestrates local collection before any server communication occurs.

Detecting the Repository Root

If no --repo-path is specified, the CLI calls discover_repo_root() to locate the nearest .git directory using libgit2. When no repository is found, it falls back to the current working directory and disables git-history collection.

// From crates/ai-memory-consolidate/src/bootstrap.rs
pub fn discover_repo_root() -> anyhow::Result<PathBuf> {
    // Uses libgit2 to find .git directory
    // Falls back to cwd if not found
}

Source: discover_repo_root() in bootstrap.rs [L35-L42].

The Five Source Collectors

The collect_sources() function in crates/ai-memory-consolidate/src/bootstrap.rs gathers five distinct artifact types. Each collector returns a Vec<BootstrapSource> containing the source kind, a short label, and the full text.

Collector	Purpose	Key Filter
collect_git_commits()	Formats git history as paragraphs	Skips trivial commits (~120 chars) unless they match conventional-commit prefixes
collect_readme()	Reads top-level README files	Matches common README filenames at repo root
collect_docs()	Recursively harvests docs/**/*.md	None—all markdown files included
collect_rust_module_headers()	Extracts //! doc comments	Ignores target/, node_modules/, typical build directories
collect_project_rules()	Picks up AI context files	Matches CLAUDE.md, AGENTS.md, and similar

Source: collect_sources() and helpers in bootstrap.rs [L49-L84].

CLI-to-Server Transmission

After collection, the CLI serializes the bundle—including source count, token-budget settings, and the dry_run flag—into JSON and POSTs it to /admin/bootstrap.

// From crates/ai-memory-cli/src/commands/bootstrap.rs
let body = serde_json::to_string(&BootstrapRequest {
    sources,
    config: config.into(),
})?;
let resp = client
    .post(format!("{}/admin/bootstrap", server_url))
    .bearer_auth(token)
    .json(&body)
    .send()
    .await?;

Source: run() in bootstrap.rs (CLI) [L32-L44].

Server-Side Processing Pipeline

The server receives the bundle in crates/ai-memory-consolidate/src/bootstrap.rs. The Bootstrap::process_sources() method implements the core transformation logic.

Idempotency Check

Before processing, the server verifies that bootstrap.md does not already exist for the target workspace/project. If it exists and --force was not specified, the request fails with BootstrapError::AlreadyBootstrapped. Re-running requires explicit --force.

Source: process_sources() early block [L97-L107].

Token Budget Pruning

The prune_sources_to_budget() function ensures the input fits within max_input_tokens. It removes lower-priority sources in this order until the estimated token count (characters ÷ 4) satisfies the budget:

1. Rust module headers (lowest priority)
2. Git commits
3. Docs folder contents
4. README files
5. Project-rule files (highest priority, never dropped)

Source: prune_sources_to_budget() [L1005-L1029].

Optional Chunking for Large Inputs

When chunk_input_tokens is non-zero and the pruned sources exceed this per-chunk budget, plan_bootstrap_chunks() splits the sources into sequential chunks. Each chunk reserves ~1,000 tokens for the system prompt.

Source: chunk planning code [L50-L71].

LLM Request Construction

For each chunk, build_chunk_request() assembles a ChatRequest with:

• System prompt from prompts/bootstrap_system.md
• Enumerated sources with their labels
• Temperature: 0.2
• Per-chunk max_tokens limit

Source: build_chunk_request() [L1560-L1586].

Structured LLM Output

The ai_memory_llm::LlmProvider invokes complete_structured to receive a BootstrapBatch containing:

• List of BootstrapPage structs (path, title, markdown body, optional tags)
• Human-readable rationale explaining the generation choices

Source: LLM call loop [L138-L146].

Wiki Persistence and Deduplication

Page Deduplication

As chunks are processed, pages are stored in a BTreeMap keyed by path. Later chunks that would produce duplicate paths are silently ignored with a warning, ensuring idempotent results across chunked runs.

Source: insert_bootstrap_page() [L1310-L1320].

Atomic Batch Writes

Each BootstrapPage becomes a WritePageRequest with bootstrapped_at front-matter. The manifest (bootstrap.md) is rendered by render_manifest_body(). All writes occur in a single batch via wiki.apply_batch() followed by commit_all().

// From crates/ai-memory-consolidate/src/bootstrap.rs
let batch: Vec<WritePageRequest> = pages
    .into_values()
    .map(|p| WritePageRequest {
        path: p.path,
        body: format!("---\nbootstrapped_at: {}\n---\n\n{}", 
            now, p.body),
        ..Default::default()
    })
    .collect();
wiki.apply_batch(&batch).await?;
wiki.commit_all(&[manifest_path]).await?;

Sources: page-writing loop [L94-L106] and manifest rendering [L1570-L1592].

Outcome Reporting

The BootstrapOutcome struct reports:

• Counts: collected, sent, dropped sources
• Per-kind tallies
• Estimated token usage
• Pages written
• LLM rationale

The CLI prints a human-friendly summary and emits JSON for scripting.

Source: BootstrapOutcome and print_human_report() in CLI [L106-L122].

Practical Examples

Command-Line Usage

# Dry-run to preview what would be collected

$ ai-memory bootstrap --workspace my-workspace \
    --project my-project --dry-run

# Full bootstrap with all source types

$ export AI_MEMORY_SERVER_URL=http://localhost:49374
$ export AI_MEMORY_AUTH_TOKEN=sk-...
$ ai-memory bootstrap --workspace my-workspace \
    --project my-project --force

Programmatic Library Usage

use ai_memory_consolidate::{
    Bootstrap, BootstrapConfig, collect_sources,
};
use ai_memory_store::ReaderPool;
use ai_memory_wiki::Wiki;
use ai_memory_llm::providers::OpenAiProvider;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let reader = ReaderPool::new(/* ... */)?;
    let wiki = Wiki::open("./wiki")?;
    let llm = Arc::new(OpenAiProvider::new("gpt-4-turbo", None)?);
    let bootstrap = Bootstrap { reader, wiki, llm };

    let cfg = BootstrapConfig {
        repo_path: std::path::PathBuf::from("/path/to/repo"),
        workspace_id: "ws".into(),
        project_id: "proj".into(),
        max_input_tokens: 150_000,
        chunk_input_tokens: 24_000,
        sources_collected: None,
        include_git: true,
        include_readme: true,
        include_docs: true,
        include_code: true,
        since: None,
        dry_run: false,
        force: false,
    };

    let sources = collect_sources(
        &cfg.repo_path,
        cfg.since.as_deref(),
        cfg.include_git,
        cfg.include_readme,
        cfg.include_docs,
        cfg.include_code,
    )?;
    let outcome = bootstrap.process_sources(&cfg, sources).await?;
    println!("Bootstrap wrote {} pages", outcome.pages_written.len());
    Ok(())
}

Key Implementation Files

File	Role
crates/ai-memory-cli/src/commands/bootstrap.rs	CLI driver: argument parsing, local collection, HTTP POST to server
crates/ai-memory-consolidate/src/bootstrap.rs	Core server logic: budgeting, chunking, LLM calls, wiki persistence
crates/ai-memory-consolidate/src/lib.rs	Public API exports (Bootstrap, collect_sources)
crates/ai-memory-wiki/src/wiki.rs	Atomic markdown writer and batch operations
crates/ai-memory-llm/src/openai.rs (or provider)	LLM API implementation
crates/ai-memory-consolidate/prompts/bootstrap_system.md	System prompt guiding wiki page generation

Summary

• The ai-memory bootstrap command operates in two stages: local CLI collection and server-side LLM processing.
• Five artifact types are harvested: git commits, README, docs, Rust module headers, and project-rule files.
• Token budgeting prunes lower-priority sources when inputs exceed limits; chunking handles large repositories.
• The implementation is idempotent by default—re-running requires --force to overwrite existing bootstrap.md.
• Pages are written atomically via the Wiki API with automatic deduplication and manifest generation.

Frequently Asked Questions

What happens if my repository is too large for the token budget?

The prune_sources_to_budget() function automatically drops lower-priority sources until the fit complies. Module headers are removed first, then git commits, docs, and README files—with project-rule files (CLAUDE.md, AGENTS.md) preserved as highest priority. For finer control, set chunk_input_tokens to process large inputs across multiple LLM calls.

Can I run bootstrap multiple times on the same project?

No—bootstrap is idempotent. The server checks for existing bootstrap.md and rejects duplicate requests unless you pass --force. This prevents accidental overwrites of manually curated wiki content. Use --force explicitly when you want to regenerate from scratch.

Does bootstrap capture code files or only documentation?

By default, bootstrap captures Rust module headers (//! comments) via collect_rust_module_headers(), not full source code. The include_code flag controls this behavior. For other languages or full file ingestion, extend the collector or pre-process files into the docs/ directory.

How does the LLM decide what pages to create and what to name them?

The system prompt in prompts/bootstrap_system.md instructs the LLM to generate BootstrapPage structs with descriptive paths and titles. The model analyzes source content themes and creates logical groupings—often producing pages like architecture.md, api-reference.md, or decision-log.md based on the input artifacts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}