# How Litho's Intelligent Research and Analysis Stage Works in deepwiki-rs

> Discover how Litho's intelligent research and analysis stage automatically extracts architectural knowledge from codebases using a five-layer, multi-agent pipeline.

- Repository: [Sopaco/deepwiki-rs](https://github.com/sopaco/deepwiki-rs)
- Tags: deep-dive
- Published: 2026-02-16

---

**Litho's intelligent research and analysis stage uses a multi-agent pipeline orchestrated by `ResearchOrchestrator` to automatically extract architectural knowledge from codebases through five deterministic layers: macro context, domain architecture, workflows, key modules, and boundary analysis.**

The *deepwiki-rs* engine (codenamed Litho) generates comprehensive documentation by first executing an intelligent research phase that explores a project's structure without manual annotation. This stage is implemented in the `sopaco/deepwiki-rs` repository and operates through a series of specialized agents that analyze code at increasing levels of granularity.

## The Multi-Agent Research Pipeline

At the heart of Litho's intelligent research and analysis stage lies a deterministic pipeline where each agent builds upon the findings of its predecessors. The architecture follows a layered approach inspired by the C4 model, progressing from macro system context down to micro-level implementation details.

### ResearchOrchestrator as the Central Controller

The `ResearchOrchestrator` struct in [`src/generator/research/orchestrator.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/orchestrator.rs) serves as the central dispatcher that coordinates all research activities. It implements the `execute_research_pipeline` method, which sequences agent execution and manages data dependencies between layers.

```rust
// src/generator/workflow.rs – launch of the research stage
let research_orchestrator = ResearchOrchestrator::default();
research_orchestrator
    .execute_research_pipeline(&context)
    .await?;

```

### The Five-Layer Analysis Strategy

The pipeline executes agents across five distinct analytical layers:

1. **C1 Macro** – Captures overall purpose, business value, and system boundaries
2. **C2 Meso** – Identifies domain divisions, architecture, and core workflows  
3. **C3-C4 Micro** – Drills into critical modules and implementation details
4. **Boundary Analysis** – Enumerates external entry points and integration surfaces
5. **Database Overview** – Models SQL schemas and data flows when applicable

## Layer-by-Agent Breakdown

Each layer in Litho's intelligent research and analysis stage is handled by specialized agents that implement the `StepForwardAgent` trait, defining their own prompt templates, data requirements, and output schemas.

### C1 Macro Layer – System Context

The [`SystemContextResearcher`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/agents/system_context_researcher.rs) initiates the pipeline by analyzing project objectives, target users, and external system dependencies. Located in [`src/generator/research/agents/system_context_researcher.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/agents/system_context_researcher.rs), this agent defines the system's contextual boundaries and establishes the foundation for subsequent analysis.

### C2 Meso Layer – Domain and Architecture

Three agents collaborate to map the intermediate architectural layer:

**[`DomainModulesDetector`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/agents/domain_modules_detector.rs)** ([`src/generator/research/agents/domain_modules_detector.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/agents/domain_modules_detector.rs)) identifies top-level business domains, sub-modules, and inter-domain relationships.

**[`ArchitectureResearcher`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/agents/architecture_researcher.rs)** ([`src/generator/research/agents/architecture_researcher.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/agents/architecture_researcher.rs)) summarizes the system's structural layout, including components, layers, and deployment topology, producing **Mermaid diagrams** for visual representation.

**[`WorkflowResearcher`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/agents/workflow_researcher.rs)** ([`src/generator/research/agents/workflow_researcher.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/agents/workflow_researcher.rs)) extracts main functional workflows, mapping process steps and business flows from code analysis and previous research reports.

### C3-C4 Micro Layer – Key Module Insights

The [`KeyModulesInsight`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/agents/key_modules_insight.rs) agent ([`src/generator/research/agents/key_modules_insight.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/agents/key_modules_insight.rs)) performs parallel micro-analysis of each domain discovered in previous layers. For every critical module, it generates a detailed `KeyModuleReport` containing:

- Module description and responsibilities
- Interface definitions
- Implementation code snippets
- Flow charts and sequence diagrams

This agent utilizes `do_parallel_with_limit` to respect the `max_parallels` configuration setting, ensuring efficient resource utilization during deep analysis.

### Boundary and Database Analysis

**[`BoundaryAnalyzer`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/agents/boundary_analyzer.rs)** ([`src/generator/research/agents/boundary_analyzer.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/agents/boundary_analyzer.rs)) scans code insights for entry-point purposes including `Entry`, `Api`, `Router`, and `Config` types. It returns a structured JSON report enumerating all external integration surfaces and suggesting potential third-party integrations.

**[`DatabaseOverviewAnalyzer`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/agents/database_overview_analyzer.rs)** ([`src/generator/research/agents/database_overview_analyzer.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/agents/database_overview_analyzer.rs)) conditionally executes when SQL artifacts are present in the codebase. This agent filters SQL-related insights to extract tables, views, stored procedures, and relationships, producing a concise database schema overview.

## Execution Flow and Data Management

The intelligent research and analysis stage implements sophisticated prompt engineering and memory management to ensure coherent, contextually aware analysis across all agents.

### Prompt Engineering and LLM Integration

Each agent implements the `StepForwardAgent` trait, defining a `prompt_template()` method that returns a `PromptTemplate` struct containing:

- **System prompt** – Defines the agent's persona and expertise
- **Opening/Closing instructions** – Frames the specific analysis task
- **LLM call mode** – Either `Extract` (for JSON-typed results) or `PromptWithTools` (for richer artifacts like Mermaid diagrams)
- **Formatter configuration** – Controls output formatting

The `ResearchOrchestrator` resolves localized agent names via `AgentType::display_name` for clear console output, then invokes `agent.execute(context).await`. Internally, this builds the complete prompt and sends it to the configured LLM via `LLMClient`.

### Parallel Processing and Memory Scopes

Research results are stored in the **research memory scope** (`MemoryScope::STUDIES_RESEARCH`), making them available to downstream agents through `DataSource::ResearchResult`. The orchestrator guarantees execution order such that prerequisite agents complete before dependent agents begin.

The `KeyModulesInsight` agent leverages parallel execution via `do_parallel_with_limit`, respecting the `max_parallels` setting from the global configuration to prevent overwhelming the LLM API while maximizing throughput.

## Implementing Custom Research Agents

Extending Litho's intelligent research and analysis stage requires implementing the `StepForwardAgent` trait and registering the new agent in the orchestrator:

```rust
// src/generator/research/agents/custom_security_researcher.rs
#[derive(Default)]
pub struct SecurityResearcher;

impl StepForwardAgent for SecurityResearcher {
    type Output = SecurityReport;

    fn agent_type(&self) -> String { AgentType::SecurityResearcher.to_string() }
    fn memory_scope_key(&self) -> String { MemoryScope::STUDIES_RESEARCH.to_string() }

    fn data_config(&self) -> AgentDataConfig {
        AgentDataConfig {
            required_sources: vec![DataSource::CODE_INSIGHTS],
            optional_sources: vec![],
        }
    }

    fn prompt_template(&self) -> PromptTemplate {
        PromptTemplate {
            system_prompt: "You are a security analyst...".into(),
            opening_instruction: "Analyze the following code for security risks:".into(),
            closing_instruction: "".into(),
            llm_call_mode: LLMCallMode::Extract,
            formatter_config: FormatterConfig::default(),
        }
    }
}

```

To activate the custom agent, add the execution call in `ResearchOrchestrator::execute_research_pipeline` within [`src/generator/research/orchestrator.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/orchestrator.rs):

```rust
self.execute_agent(&SecurityResearcher::default(), context).await?;

```

## Summary

- **Litho's intelligent research and analysis stage** operates through a deterministic multi-agent pipeline orchestrated by `ResearchOrchestrator` in [`src/generator/research/orchestrator.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/orchestrator.rs).
- The pipeline progresses through **five analytical layers**: macro system context, meso domain architecture, micro module details, boundary analysis, and optional database overview.
- Each specialized agent implements the `StepForwardAgent` trait, defining custom **prompt templates**, **data dependencies**, and **output schemas** stored in the research memory scope.
- **Parallel execution** is utilized for intensive micro-analysis via `KeyModulesInsight`, while the orchestrator ensures prerequisite data is available before dependent agents execute.
- Developers can **extend the pipeline** by implementing custom agents and registering them in the orchestrator's execution sequence.

## Frequently Asked Questions

### What triggers the intelligent research and analysis stage in Litho?

The research stage is triggered when the main documentation workflow begins via `launch()` in [`src/generator/workflow.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/workflow.rs). This function instantiates `ResearchOrchestrator` and calls `execute_research_pipeline()`, which sequentially runs all research agents. The stage executes automatically as part of the standard documentation generation process without requiring manual intervention.

### How does Litho ensure research agents receive the correct context from previous steps?

The orchestrator manages data dependencies through `AgentDataConfig`, where each agent declares `required_sources` and `optional_sources` such as `DataSource::CODE_INSIGHTS` or `DataSource::ResearchResult`. The orchestrator executes agents in a deterministic order that guarantees prerequisites complete first. Results are stored in `MemoryScope::STUDIES_RESEARCH`, allowing downstream agents to retrieve structured data via the context's memory system.

### Can the research pipeline handle large codebases efficiently?

Yes, the pipeline incorporates parallelization specifically for intensive analysis tasks. The `KeyModulesInsight` agent in [`src/generator/research/agents/key_modules_insight.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/generator/research/agents/key_modules_insight.rs) uses `do_parallel_with_limit` to analyze multiple domains simultaneously, respecting the `max_parallels` configuration setting. This prevents API rate limiting while maximizing throughput. Additionally, the conditional execution of `DatabaseOverviewAnalyzer` ensures SQL analysis only runs when relevant artifacts are detected, avoiding unnecessary processing.