# ReportEngine Architecture and Report Generation Pipeline: How the Bettafish Codebase Converts Agent Outputs into Structured Reports

> Explore the ReportEngine architecture in bettafish. See how its four-layer LLM-driven pipeline converts agent outputs into structured HTML, Markdown, or PDF reports.

- Repository: [BaiFu/bettafish](https://github.com/666ghj/bettafish)
- Tags: architecture
- Published: 2026-02-23

---

**The ReportEngine in the 666ghj/bettafish repository implements a modular, four-layer pipeline that transforms raw outputs from Query, Media, and Insight agents into polished HTML, Markdown, or PDF documents through orchestrated LLM-driven processing nodes and intermediate representation stitching.**

The ReportEngine serves as the central reporting service within the Bettafish project, designed to synthesize heterogeneous data sources into publication-ready documents. Its architecture separates concerns between orchestration, processing, utilities, and rendering to maintain stateless, retry-safe operation while supporting real-time streaming feedback.

## Four-Layer Architecture Overview

The ReportEngine architecture divides functionality into four logical layers that process data sequentially from raw inputs to final output files.

### Orchestration Layer

The **ReportAgent** class in [`ReportEngine/agent.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/agent.py) acts as the central coordinator, maintaining workflow state and driving each processing stage. It manages the end-to-end lifecycle from input normalization through final persistence.

### Processing Nodes

Individual LLM-driven steps handle specific transformation tasks:
- `TemplateSelectionNode` → Selects appropriate Markdown skeletons
- `DocumentLayoutNode` → Designs document-wide structure and themes
- `WordBudgetNode` → Calculates per-chapter word targets
- `ChapterGenerationNode` → Synthesizes content via LLM calls
- `GraphRAGQueryNode` → Enhances chapters with knowledge graph context

### Core Utilities

Helper modules in `ReportEngine/core/` and `ReportEngine/utils/` manage template parsing, intermediate representation (IR) stitching, chapter storage, configuration loading, and JSON validation.

### Renderers and State

The rendering layer in `ReportEngine/renderers/` converts IR into target formats (HTML, Markdown, PDF), while [`ReportEngine/state/state.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/state/state.py) and [`ReportEngine/flask_interface.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/flask_interface.py) provide runtime state management and HTTP API access.

## Orchestration Layer: The ReportAgent Workflow

The `ReportAgent` class implements the primary `generate_report()` method, which executes an 11-step pipeline to convert agent outputs into structured documents.

### Input Normalization and Template Selection

First, `_normalize_reports()` converts the three agent outputs (Query, Media, Insight) plus optional forum logs into plain strings. Then `_select_template()` invokes `TemplateSelectionNode` to analyze the query and content, selecting an appropriate Markdown template from the local repository.

### Template Parsing and Layout Design

The `_slice_template()` method calls `parse_template_sections` from [`ReportEngine/core/template_parser.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/core/template_parser.py) to decompose the template into `TemplateSection` objects, each assigned a stable `chapter_id`. Subsequently, `DocumentLayoutNode` generates the document-wide design including titles, hero images, table of contents entries, and theme tokens.

### Budget Planning and Context Building

`WordBudgetNode` computes per-chapter word targets and global writing guidelines. The agent then aggregates all inputs via `_build_generation_context()`, creating a single dictionary containing the query, normalized reports, layout specifications, and budget constraints for downstream consumption.

### Optional GraphRAG Enhancement

When `GRAPHRAG_ENABLED` is true, `_build_knowledge_graph()` constructs a graph from the engine states and forum logs. During chapter generation, `GraphRAGQueryNode` executes multi-turn queries against this graph to enrich content with contextual relationships.

### Chapter Generation and IR Stitching

For each `TemplateSection`, the agent invokes `ChapterGenerationNode.run()`, which may retry on JSON parsing errors or content-filter failures using fallback LLM chains (`json_rescue_clients`). Streaming callbacks (`stream_handler`) emit real-time progress events to UI clients.

After all chapters complete, `DocumentComposer.build_document()` in [`ReportEngine/core/stitcher.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/core/stitcher.py) performs IR stitching: sorting chapters, injecting unique anchors, merging metadata, and versioning the document structure.

### Rendering and Persistence

Finally, `HTMLRenderer` (or `MarkdownRenderer`) consumes the IR via `self.renderer.render(document_ir)` to produce the output string. If `save_report=True`, `_save_report()` writes the HTML, IR JSON, and status files to `self.config.OUTPUT_DIR/<report_id>/`.

## Processing Nodes: LLM-Driven Pipeline Steps

Each node inherits from `BaseNode` in [`ReportEngine/nodes/base_node.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/nodes/base_node.py) and implements specific transformation logic.

### TemplateSelectionNode

Located in [`ReportEngine/nodes/template_selection_node.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/nodes/template_selection_node.py), this node calls the LLM with the query, engine reports, forum logs, and available templates. It returns a dictionary containing `template_name`, `template_content`, and `selection_reason`.

### DocumentLayoutNode

This node in [`ReportEngine/nodes/document_layout_node.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/nodes/document_layout_node.py) generates the high-level document structure, determining title placement, hero imagery, navigation structure, and color theming.

### WordBudgetNode

Implemented in [`ReportEngine/nodes/word_budget_node.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/nodes/word_budget_node.py), this component analyzes template complexity and content depth to allocate word counts per chapter while establishing global writing constraints.

### ChapterGenerationNode

Theheavy lifting occurs in [`ReportEngine/nodes/chapter_generation_node.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/nodes/chapter_generation_node.py), which takes a `TemplateSection` and generation context to produce chapter JSON. It utilizes `utils/json_parser.RobustJSONParser` to handle malformed LLM outputs and supports configurable retry logic.

### GraphRAGQueryNode

When enabled, [`ReportEngine/nodes/graphrag_query_node.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/nodes/graphrag_query_node.py) queries the knowledge graph built from agent states, returning `graph_results` and `graph_enhancement_prompt` to augment chapter content with relationship-aware context.

## Core Utilities and IR Management

The utility layer ensures data integrity and transformation consistency throughout the pipeline.

### Template Parsing

[`ReportEngine/core/template_parser.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/core/template_parser.py) defines the `TemplateSection` dataclass and `parse_template_sections()` function, which extracts hierarchical section numbers, URL slugs, and content outlines from Markdown templates.

### IR Stitching

[`ReportEngine/core/stitcher.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/core/stitcher.py) contains `DocumentComposer`, which guarantees unique HTML anchors, assigns default `chapterId` values, and creates the final IR version stamped with `IR_VERSION` constants.

### Chapter Storage and Validation

[`ReportEngine/core/chapter_storage.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/core/chapter_storage.py) creates per-run directories for debugging intermediate outputs. Supporting validation occurs in [`ReportEngine/utils/chart_validator.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/utils/chart_validator.py) and [`chart_review_service.py`](https://github.com/666ghj/bettafish/blob/main/chart_review_service.py), which verify chart block well-formedness before rendering.

### Configuration

[`ReportEngine/utils/config.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/utils/config.py) centralizes settings including API keys, model names, output directories, and feature toggles like `GRAPHRAG_ENABLED`.

## Rendering Pipeline

The renderer layer converts the intermediate representation into human-readable formats.

### HTMLRenderer

[`ReportEngine/renderers/html_renderer.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/renderers/html_renderer.py) transforms IR into complete HTML pages, injecting CSS assets, anchor links, and dynamic table of contents navigation.

### MarkdownRenderer

[`ReportEngine/renderers/markdown_renderer.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/renderers/markdown_renderer.py) provides a fallback conversion that degrades unsupported block types (charts, word-clouds, widgets) into tables or textual placeholders, ensuring content accessibility even when advanced visualizations cannot render.

### PDFRenderer

[`ReportEngine/renderers/pdf_renderer.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/renderers/pdf_renderer.py) generates PDF documents by first rendering to HTML, then using `weasyprint` or headless browser engines to produce the final binary output.

## State Management and Flask API

[`ReportEngine/state/state.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/state/state.py) maintains the `ReportState` object, tracking metadata, generation timestamps, success/failure flags, and HTML content throughout the lifecycle.

The [`ReportEngine/flask_interface.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/flask_interface.py) exposes REST endpoints including `/run` to initiate generation and `/status/<task_id>` for polling progress, enabling integration with external microservices and frontend applications.

## End-to-End Workflow Example

### Basic Synchronous Usage

```python
from ReportEngine import ReportAgent
from config import settings

# Initialise the agent

agent = ReportAgent()

# Inputs from the three sub-engines

query_output   = "### Query engine summary …"

media_output   = "### Media analysis …"

insight_output = "### Insight findings …"

forum_logs     = "User A: …\nUser B: …"

# Generate report

result = agent.generate_report(
    query="武汉大学舆情分析",
    reports=[query_output, media_output, insight_output],
    forum_logs=forum_logs,
    custom_template="",   # Auto-select template

    save_report=True,
)

print("HTML length:", len(result["html_content"]))
print("Files:", result.get("html_path"), result.get("ir_path"))

```

### Streaming Progress Events

```python
def stream_handler(event_type, payload):
    # Forward to WebSocket or SSE client

    print(f"[{event_type}] {payload}")

html_result = agent.generate_report(
    query="武汉大学舆情分析",
    reports=[q, m, i],
    forum_logs=logs,
    stream_handler=stream_handler,
    save_report=False,
)

```

The `stream_handler` receives `stage`, `progress`, `chapter_chunk`, and `error` events emitted during the generation loop.

### Disabling GraphRAG for Lightweight Runs

```python

# Override at runtime

agent.config.GRAPHRAG_ENABLED = False

report = agent.generate_report(
    query="疫情舆情报告",
    reports=[q, m, i],
    forum_logs=logs,
    save_report=True,
)

```

When disabled, the engine skips graph building (lines ~600-630 of [`agent.py`](https://github.com/666ghj/bettafish/blob/main/agent.py)) and proceeds directly to chapter generation.

## Summary

- **The ReportEngine architecture comprises four layers**: Orchestration (ReportAgent), Processing Nodes (LLM-driven transformers), Core Utilities (IR management), and Renderers (format converters).
- **The pipeline is stateless between runs**, persisting all intermediate data to per-run directories under `OUTPUT_DIR/<report_id>/`.
- **ReportAgent.generate_report() implements an 11-step workflow** from input normalization through template selection, chapter generation, IR stitching, and final rendering.
- **Processing nodes inherit from BaseNode** and handle specific tasks like template selection, layout design, word budgeting, and GraphRAG enhancement.
- **IR stitching via DocumentComposer** guarantees unique anchors and versioned metadata before HTML/Markdown rendering.
- **Flask interface endpoints** (`/run`, `/status/<task_id>`) enable asynchronous integration with external services.

## Frequently Asked Questions

### How does ReportEngine handle malformed LLM outputs during chapter generation?

The `ChapterGenerationNode` in [`ReportEngine/nodes/chapter_generation_node.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/nodes/chapter_generation_node.py) implements retry logic wrapped around `utils/json_parser.RobustJSONParser`. When JSON parsing fails, content filters trigger, or outputs are too sparse, the node automatically attempts regeneration using fallback LLM clients (`json_rescue_clients`) to ensure valid chapter structure without manual intervention.

### What file formats can the ReportEngine generate?

According to the renderer implementations in `ReportEngine/renderers/`, the engine supports HTML via `HTMLRenderer`, Markdown via `MarkdownRenderer`, and PDF via `PDFRenderer` (which uses `weasyprint` or headless browser conversion). The Markdown renderer serves as a robust fallback that converts unsupported visual elements like charts into textual tables.

### Is the ReportEngine architecture stateless, and where are outputs stored?

Yes, the architecture is designed to be stateless between runs. All intermediate chapter JSON files, final HTML, IR documents, and status logs are written to a unique per-run directory structure under `self.config.OUTPUT_DIR/<report_id>/` as implemented in [`ReportEngine/core/chapter_storage.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/core/chapter_storage.py) and the `_save_report()` method of `ReportAgent`.

### How does the ReportEngine integrate with the three analysis agents (Query, Media, Insight)?

The engine receives raw string outputs from these agents via the `reports` parameter of `generate_report()`, which expects a list containing the Query, Media, and Insight outputs in sequence. The `_normalize_reports()` method standardizes these inputs before the `TemplateSelectionNode` and subsequent processing nodes synthesize them into coherent chapters, optionally enhanced by `GraphRAGQueryNode` when knowledge graph features are enabled.