# How to Troubleshoot Incomplete Outputs or Errors from BettaFish Agents

> Troubleshoot incomplete BettaFish agent outputs and errors by inspecting node execution LLM reliability and logs. Learn to fix crashes with targeted retries and configuration adjustments.

- Repository: [BaiFu/bettafish](https://github.com/666ghj/bettafish)
- Tags: how-to-guide
- Published: 2026-02-23

---

**When BettaFish agents return incomplete content or crash, trace the failure through three layers—node execution, LLM reliability, and logging—by inspecting stream events, log files, and state snapshots, then apply targeted retries or configuration adjustments.**

BettaFish is an autonomous research platform built around specialized agents like the **ReportEngine**, **QueryEngine**, and **Deep-Search Agent**. When these agents produce incomplete analysis or raise exceptions during execution, systematic troubleshooting requires understanding the pipeline architecture and inspecting specific failure points in the 666ghj/bettafish source code. This guide shows you how to diagnose and resolve agent failures using the actual implementation details from the repository.

## Understanding the Three Failure Layers

Every BettaFish agent follows a pipeline of nodes that call a large-language-model (LLM), process the response, and write results to a shared state. Failures cluster into three distinct architectural layers.

### Node Execution Layer

Individual nodes invoke the LLM and expect structured JSON payloads. When the model omits required keys like `title` or `hero`, the pipeline raises `ChapterJsonParseError` or `ChapterContentError`. The primary error handling logic resides in [`ReportEngine/agent.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/agent.py) within the `try/except` loops surrounding `chapter_generation_node.run` (lines 13,400–13,560). If you see exceptions mentioning malformed JSON or missing fields, the failure originated here.

### LLM and Tool Reliability Layer

Network flakiness, rate limits, and content moderation blocks trigger `RetryableError` exceptions. The [`utils/retry_helper.py`](https://github.com/666ghj/bettafish/blob/main/utils/retry_helper.py) file implements exponential back-off through the `with_retry` decorator (lines 57–112) and the `LLM_RETRY_CONFIG` configuration object (lines 27–33). HTTP timeouts or provider-side content filters manifest as repeated retry attempts with escalating delays before the agent finally surfaces the error.

### Logging and State Layer

Errors are persisted in per-engine log files and serialized state objects. The `ReportState` class in [`ReportEngine/state/state.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/state/state.py) stores progress and errors, while [`utils/knowledge_logger.py`](https://github.com/666ghj/bettafish/blob/main/utils/knowledge_logger.py) initializes the logging infrastructure. Symptoms like missing `forum.log` files or stale [`state.json`](https://github.com/666ghj/bettafish/blob/main/state.json) snapshots indicate problems at this persistence layer.

## Identifying the Failure Point

Before fixing the issue, locate exactly where the pipeline broke. BettaFish provides three diagnostic signals.

**Stream Events** – The front-end or CLI receives a stream of `stage`, `progress`, `chapter_status`, and `error` events (see `emit` calls in [`ReportEngine/agent.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/agent.py) lines 7,990–7,992). A `stage: agent_failed` or `chapter_status` with `status: error` confirms the failing node.

**Log Files** – Each engine writes to dedicated log files defined in [`config.py`](https://github.com/666ghj/bettafish/blob/main/config.py) (`REPORT_ENGINE_LOG_FILE`, `QUERY_ENGINE_LOG_FILE`). The `_setup_logging` method in [`ReportEngine/agent.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/agent.py) (lines 64–94) filters noise and flushes immediately. Open the relevant log and search for the last `WARNING` or `ERROR` entry.

**State Snapshots** – After each successful node, the system persists state via `self.state.mark_completed()` (line 7,071 in [`ReportEngine/agent.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/agent.py)). Inspect `final_reports/ir/<report_id>.json` or the `state_*.json` files produced by the Deep-Search Agent (`DeepSearchAgent/_save_report` lines 233–247) to verify which fields are missing or malformed.

## Common Root Causes and Fixes

| Root Cause | Diagnostic Signal | Solution |
|------------|-------------------|----------|
| **Malformed JSON Output** | `json.JSONDecodeError` in logs; missing keys in state snapshots | The `_run_stage_with_retry` method (line 13,340) catches parse errors. Increase `CHAPTER_JSON_MAX_ATTEMPTS` in [`config.py`](https://github.com/666ghj/bettafish/blob/main/config.py) or refine the prompt templates in `ReportEngine/prompts/`. |
| **Content Moderation Blocks** | `content_filter` retries; keywords like "inappropriate content" in logs | The `_should_retry_inappropriate_content_error` helper (lines 11,121–11,133) detects these. Configure a fallback LLM in `_initialize_rescue_llms` (lines 7,018–7,023) or adjust prompts to be more neutral. |
| **Network Timeouts** | `RetryableError` with exponential back-off delays | All HTTP calls use `with_retry` from [`utils/retry_helper.py`](https://github.com/666ghj/bettafish/blob/main/utils/retry_helper.py). Pass a custom `RetryConfig` with higher `max_retries` to the LLM client constructor. |
| **Missing Forum Logs** | GraphRAG stages fail with empty `forum.log` | The `init_knowledge_log(force_reset=True)` call (line 7,074) clears stale logs. Ensure [`ForumEngine/monitor.py`](https://github.com/666ghj/bettafish/blob/main/ForumEngine/monitor.py) is running and `LOG_FILE` points to a writable path. |
| **Incorrect Tool Parameters** | Date validation failures in search tools | The Deep-Search Agent validates dates via `_validate_date_format` in [`QueryEngine/agent.py`](https://github.com/666ghj/bettafish/blob/main/QueryEngine/agent.py) (line 775). Use `YYYY-MM-DD` format or extend the validation regex for custom formats. |

## Step-by-Step Troubleshooting Workflow

Follow this sequence to isolate and resolve agent failures:

1. **Check the UI or CLI stream** – Identify the last `stage` or `error` event emitted before the hang or crash.

2. **Open the corresponding log file** – Navigate to `logs/report_engine.log` or `logs/query_engine.log` and locate the most recent `WARNING` or `ERROR` entry.

3. **Inspect persisted state** – Open the latest `state_*.json` (Deep-Search) or `final_reports/ir/<report_id>.json`. Missing keys indicate JSON parsing failures.

4. **Re-run the failing node in isolation** – Many nodes expose a `run` method callable from a Python REPL for debugging:

```python
from ReportEngine.nodes import ChapterGenerationNode
from ReportEngine.llms import LLMClient
from ReportEngine.utils.config import settings

llm = LLMClient(
    api_key=settings.REPORT_ENGINE_API_KEY,
    model_name=settings.REPORT_ENGINE_MODEL_NAME,
    base_url=settings.REPORT_ENGINE_BASE_URL,
)
node = ChapterGenerationNode(llm, validator, storage, json_error_dir, error_log_dir)
payload = node.run(section, context, run_dir)

```

5. **Adjust configuration** – Increase retry limits, modify `MAX_CONTENT_LENGTH`, or swap the rescue LLM in `_initialize_rescue_llms`.

6. **Clear baseline files** – If `FileCountBaseline` detects no new outputs, delete [`logs/report_baseline.json`](https://github.com/666ghj/bettafish/blob/main/logs/report_baseline.json) to force a fresh scan (see `initialize_baseline` lines 12,212–12,230).

7. **Re-run with verbose logging** – Execute `python report_engine_only.py --verbose` to observe all stages in real-time.

## Debugging with Code Examples

### Wrapping Custom HTTP Calls with Retry Logic

When integrating external APIs, use the same retry mechanism as the core agents:

```python
from utils.retry_helper import with_retry, RetryConfig

# Custom config: 5 attempts, start at 2s, back-off factor 1.5

my_cfg = RetryConfig(max_retries=5, initial_delay=2.0, backoff_factor=1.5)

@with_retry(my_cfg)
def fetch_news(url: str) -> dict:
    import requests
    r = requests.get(url, timeout=10)
    r.raise_for_status()
    return r.json()

# Usage – automatically retries on network errors

response = fetch_news("https://api.tavily.com/v1/search?query=AI")

```

*(Source: [`utils/retry_helper.py`](https://github.com/666ghj/bettafish/blob/main/utils/retry_helper.py) lines 57–112)*

### Handling Malformed Chapter Payloads

Catch and handle JSON parsing errors programmatically:

```python
from ReportEngine.nodes import ChapterGenerationNode
from ReportEngine.llms import LLMClient
from ReportEngine.utils.config import settings

client = LLMClient(
    api_key=settings.REPORT_ENGINE_API_KEY,
    model_name=settings.REPORT_ENGINE_MODEL_NAME,
    base_url=settings.REPORT_ENGINE_BASE_URL,
)

node = ChapterGenerationNode(
    llm_client=client,
    validator=...,            # IRValidator instance

    chapter_storage=...,      # ChapterStorage instance

    fallback_llm_clients=[("report_engine", client)],
    error_log_dir=settings.JSON_ERROR_LOG_DIR,
)

try:
    chapter = node.run(section, context, run_dir)
except ChapterJsonParseError as e:
    logger.error(f"章节解析失败: {e}")
    # Invoke fallback for sparse content

    chapter = node.fallback_to_sparse(context)

```

*(Source: [`ReportEngine/agent.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/agent.py) error handling lines 13,400–13,560)*

### Forcing a Fresh File-Baseline Scan

When the agent incorrectly reports no new files:

```bash
rm logs/report_baseline.json   # delete old snapshot

python report_engine_only.py   # re-initializes baseline automatically

```

*(Source: [`ReportEngine/agent.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/agent.py) `FileCountBaseline.initialize_baseline` lines 12,212–12,230)*

### Listening to Stream Events in Custom Front-Ends

Build real-time monitoring by handling the event stream:

```python
def stream_handler(event_type, payload):
    if event_type == "error":
        print(f"[ERROR] {payload['message']}")
    elif event_type == "chapter_status":
        print(f"[{payload['status'].upper()}] {payload['title']} (attempt {payload.get('attempt',1)})")
    elif event_type == "progress":
        print(f"[PROGRESS] {payload['progress']}% – {payload['message']}")

agent = ReportAgent()
agent.generate_report(
    query="武汉大学舆情分析",
    reports=[q_md, m_md, i_md],
    stream_handler=stream_handler,
)

```

*(Source: [`ReportEngine/agent.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/agent.py) `emit` helper lines 7,979–7,992)*

## Summary

- BettaFish agent failures occur at three layers: **node execution** (JSON parsing), **LLM/tool reliability** (network/moderation), and **logging/state** (persistence).
- Diagnose issues by monitoring **stream events**, reading per-engine **log files** (`logs/report_engine.log`), and inspecting **state snapshots** (`state_*.json`).
- Resolve JSON parsing errors by increasing `CHAPTER_JSON_MAX_ATTEMPTS` or tightening prompts in `ReportEngine/prompts/`.
- Handle transient network failures using the `with_retry` decorator from [`utils/retry_helper.py`](https://github.com/666ghj/bettafish/blob/main/utils/retry_helper.py) with custom `RetryConfig` settings.
- Clear stale data by removing [`logs/report_baseline.json`](https://github.com/666ghj/bettafish/blob/main/logs/report_baseline.json) or calling `init_knowledge_log(force_reset=True)` to reset knowledge logs.

## Frequently Asked Questions

### Why does my agent return incomplete or malformed JSON chapters?

The LLM occasionally omits required keys like `title` or `hero`, triggering `ChapterJsonParseError` in [`ReportEngine/agent.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/agent.py) (lines 13,400–13,560). The `_run_stage_with_retry` wrapper attempts to recover, but if `CHAPTER_JSON_MAX_ATTEMPTS` (default 3) is exceeded, the node fails. Increase this limit in [`config.py`](https://github.com/666ghj/bettafish/blob/main/config.py) or add stricter JSON schema enforcement in the prompt templates.

### How do I fix content moderation blocks from the LLM provider?

When providers return "inappropriate content" errors, the `_should_retry_inappropriate_content_error` helper (lines 11,121–11,133) detects specific keywords and triggers a retry. If persistent, configure a fallback LLM in `_initialize_rescue_llms` (lines 7,018–7,023) or rewrite the query to avoid flagged topics. The system will automatically route sensitive requests to the fallback model.

### Where are agent errors logged in BettaFish?

Each engine writes to separate log files defined in [`config.py`](https://github.com/666ghj/bettafish/blob/main/config.py): `REPORT_ENGINE_LOG_FILE`, `QUERY_ENGINE_LOG_FILE`, etc. The `_setup_logging` method in [`ReportEngine/agent.py`](https://github.com/666ghj/bettafish/blob/main/ReportEngine/agent.py) (lines 64–94) ensures immediate flushing. For structured error tracking, inspect the `error_log_dir` specified in the `ChapterGenerationNode` constructor, which stores raw LLM outputs that failed JSON validation.

### Can I retry a specific failed node without running the full pipeline?

Yes. Most nodes expose a `run` method accepting the current `section`, `context`, and `run_dir`. Instantiate the node with the same `LLMClient` and validators used in the pipeline, then call `node.run()` directly from a Python REPL or script. This isolates the failure and displays the exact exception and raw LLM output without executing preceding stages.