How to Troubleshoot Incomplete Outputs or Errors from BettaFish Agents

When BettaFish agents return incomplete content or crash, trace the failure through three layers—node execution, LLM reliability, and logging—by inspecting stream events, log files, and state snapshots, then apply targeted retries or configuration adjustments.

BettaFish is an autonomous research platform built around specialized agents like the ReportEngine, QueryEngine, and Deep-Search Agent. When these agents produce incomplete analysis or raise exceptions during execution, systematic troubleshooting requires understanding the pipeline architecture and inspecting specific failure points in the 666ghj/bettafish source code. This guide shows you how to diagnose and resolve agent failures using the actual implementation details from the repository.

Understanding the Three Failure Layers

Every BettaFish agent follows a pipeline of nodes that call a large-language-model (LLM), process the response, and write results to a shared state. Failures cluster into three distinct architectural layers.

Node Execution Layer

Individual nodes invoke the LLM and expect structured JSON payloads. When the model omits required keys like title or hero, the pipeline raises ChapterJsonParseError or ChapterContentError. The primary error handling logic resides in ReportEngine/agent.py within the try/except loops surrounding chapter_generation_node.run (lines 13,400–13,560). If you see exceptions mentioning malformed JSON or missing fields, the failure originated here.

LLM and Tool Reliability Layer

Network flakiness, rate limits, and content moderation blocks trigger RetryableError exceptions. The utils/retry_helper.py file implements exponential back-off through the with_retry decorator (lines 57–112) and the LLM_RETRY_CONFIG configuration object (lines 27–33). HTTP timeouts or provider-side content filters manifest as repeated retry attempts with escalating delays before the agent finally surfaces the error.

Logging and State Layer

Errors are persisted in per-engine log files and serialized state objects. The ReportState class in ReportEngine/state/state.py stores progress and errors, while utils/knowledge_logger.py initializes the logging infrastructure. Symptoms like missing forum.log files or stale state.json snapshots indicate problems at this persistence layer.

Identifying the Failure Point

Before fixing the issue, locate exactly where the pipeline broke. BettaFish provides three diagnostic signals.

Stream Events – The front-end or CLI receives a stream of stage, progress, chapter_status, and error events (see emit calls in ReportEngine/agent.py lines 7,990–7,992). A stage: agent_failed or chapter_status with status: error confirms the failing node.

Log Files – Each engine writes to dedicated log files defined in config.py (REPORT_ENGINE_LOG_FILE, QUERY_ENGINE_LOG_FILE). The _setup_logging method in ReportEngine/agent.py (lines 64–94) filters noise and flushes immediately. Open the relevant log and search for the last WARNING or ERROR entry.

State Snapshots – After each successful node, the system persists state via self.state.mark_completed() (line 7,071 in ReportEngine/agent.py). Inspect final_reports/ir/<report_id>.json or the state_*.json files produced by the Deep-Search Agent (DeepSearchAgent/_save_report lines 233–247) to verify which fields are missing or malformed.

Common Root Causes and Fixes

Root Cause	Diagnostic Signal	Solution
Malformed JSON Output	json.JSONDecodeError in logs; missing keys in state snapshots	The _run_stage_with_retry method (line 13,340) catches parse errors. Increase CHAPTER_JSON_MAX_ATTEMPTS in config.py or refine the prompt templates in ReportEngine/prompts/.
Content Moderation Blocks	content_filter retries; keywords like "inappropriate content" in logs	The _should_retry_inappropriate_content_error helper (lines 11,121–11,133) detects these. Configure a fallback LLM in _initialize_rescue_llms (lines 7,018–7,023) or adjust prompts to be more neutral.
Network Timeouts	RetryableError with exponential back-off delays	All HTTP calls use with_retry from utils/retry_helper.py. Pass a custom RetryConfig with higher max_retries to the LLM client constructor.
Missing Forum Logs	GraphRAG stages fail with empty forum.log	The init_knowledge_log(force_reset=True) call (line 7,074) clears stale logs. Ensure ForumEngine/monitor.py is running and LOG_FILE points to a writable path.
Incorrect Tool Parameters	Date validation failures in search tools	The Deep-Search Agent validates dates via _validate_date_format in QueryEngine/agent.py (line 775). Use YYYY-MM-DD format or extend the validation regex for custom formats.

Step-by-Step Troubleshooting Workflow

Follow this sequence to isolate and resolve agent failures:

1. Check the UI or CLI stream – Identify the last stage or error event emitted before the hang or crash.

2. Open the corresponding log file – Navigate to logs/report_engine.log or logs/query_engine.log and locate the most recent WARNING or ERROR entry.

3. Inspect persisted state – Open the latest state_*.json (Deep-Search) or final_reports/ir/<report_id>.json. Missing keys indicate JSON parsing failures.

4. Re-run the failing node in isolation – Many nodes expose a run method callable from a Python REPL for debugging:

from ReportEngine.nodes import ChapterGenerationNode
from ReportEngine.llms import LLMClient
from ReportEngine.utils.config import settings

llm = LLMClient(
    api_key=settings.REPORT_ENGINE_API_KEY,
    model_name=settings.REPORT_ENGINE_MODEL_NAME,
    base_url=settings.REPORT_ENGINE_BASE_URL,
)
node = ChapterGenerationNode(llm, validator, storage, json_error_dir, error_log_dir)
payload = node.run(section, context, run_dir)

5. Adjust configuration – Increase retry limits, modify MAX_CONTENT_LENGTH, or swap the rescue LLM in _initialize_rescue_llms.

6. Clear baseline files – If FileCountBaseline detects no new outputs, delete logs/report_baseline.json to force a fresh scan (see initialize_baseline lines 12,212–12,230).

7. Re-run with verbose logging – Execute python report_engine_only.py --verbose to observe all stages in real-time.

Debugging with Code Examples

Wrapping Custom HTTP Calls with Retry Logic

When integrating external APIs, use the same retry mechanism as the core agents:

from utils.retry_helper import with_retry, RetryConfig

# Custom config: 5 attempts, start at 2s, back-off factor 1.5

my_cfg = RetryConfig(max_retries=5, initial_delay=2.0, backoff_factor=1.5)

@with_retry(my_cfg)
def fetch_news(url: str) -> dict:
    import requests
    r = requests.get(url, timeout=10)
    r.raise_for_status()
    return r.json()

# Usage – automatically retries on network errors

response = fetch_news("https://api.tavily.com/v1/search?query=AI")

(Source: utils/retry_helper.py lines 57–112)

Handling Malformed Chapter Payloads

Catch and handle JSON parsing errors programmatically:

from ReportEngine.nodes import ChapterGenerationNode
from ReportEngine.llms import LLMClient
from ReportEngine.utils.config import settings

client = LLMClient(
    api_key=settings.REPORT_ENGINE_API_KEY,
    model_name=settings.REPORT_ENGINE_MODEL_NAME,
    base_url=settings.REPORT_ENGINE_BASE_URL,
)

node = ChapterGenerationNode(
    llm_client=client,
    validator=...,            # IRValidator instance

    chapter_storage=...,      # ChapterStorage instance

    fallback_llm_clients=[("report_engine", client)],
    error_log_dir=settings.JSON_ERROR_LOG_DIR,
)

try:
    chapter = node.run(section, context, run_dir)
except ChapterJsonParseError as e:
    logger.error(f"章节解析失败: {e}")
    # Invoke fallback for sparse content

    chapter = node.fallback_to_sparse(context)

(Source: ReportEngine/agent.py error handling lines 13,400–13,560)

Forcing a Fresh File-Baseline Scan

When the agent incorrectly reports no new files:

rm logs/report_baseline.json   # delete old snapshot

python report_engine_only.py   # re-initializes baseline automatically

(Source: ReportEngine/agent.py FileCountBaseline.initialize_baseline lines 12,212–12,230)

Listening to Stream Events in Custom Front-Ends

Build real-time monitoring by handling the event stream:

def stream_handler(event_type, payload):
    if event_type == "error":
        print(f"[ERROR] {payload['message']}")
    elif event_type == "chapter_status":
        print(f"[{payload['status'].upper()}] {payload['title']} (attempt {payload.get('attempt',1)})")
    elif event_type == "progress":
        print(f"[PROGRESS] {payload['progress']}% – {payload['message']}")

agent = ReportAgent()
agent.generate_report(
    query="武汉大学舆情分析",
    reports=[q_md, m_md, i_md],
    stream_handler=stream_handler,
)

(Source: ReportEngine/agent.py emit helper lines 7,979–7,992)

Summary

• BettaFish agent failures occur at three layers: node execution (JSON parsing), LLM/tool reliability (network/moderation), and logging/state (persistence).
• Diagnose issues by monitoring stream events, reading per-engine log files (logs/report_engine.log), and inspecting state snapshots (state_*.json).
• Resolve JSON parsing errors by increasing CHAPTER_JSON_MAX_ATTEMPTS or tightening prompts in ReportEngine/prompts/.
• Handle transient network failures using the with_retry decorator from utils/retry_helper.py with custom RetryConfig settings.
• Clear stale data by removing logs/report_baseline.json or calling init_knowledge_log(force_reset=True) to reset knowledge logs.

Frequently Asked Questions

Why does my agent return incomplete or malformed JSON chapters?

The LLM occasionally omits required keys like title or hero, triggering ChapterJsonParseError in ReportEngine/agent.py (lines 13,400–13,560). The _run_stage_with_retry wrapper attempts to recover, but if CHAPTER_JSON_MAX_ATTEMPTS (default 3) is exceeded, the node fails. Increase this limit in config.py or add stricter JSON schema enforcement in the prompt templates.

How do I fix content moderation blocks from the LLM provider?

When providers return "inappropriate content" errors, the _should_retry_inappropriate_content_error helper (lines 11,121–11,133) detects specific keywords and triggers a retry. If persistent, configure a fallback LLM in _initialize_rescue_llms (lines 7,018–7,023) or rewrite the query to avoid flagged topics. The system will automatically route sensitive requests to the fallback model.

Where are agent errors logged in BettaFish?

Each engine writes to separate log files defined in config.py: REPORT_ENGINE_LOG_FILE, QUERY_ENGINE_LOG_FILE, etc. The _setup_logging method in ReportEngine/agent.py (lines 64–94) ensures immediate flushing. For structured error tracking, inspect the error_log_dir specified in the ChapterGenerationNode constructor, which stores raw LLM outputs that failed JSON validation.

Can I retry a specific failed node without running the full pipeline?

Yes. Most nodes expose a run method accepting the current section, context, and run_dir. Instantiate the node with the same LLMClient and validators used in the pipeline, then call node.run() directly from a Python REPL or script. This isolates the failure and displays the exact exception and raw LLM output without executing preceding stages.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}