How the Information Richness Rating System in AI Berkshire Prevents the Illusion of Certainty

The information richness rating system in AI Berkshire prevents the illusion of certainty by explicitly categorizing data availability into A, B, and C tiers, automatically tagging confidence levels, and triggering stricter anti-bias checks for low-information scenarios.

The xbtlin/ai-berkshire repository implements a proprietary information richness rating system that combats the cognitive bias equating data volume with predictive accuracy. By embedding an A/B/C classification directly into every research workflow, the system forces AI agents to declare uncertainty rather than fabricate confidence.

How the Information Richness Rating System Categorizes Data Availability

The rating system operates on three explicit tiers defined in README_EN.md (lines 89‑90):

• A‑level – Information is abundant. The system detects multiple public filings, extensive analyst coverage, and high‑frequency market data.
• B‑level – Data is moderate. Analysis relies on partial financial statements, limited analyst notes, and estimated metrics (e.g., "Pop Mart rated B: limited data, estimated metrics flagged with confidence levels").
• C‑level – Information is scarce. The target is a private company with no audited numbers, requiring heavy inference and primary research.

This rating is injected into the report header and repeatedly referenced in the conclusion. By making the quantity of data visible to readers, the system dismantles the assumption that longer outputs or more verbose analysis equate to higher certainty.

Automatic Confidence Tagging and Uncertainty Declaration

For every numeric estimate, the pipeline attaches a confidence label proportional to the rating tier. In skills/investment-research.md (lines 7‑9), the system enforces inclusion of this rating alongside an AI‑research limitation statement.

C‑level reports trigger a mandatory "need‑verification checklist" at the end of the document, listing questions that must be answered with primary research rather than inference. This splits the confidence assessment from the raw number, preventing the cognitive bias that a longer output feels more trustworthy.

Cross‑Validation Gates and Anti‑Bias Enforcement

The information richness rating acts as a gate for other anti‑bias mechanisms. According to the source code in README_EN.md (lines 85‑93), the rating determines which skepticism tests are applied:

• A‑level reports still run the Munger‑style inversion test.
• C‑level reports trigger the "Quick‑Kill Checklist" and "Intellectual Honesty" rules, explicitly marking unknowns instead of fabricating certainty.

The skills/portfolio-review.md skill (lines 45‑46) uses this rating to downgrade the weight of C‑level holdings, ensuring that the final portfolio reflects underlying data reliability rather than AI overconfidence.

Implementation in the AI Berkshire Codebase

The rating logic is implemented in Python functions that assess data completeness before report generation:

def rate_richness(info):
    """
    info: dict with keys:
        - filings (int)      # number of SEC/HK filings found

        - analyst_reports (int)
        - estimate_needed (bool)
    Returns: "A", "B", or "C"
    """
    if info["filings"] >= 5 and info["analyst_reports"] >= 3:
        return "A"
    if info["filings"] >= 2:
        return "B"
    return "C"

# Usage in a research skill

rating = rate_richness({
    "filings": 1,
    "analyst_reports": 0,
    "estimate_needed": True,
})
print(f"Information-Richness Rating: {rating}")   # → C

The system automatically injects the rating into report headers:

cat report.md | grep "信息丰富度评级"

# Output:

## 信息丰富度评级：C级（信息稀缺） — 未上市公司，财务数据仅为估算，需现场核实

For C‑level scenarios, a YAML verification checklist is auto‑generated:

verification_checklist:
  - "获取最新融资披露"
  - "采访核心管理层"
  - "核实供应链合同规模"

The skills/investment-team.md (line 23) further requires that the team lead assess each agent's "information richness" before synthesis, ensuring the rating propagates through the entire decision chain.

Summary

• Explicit categorization: The A/B/C rating makes data scarcity visible in report headers, preventing hidden assumptions about data quality.
• Confidence separation: Numeric estimates carry tier‑based confidence labels, with C‑level reports requiring verification checklists.
• Mechanical enforcement: The rating gates anti‑bias tests and portfolio weights, as implemented in portfolio-review.md and investment-research.md.
• Code‑level integration: The rate_richness() function and auto‑injection pipelines ensure consistent application across all research workflows.

Frequently Asked Questions

What is the illusion of certainty in AI investment research?

The illusion of certainty is the cognitive bias where larger data outputs or more verbose analysis create a false sense of confidence. AI models may generate detailed narratives that appear authoritative despite being built on sparse or estimated data. The information richness rating system combats this by forcing explicit declarations of data scarcity.

How does the A/B/C rating affect portfolio construction in AI Berkshire?

According to skills/portfolio-review.md, the rating directly influences position sizing. C‑level holdings receive downgraded weights in the final portfolio allocation, ensuring that capital deployment reflects the underlying reliability of the data rather than the AI's rhetorical confidence.

What happens when a report receives a C‑level information richness rating?

C‑level reports trigger mandatory safeguards including a "need‑verification checklist" requiring primary research answers, application of the "Quick‑Kill Checklist" to challenge assumptions, and explicit "Intellectual Honesty" rules that mark unknowns rather than fabricating certainty.

Where is the information richness rating system documented in the codebase?

The primary documentation resides in README_EN.md (lines 85‑93) with the Chinese equivalent in README.md. Implementation details appear in skills/investment-research.md (lines 7‑9) for report generation, skills/portfolio-review.md (lines 45‑46) for weight management, and skills/investment-team.md (line 23) for team‑lead assessment protocols.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}