Integrating MetaClaw Bridge for Cross-Run Learning in AutoResearchClaw

The MetaClaw bridge in AutoResearchClaw enables cross-run learning by capturing failure lessons from each pipeline execution, converting them into reusable skills, and injecting session-aware headers into LLM requests to improve subsequent research runs.

AutoResearchClaw (ARC) provides a MetaClaw bridge that allows the platform to learn from previous experiment outcomes and apply that knowledge to future runs. This integration transforms isolated pipeline executions into a continuous learning system where high-severity failures are automatically converted into persistent skills. By configuring the bridge in researchclaw/config.py, researchers can enable cross-run meta-learning that improves retrieval accuracy and citation quality over time.

How the MetaClaw Bridge Works

The bridge operates through seven coordinated mechanisms that span the entire pipeline lifecycle:

1. Session Initialization – Creates a MetaClawSession object at run start that generates a unique session_id for tracking all LLM calls within a single execution.

2. Request Header Injection – Appends MetaClaw-specific HTTP headers (session ID, stage name) to every LLM request when the bridge is enabled.

3. Lesson-to-Skill Conversion – Transforms high-severity failure lessons into reusable MetaClaw skills after each pipeline stage completes.

4. Skill Effectiveness Tracking – Records success and failure metrics for each generated skill to enable data-driven skill evolution.

5. PRM Quality Gating – Applies MetaClaw's Performance-Risk-Metric (PRM) scorer to validate stage outputs before acceptance.

6. Stage-to-Skill Mapping – Maps ARC pipeline stages to appropriate MetaClaw skill categories to ensure relevant skill generation.

7. Lifecycle Hooks – Orchestrates the entire flow through hooks in the pipeline runner that trigger at run start and completion.

Configuration

All bridge settings are centralized in researchclaw/config.py under the MetaClawBridgeConfig class (defined at line 32). The configuration supports proxy routing, quality thresholds, and skill storage locations.

Key configuration parameters include:

• enabled – Boolean switch to activate the entire bridge (default: False)
• proxy_url – MetaClaw proxy endpoint for request interception (OpenAI-compatible)
• fallback_url – Direct LLM endpoint if the proxy is unreachable
• prm – PRM gate thresholds for output validation
• lesson_to_skill – Severity thresholds and skill directory paths

Enable the bridge by adding this to your config.yaml:

metaclaw_bridge:
  enabled: true
  proxy_url: "http://localhost:8000/v1"
  fallback_url: "https://api.openai.com/v1"
  prm:
    min_score: 0.7
  lesson_to_skill:
    min_severity: "high"
    skills_dir: "~/.metaclaw/skills"

Implementation Details

Session Management

The MetaClawSession class in researchclaw/metaclaw_bridge/session.py (constructor at line 15) manages the lifecycle of a cross-run learning session. When instantiated, it generates a unique identifier that associates all subsequent LLM calls with the current pipeline execution.

At the start of each run, the pipeline runner creates this session:

from researchclaw.metaclaw_bridge.session import MetaClawSession

# researchclaw/pipeline/runner.py line 1798

session = MetaClawSession(run_id="run-2024-05-28-01")

The session object provides get_headers() to generate the HTTP headers required for MetaClaw tracking, including the session ID and current stage name.

Header Injection

The LLM client in researchclaw/llm/client.py integrates MetaClaw at line 470 by checking the global configuration and rewriting requests to route through the MetaClaw proxy. When the bridge is enabled, the client retrieves headers from the active session and injects them into every outgoing request:

# Simplified from researchclaw/llm/client.py line 470

headers = {
    **default_headers,
    **session.get_headers(stage_name="search")
}
response = httpx.post(proxy_url, json=payload, headers=headers)

This ensures MetaClaw can correlate specific LLM calls with pipeline stages and outcomes.

Lesson-to-Skill Conversion

After each stage completes, ARC collects LessonEntry objects representing failures or inefficiencies. The convert_lessons_to_skills() function in researchclaw/metaclaw_bridge/lesson_to_skill.py (line 137) processes these entries to generate persistent skill files.

The conversion pipeline:

1. Formats raw lessons using _format_lessons (line 61)
2. Calls the MetaClaw LLM to generate skill definitions
3. Writes *.skill files to the configured skills directory (default: ~/.metaclaw/skills/)

from researchclaw.metaclaw_bridge.lesson_to_skill import convert_lessons_to_skills

# researchclaw/metaclaw_bridge/lesson_to_skill.py line 137

new_skills = convert_lessons_to_skills(
    lessons=high_severity_lessons,
    skills_dir=Path("~/.metaclaw/skills"),
    session=session,
    stage_name="citation_extraction"
)

Skill Feedback Loop

To enable continuous improvement, researchclaw/metaclaw_bridge/skill_feedback.py provides the SkillFeedbackStore class (line 32) and record_stage_skills() function (line 98). These components aggregate success metrics, latency data, and failure rates for each skill applied during a run.

from researchclaw.metaclaw_bridge.skill_feedback import SkillFeedbackStore

# researchclaw/metaclaw_bridge/skill_feedback.py line 32

store = SkillFeedbackStore(Path("~/.metaclaw/skill_feedback.json"))

# researchclaw/metaclaw_bridge/skill_feedback.py line 98

store.record_stage_skills(
    stage_name="search",
    skills_applied=["arc-search-improvement"],
    success=True,
    latency_ms=212
)

This feedback allows MetaClaw to prioritize high-performing skills and deprecate ineffective ones across subsequent runs.

PRM Quality Gates

The PRM (Performance-Risk-Metric) gate in researchclaw/metaclaw_bridge/prm_gate.py validates stage outputs before they are accepted into the final research artifact. The MetaClawPRMConfig factory (referenced at line 130) configures thresholds that determine whether a stage output passes quality checks.

from researchclaw.metaclaw_bridge.prm_gate import MetaClawPRMScorer

scorer = MetaClawPRMScorer(session=session)
score = scorer.evaluate(stage_output)

if score < 0.7:
    raise RuntimeError("PRM gate rejected stage output")

Low-scoring outputs trigger retry logic or failure handling, ensuring only high-quality results contribute to the skill learning process.

Stage-to-Skill Mapping

To ensure relevant skill generation, researchclaw/metaclaw_bridge/stage_skill_map.py maintains a mapping table between ARC pipeline stages and MetaClaw skill categories. The get_stage_config() function (line 143) returns the appropriate MetaClaw task type for any given ARC stage name, ensuring that search failures generate retrieval skills while citation errors generate verification skills.

Code Examples

Enabling the Bridge in Configuration

Create or modify your config.yaml to activate cross-run learning:

metaclaw_bridge:
  enabled: true
  proxy_url: "http://localhost:8000/v1"
  fallback_url: "https://api.openai.com/v1"
  prm:
    min_score: 0.6
  lesson_to_skill:
    min_severity: "high"
    skills_dir: "~/.metaclaw/skills"

Manual Session Control

For custom pipeline integrations, manually manage the MetaClaw session lifecycle:

from researchclaw.metaclaw_bridge.session import MetaClawSession

# Initialize at run start

session = MetaClawSession(run_id="custom-run-001")

# Use throughout pipeline...

# session.get_headers(stage_name="your_stage")

# Clean up at run end

session.end()

Converting Lessons to Skills Programmatically

Process failure lessons outside the standard pipeline hooks:

from pathlib import Path
from researchclaw.metaclaw_bridge.lesson_to_skill import convert_lessons_to_skills

lessons = [
    {"severity": "high", "description": "Retrieved outdated paper", "stage": "search"},
    {"severity": "critical", "description": "Hallucinated citation", "stage": "citation"}
]

skills = convert_lessons_to_skills(
    lessons=lessons,
    skills_dir=Path("~/.metaclaw/skills"),
    session=session,
    stage_name="post_processing"
)

print(f"Generated {len(skills)} new skills: {[s.name for s in skills]}")

Recording Skill Effectiveness

Track how well specific skills perform during execution:

from researchclaw.metaclaw_bridge.skill_feedback import SkillFeedbackStore

store = SkillFeedbackStore(Path("~/.metaclaw/skill_feedback.json"))

# After applying a skill

store.append({
    "skill_name": "arc-citation-verification",
    "run_id": "run-2024-05-28-01",
    "stage": "citation_extraction",
    "success": True,
    "latency_ms": 145
})

Summary

• MetaClawSession in researchclaw/metaclaw_bridge/session.py (line 15) provides the core session management that tracks cross-run learning contexts.
• Header injection at line 470 of researchclaw/llm/client.py ensures every LLM request carries MetaClaw tracking metadata.
• Lesson conversion via convert_lessons_to_skills() in lesson_to_skill.py (line 137) transforms failures into persistent skill files stored in ~/.metaclaw/skills/.
• Feedback collection through SkillFeedbackStore (line 32 of skill_feedback.py) enables data-driven skill optimization across runs.
• Quality gating using the PRM scorer in prm_gate.py (line 130) validates outputs before they influence the skill library.
• Configuration is centralized in researchclaw/config.py (line 32) under the MetaClawBridgeConfig class, allowing simple YAML-based activation.

Frequently Asked Questions

What is the MetaClaw bridge in AutoResearchClaw?

The MetaClaw bridge is an integration layer that connects AutoResearchClaw to the MetaClaw meta-learning system. It captures failure patterns from research pipeline executions, converts them into reusable skills, and applies those skills to improve LLM interactions in subsequent runs. According to the source code in researchclaw/metaclaw_bridge/, the bridge manages session tracking, header injection, and skill persistence.

How does cross-run learning improve research quality?

Cross-run learning improves quality by accumulating institutional knowledge across experiments. When the bridge is enabled, high-severity failures from previous runs—such as retrieving irrelevant papers or generating hallucinated citations—are converted into skills that modify future LLM prompts. The SkillFeedbackStore in skill_feedback.py (line 32) tracks which skills actually improve outcomes, allowing the system to learn which interventions work best for specific research stages.

Where are MetaClaw skills stored and how are they formatted?

Skills are stored as *.skill files in the directory specified by lesson_to_skill.skills_dir (default: ~/.metaclaw/skills/). The convert_lessons_to_skills() function in lesson_to_skill.py (line 137) generates these files by formatting lesson data and prompting the MetaClaw LLM to create structured skill definitions. Each skill file contains the learned pattern, applicable stage names, and intervention logic for preventing similar failures.

Can I use the MetaClaw bridge with custom LLM endpoints?

Yes. The bridge supports custom endpoints through the proxy_url and fallback_url configuration options in MetaClawBridgeConfig (line 32 of config.py). Set proxy_url to your MetaClaw proxy instance and fallback_url to your direct LLM provider (e.g., OpenAI). If the proxy is unreachable, the client automatically falls back to the direct endpoint while logging the connectivity issue.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}