Customizing Citation Verification Layers (arXiv, CrossRef, DataCite) in AutoResearchClaw

AutoResearchClaw validates every BibTeX entry through a configurable four-layer pipeline that assigns VERIFIED, SUSPICIOUS, HALLUCINATED, or SKIPPED status using arXiv IDs, CrossRef DOIs, and title-search fallbacks.

The researchclaw/literature/verify.py module implements a citation-verification engine designed to protect scientific integrity by confirming that every reference in a generated paper actually exists. By customizing the order, thresholds, and sources of these verification layers, you can balance accuracy against API rate limits and tailor the system to your specific domain.

Understanding the Four-Layer Verification Pipeline

The verification engine orchestrates four distinct layers in verify_citations (lines 663‑720). Each layer returns a CitationResult containing the resolved title, a VerifyStatus enum (defined at lines 43‑49), a confidence score, and the method used.

Layer 1 – arXiv ID Lookup

The fastest path triggers when a BibTeX entry contains an eprint field. The verify_by_arxiv_id function (lines 83‑128) queries the arXiv OAI API to perform a direct ID-to-metadata match. This layer minimizes network overhead for pre-print heavy bibliographies.

Layer 2 – DOI Verification (CrossRef/DataCite)

When a doi field is present, verify_by_doi (lines 358‑425) first attempts a CrossRef /works/{doi} lookup. For arXiv DOIs, the engine falls back to DataCite. This layer offers high rate limits and serves as the primary anchor for published literature.

Layer 3 – OpenAlex Title Search

If no DOI or arXiv ID exists, verify_by_openalex (lines 449‑527) executes a title search against the OpenAlex /works endpoint. With a generous allowance of 10,000 calls per day, this layer provides broad coverage before resorting to more expensive searches.

Layer 4 – Semantic Scholar and arXiv Title Search

The final fallback, verify_by_title_search (lines 777‑845), uses a unified search_papers wrapper that scans both Semantic Scholar and arXiv by title. This exhaustive layer catches edge cases but incurs the highest latency.

Core Data Models and Return Types

Each layer returns a CitationResult dataclass that encapsulates the original citation key, resolved Paper object (defined in researchclaw/literature/models.py lines 15‑78), verification status, and optional raw API details. The Paper model stores immutable metadata including authors, DOI, and arXiv ID, and can regenerate a clean BibTeX entry via to_bibtex().

The orchestration respects rate-limit-aware delays defined as _DELAY_ARXIV, _DELAY_CROSSREF, and _DELAY_OPENALEX, and enforces a global five-minute timeout to prevent pipeline stalls.

Customizing the Verification Layers

Reordering or Disabling Layers

To change the verification priority, edit the conditional blocks inside verify_citations (around lines 779‑845). The default order checks DOI first, then OpenAlex, then arXiv ID, and finally title search.

To prioritize arXiv ID for pre-print workflows, move the block beginning at line 794 (if result is None and arxiv_id:) ahead of the DOI verification block. This reduces latency when most citations originate from arXiv.

Adding a New Verification Source

Implement a new verification function following the signature verify_by_<source>(doi: str, expected_title: str) -> CitationResult | None. Insert the call into the verify_citations loop, respecting the delay pattern:

def verify_by_custom_api(doi: str, expected_title: str) -> CitationResult | None:
    # Query your proprietary database

    # Return CitationResult on match, None on failure

    ...

# Inside verify_citations, after existing layers:

if result is None:
    if api_call_count > 0:
        time.sleep(_DELAY_CUSTOM_API)
    result = verify_by_custom_api(doi, title)
    api_call_count += 1

Tuning Confidence Thresholds

The status decision logic uses hard-coded similarity cut-offs. Adjust these thresholds at the return points in each layer to make the engine stricter or more permissive:

• arXiv layer: Lines 138‑144 (sim >= 0.80 → VERIFIED)
• DOI layer: Lines 136‑144
• OpenAlex layer: Lines 101‑107
• Title search layer: Lines 127‑133

Extending Cache Behavior

The verification cache persists to ~/.cache/researchclaw/citation_verify. To store additional fields such as raw API payloads, override the _read_cache and _write_cache helper functions (lines 440‑474).

Integrating Verification into Your Pipeline

The verification report feeds into downstream quality gates. filter_verified_bibtex (lines 667‑701) prunes a BibTeX file to retain only VERIFIED or optionally SUSPICIOUS entries. The annotate_paper_hallucinations function (lines 504‑557) scrubs hallucinated citations from manuscript text, handling both LaTeX \cite{} and Markdown [key] syntax.

These utilities are invoked during the Paper Writing stage (_paper_writing.py, line 2021) and the Publish Review stage (_review_publish.py, line 2664), ensuring hallucinated references never reach final output.

Code Examples

Running Basic Verification

from researchclaw.literature.verify import verify_citations, filter_verified_bibtex

# Load raw BibTeX from file

with open("references.bib", "r", encoding="utf-8") as f:
    raw_bib = f.read()

# Execute 4-layer verification (OpenAlex requires no API key)

report = verify_citations(raw_bib, inter_verify_delay=0.5)

print(f"Integrity score: {report.integrity_score}")
print(f"Verified: {report.verified}/{report.total}")

# Export clean bibliography

clean_bib = filter_verified_bibtex(raw_bib, report, include_suspicious=False)
with open("references_clean.bib", "w", encoding="utf-8") as f:
    f.write(clean_bib)

Prioritizing arXiv ID Checks

To reconfigure the pipeline to check arXiv IDs before DOIs, modify researchclaw/literature/verify.py:

# Locate the verification block (lines ~779-845)

# Move the arXiv ID block above the DOI block:

if result is None and arxiv_id:
    # Add delay if not first call

    result = verify_by_arxiv_id(arxiv_id, title)
    api_call_count += 1

if result is None and doi:
    # CrossRef/DataCite verification

    ...

This configuration exploits arXiv's low-latency API for pre-print validation before hitting CrossRef.

Implementing a Custom API Source

Add Microsoft Academic or another proprietary database:

def verify_by_msacademic(doi: str, expected_title: str) -> CitationResult | None:
    """Query Microsoft Academic Graph API."""
    # Implement request logic using urllib.request

    # Parse response and return CitationResult if matched

    pass

# Insert into verify_citations orchestration:

if result is None:
    if api_call_count > 0:
        time.sleep(_DELAY_MSACADEMIC)  # Define constant (e.g., 0.4)

    result = verify_by_msacademic(doi, title)
    api_call_count += 1

Summary

• AutoResearchClaw provides a four-layer verification engine in researchclaw/literature/verify.py that classifies citations as VERIFIED, SUSPICIOUS, HALLUCINATED, or SKIPPED.
• The pipeline prioritizes DOI and arXiv ID lookups before falling back to OpenAlex and Semantic Scholar title searches.
• You can reorder layers by adjusting the conditional blocks in verify_citations to optimize for your specific citation distribution.
• Confidence thresholds are hard-coded in each layer and can be tuned to control strictness.
• The system includes caching at ~/.cache/researchclaw/citation_verify and utilities to filter hallucinated citations from final manuscripts.

Frequently Asked Questions

What are the rate limits for each verification layer?

arXiv implements strict rate limiting requiring inter-request delays of approximately three seconds. CrossRef offers more generous limits suitable for bulk DOI verification. OpenAlex permits roughly 10,000 calls per day, making it ideal for title-based fallback searches. The engine enforces these limits through _DELAY_ARXIV, _DELAY_CROSSREF, and _DELAY_OPENALEX constants.

How does AutoResearchClaw handle hallucinated citations?

The annotate_paper_hallucinations function (lines 504‑557) identifies citations marked as HALLUCINATED in the verification report and removes their corresponding \cite{} or [key] references from the manuscript text. During the paper writing stage, filter_verified_bibtex (lines 667‑701) can optionally strip unverified entries from the bibliography entirely.

Can I use the verification engine standalone without the full pipeline?

Yes. Import verify_citations and filter_verified_bibtex directly from researchclaw.literature.verify. The module has no dependencies on the broader AutoResearchClaw agent architecture, requiring only the Paper and CitationResult models from researchclaw/literature/models.py.

Where does the citation cache store its data?

The cache persists to ~/.cache/researchclaw/citation_verify as JSON files. The _read_cache and _write_cache functions (lines 440‑474) manage serialization. You can extend these functions to cache additional API metadata or move the cache location by modifying the path generation logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}