How Article-Analyzer Parses Wikis and Extracts Implicit Relationships (Karpathy Pattern)

The article-analyzer agent in Egonex-AI/Understand-Anything uses a two-stage pipeline: first, a deterministic parser extracts explicit wikilinks and structure from Markdown files, then an LLM infers implicit entities, claims, and semantic relationships like builds_on and contradicts.

The Karpathy pattern refers to a knowledge management approach that combines explicit wiki linking with AI-driven semantic analysis. In the Understand-Anything framework, this pattern is implemented through a sophisticated agent-based architecture that transforms static Markdown wikis into rich, queryable knowledge graphs. The article-analyzer component specifically handles the extraction of implicit relationships that go beyond the explicit [[wikilinks]] found in source files.

Deterministic Wiki Parsing (The Foundation)

Before any LLM inference occurs, the project-scanner agent performs a deterministic extraction of explicit relationships from the wiki structure.

Detecting Wiki Structure

The parser identifies a valid wiki by scanning for an index.md file alongside a collection of .md files within the target directory. According to the repository README, this structure triggers the "understand-knowledge" skill pipeline described in understand-anything-plugin/skills/understand-knowledge/SKILL.md.

Extracting Explicit Connections

For each Markdown file discovered, the scanner extracts:

• Front-matter metadata including titles and tags
• Wikilinks using the [[target]] syntax, which become related edges between article nodes
• Category hierarchy derived from folder structure and headings within index.md

The output of this phase is a knowledge-meta JSON containing article nodes and explicit related edges. This deterministic step ensures that every explicit link in the wiki is preserved as a navigable connection in the final graph.

LLM-Driven Implicit Extraction (The Karpathy Pattern)

The article-analyzer agent receives the parsed wiki data and performs semantic inference to discover hidden relationships. It processes articles in batches, receiving for each article: ID, name, summary, existing wikilinks, category, and a truncated body, along with the full list of existing node IDs.

Entity Detection

The LLM identifies named entities that lack dedicated wiki pages—such as people, tools, or academic papers—and creates new entity nodes for these concepts. This ensures that important nouns mentioned in passing become first-class nodes in the knowledge graph.

Claim Extraction

The system surfaces explicit assertions, architectural decisions, and factual statements as claim nodes. These capture declarative knowledge that might otherwise remain buried in prose, making them individually addressable and verifiable.

Implicit Relationship Discovery

Beyond the explicit related edges from wikilinks, the analyzer emits semantic relationships with confidence weights. The supported relationship types defined in understand-anything-plugin/agents/article-analyzer.md include:

• builds_on – Indicates conceptual dependency or extension
• contradicts – Marks opposing viewpoints or conflicting claims
• exemplifies – Links general concepts to specific instances
• authored_by – Attributes content to specific creators
• cites – References external sources or prior work

The agent follows strict deduplication rules: it never duplicates existing related edges, only creates relationships when clear textual evidence exists, and de-duplicates entities across the processing batch. The output is a JSON file containing new entity and claim nodes plus the inferred edges, which src/context-builder.ts merges into the final knowledge graph.

Implementation and Code Examples

Execute the full pipeline programmatically using the core SDK:

import { runSkill } from '@understand-anything/core';

// Stage 1: Scan project and parse wiki structure
await runSkill('project-scanner', {
  target: './my-wiki',
  skill: 'understand-knowledge',
});

// Stage 2: Extract implicit relationships via LLM
await runSkill('article-analyzer', {
  batchSize: 15,        // Articles per LLM call
  maxTokens: 3000,      // Content truncation limit
});

// Output: .understand-anything/knowledge-graph.json

For command-line usage, the repository provides a thin wrapper:

pnpm exec understand --full ./my-wiki

This command automatically executes the sequence: project-scanner → wiki parsing → article-analyzer → graph assembly.

Key Files and Architecture

Understanding the source layout helps when customizing or debugging the pipeline:

• understand-anything-plugin/agents/article-analyzer.md – Defines the LLM prompt specifications, extraction rules, and supported edge types for implicit relationship discovery.
• understand-anything-plugin/skills/understand-knowledge/SKILL.md – Configures the skill that wires the deterministic parser and article-analyzer into a unified pipeline.
• README.md (repository root) – Documents the Karpathy-pattern wiki workflow and overall system architecture.
• src/context-builder.ts – Merges deterministic parsing output with article-analyzer results, handling deduplication and edge weighting.
• scripts/generate-large-graph.mjs – Utility for generating synthetic graphs with implicit edges for performance benchmarking.

Summary

• The article-analyzer implements a hybrid approach: deterministic parsing preserves explicit wikilinks, while LLM inference extracts implicit semantic relationships.
• The Karpathy pattern is realized through three extraction tasks: entity detection, claim extraction, and implicit relationship discovery.
• Supported semantic edges include builds_on, contradicts, exemplifies, authored_by, and cites, each weighted by confidence.
• The pipeline is configurable via batchSize and maxTokens parameters to balance cost and coverage.
• Output from understand-anything-plugin/agents/article-analyzer.md is merged by src/context-builder.ts into the final knowledge graph.

Frequently Asked Questions

What is the Karpathy pattern in knowledge management?

The Karpathy pattern combines explicit wiki linking (using [[wikilink]] syntax) with AI-driven semantic analysis to build knowledge graphs. As implemented in Egonex-AI/Understand-Anything, this pattern treats explicit links as ground truth while using LLMs to infer hidden relationships, entities, and claims that connect concepts not explicitly linked in the source Markdown.

How does article-analyzer differ from the project-scanner agent?

The project-scanner performs deterministic parsing of Markdown files to extract explicit wikilinks and folder hierarchies, outputting a base graph. The article-analyzer is an LLM-enhanced agent that processes this base graph to add implicit relationships, entities, and claims that require semantic understanding of the text content.

What relationship types does the article-analyzer extract?

According to the agent specification in understand-anything-plugin/agents/article-analyzer.md, the system extracts five semantic relationship types: builds_on for dependencies, contradicts for opposing views, exemplifies for instances of concepts, authored_by for attribution, and cites for references. Each edge includes a weight indicating the LLM's confidence in the inference.

Can I adjust the LLM processing parameters for large wikis?

Yes. The article-analyzer accepts configuration parameters including batchSize (number of articles processed per LLM call) and maxTokens (truncation limit for article bodies). These settings allow you to optimize throughput and cost when processing large repositories, as shown in the TypeScript example that calls runSkill('article-analyzer', { batchSize: 15, maxTokens: 3000 }).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}