How the Learning Subsystem Discovers and Validates Site Skills in Ego-Lite

The learning subsystem in citrolabs/ego-lite discovers site-skills by scanning the workspace's learnings directory, matching URL hostnames against manifest domains, and validates packs through a multi-stage schema check of manifests, notes, and tool files.

The learning subsystem lives in package/ego-browser/src/learning and powers the browser automation framework's ability to adapt to specific websites. This article walks through the discovery and validation pipelines with direct references to the TypeScript implementation.

How Site-Skill Discovery Works

Discovery is the first phase: given any URL, the system finds all compatible site-skill packs. The entry point is siteSkillsForUrl() in check-domain-learning.ts.

Step 1: Locate and Enumerate Learning Directories

The learningsRoot() function determines where site-skill packs live on disk:

// Returns <agent-workspace>/learnings
const root = learningsRoot();  // check-domain-learning.ts#L67

From there, iterLearningDirs(root) reads the directory, filters hidden folders, and returns a sorted list of candidate subdirectories:

// check-domain-learning.ts#L34
const candidates = iterLearningDirs(root);  // ["google", "twitter-x", "linkedin"]

Step 2: Load and Validate Manifests

Each candidate directory must contain a manifest.json. The loadLearningManifest(siteDir) parser handles this with graceful degradation:

// check-domain-learning.ts#L47
const manifest = loadLearningManifest(siteDir);  // Skips directory on parse error

Malformed manifests are caught and skipped rather than crashing the pipeline.

Step 3: Match Hostnames Against Domain Patterns

The siteSkillsForUrl(url) function extracts the hostname and checks each manifest's domains array using domainMatches(hostname, pattern):

// check-domain-learning.ts#L6
const matches = siteSkillsForUrl("https://www.google.com/search?q=ai");

Wildcard patterns like *.example.com are fully supported. The domainMatches helper implements glob-style matching at line 2 of the same file.

Step 4: Build Lightweight Learning Entries

When domains match, learningEntry(siteDir, manifest) constructs a LearningEntry object containing:

• id and name from the manifest
• path to the pack directory
• domains array
• notes file paths
• nodeTools and browserTools schemas

// check-domain-learning.ts#L66
const entry = learningEntry(siteDir, manifest);

Public API for Discovery

The learnContext(url?) helper in helpers.ts exposes discovery to agent scripts:

import { site } from "ego-browser";

const ctx = await site.learnContext("https://news.ycombinator.com");
console.log(ctx.tools.map(t => t.toolName));  // ["fetchPosts", "postComment"]

Source: src/helpers.ts#L508.

How Site-Skill Validation Works

Validation ensures packs are well-formed before execution. The CLI command npm run validate:site-skills invokes validateLearnings() from validate-learning-format.ts.

Phase 1: Manifest Structure Checks

validateLearning(siteDir) parses manifest.json and enforces:

• id must match the directory name
• name must be a non-empty string
• domains must be a non-empty array of syntactically valid domains (verified via isValidDomain)

Source: lines 28-44 of validate-learning-format.ts.

Phase 2: Note File Verification

Each path in the notes array is validated:

• Must follow notes/*.md pattern
• File must exist (requireFile)
• No temporary snapshot refs allowed (@123 or ref=123 patterns rejected by rejectTemporaryRefs)

Source: lines 45-52 of validate-learning-format.ts.

Phase 3: Node-Side Tool Validation

For every tool in nodeTools, the validator checks:

1. Schema object shape via validateToolSchema
2. path points to tools/*.js
3. File exists (pathExists)
4. Module exports the callable name defined in schema.callable
5. No temporary refs in tool source

Source: lines 54-73 of validate-learning-format.ts.

Phase 4: Browser-Side Tool Validation

browserTools undergo similar checks with adjusted paths:

• Path must be browser-tools/*.js
• File existence confirmed
• Temporary refs rejected
• No import validation (tools execute in browser context)

Source: lines 75-82 of validate-learning-format.ts.

Error Aggregation and CLI Output

All violations collect in an errors array returned at line 84. The CLI prints each error, causing CI failures on any validation rule violation.

Running Validated Tools

Once discovered and validated, tools execute through type-safe helpers:

Node-Side Tool Execution

const result = await site.runTool(
  "google",                    // siteId from manifest
  "search_and_extract",        // tool name
  { query: "openai", maxResults: 5 }
);

Documentation: src/format.ts#L801.

Browser-Side Tool Execution

await site.runBrowserTool(
  "twitter",
  "post_from_active_element"
);

Documentation: src/format.ts#L845.

CLI Validation Command

npm run validate:site-skills   # Runs validateLearnings() across all packs

Key Implementation Files

File	Purpose
src/learning/check-domain-learning.ts	Discovery logic: siteSkillsForUrl, learningEntry, domain matching
src/learning/validate-learning-format.ts	Full validation pipeline for manifests, notes, and tools
src/learning/index.ts	Public exports: learningEntry, learningsRoot, siteSkillsForUrl
src/helpers.ts	Runtime helpers: learnContext, runTool, runBrowserTool
src/format.ts	CLI documentation and tool execution references

Summary

• Discovery scans <workspace>/learnings, matches URL hostnames against manifest domains with wildcard support, and returns lightweight LearningEntry objects
• Validation enforces strict schema compliance: manifest structure, note file integrity, tool file existence, callable exports, and prohibition of temporary snapshot references
• Execution relies on validated schemas to provide type-safe runTool and runBrowserTool APIs
• The validate:site-skills CLI command enables CI/CD integration for pack quality assurance

Frequently Asked Questions

What happens if a manifest.json is malformed?

The loadLearningManifest function catches parse errors and silently skips that directory. The discovery process continues with remaining candidates. Malformed manifests do not crash the system; they simply exclude that pack from matching.

How does domain matching handle wildcards?

The domainMatches(hostname, pattern) function supports glob-style wildcards. A pattern like *.example.com matches api.example.com, www.example.com, and any other subdomain. The matching logic is implemented in check-domain-learning.ts at line 2.

Why are temporary snapshot references rejected during validation?

Temporary refs like @123 or ref=123 indicate incomplete development artifacts from the learning capture process. The rejectTemporaryRefs check ensures only finalized, stable documentation enters production packs. This prevents agents from referencing non-existent or unstable content.

Can validation be run programmatically rather than via CLI?

Yes. Import validateLearnings from src/learning/validate-learning-format.ts and call it with an optional root path. The function returns an array of error strings that can be processed programmatically or logged according to your application's needs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}