How the Ego-Browser Learning Subsystem Loads and Executes Site-Specific Browser Tools

The ego-browser learning subsystem dynamically loads JavaScript tools from skills/ego-browser/learnings/ and executes them inside the page context by wrapping the source in an async IIFE, injecting arguments via JSON serialization, and dispatching the code through Chrome DevTools Protocol (CDP).

The citrolabs/ego-lite repository provides a specialized browser automation framework where the learning subsystem manages reusable "site skills." These skills declare browser tools—JavaScript modules that run directly inside the target page's DOM—to interact with specific web applications like X.com or LinkedIn without exposing the agent to brittle DOM selectors.

Architectural Overview of Site Skills

Site-specific capabilities in ego-browser are stored under skills/ego-browser/learnings/<site-id>/. Each skill contains a manifest file and a directory of tool implementations that declare both Node-side utilities and browser-side scripts.

Manifest Declaration

The manifest.json file declares available browserTools with their filesystem paths and schemas. This separates tool metadata from implementation, allowing the system to validate requests before loading source code.

{
  "browserTools": {
    "post_from_active_element": {
      "path": "tools/post_from_active_element.js",
      "description": "Posts the active element's text to the feed",
      "args": {}
    }
  }
}

According to the source code in src/learning/index.ts, the toolSchemas function extracts the browser tool definitions from this manifest to validate execution requests against declared capabilities.

The Execution Pipeline

When an agent script invokes site.runBrowserTool(siteId, toolName, args), the system executes a six-step pipeline that moves from the agent process into the browser's JavaScript context.

Step 1: Facade Entry Point (runSiteBrowserTool)

The helper function defined in src/helpers.ts (lines 99–105) serves as the primary API surface. It coordinates loading the source, wrapping it with arguments, and dispatching it for evaluation.

// src/helpers.ts – lines 99‑105
export async function runSiteBrowserTool(siteId, toolName, args: any = {}) {
  const source = await loadBrowserToolSource(siteId, toolName, {
    agentWorkspace: state.agentWorkspace(),
  });
  return evaluate(wrapBrowserTool(source, args));
}

This function abstracts the complexity of filesystem access and CDP communication from the agent script.

Step 2: Source Location and Validation (loadBrowserToolSource)

Located in src/learning/index.ts (lines 83–92), this function resolves the site directory, validates the tool name against the manifest schema, and reads the JavaScript file from disk.

// src/learning/index.ts – lines 83‑92
export async function loadBrowserToolSource(siteId, toolName, options: any = {}) {
  const { siteDir, manifest } = await findSiteSkill(siteId, options);
  const schema = toolSchemas(manifest, "browserTools")[toolName];
  if (!schema || typeof schema !== "object") {
    throw new Error(`browser tool ${JSON.stringify(toolName)} is not declared …`);
  }
  const toolPath = relativeSitePath(siteDir, schema.path, "browser tool");
  return readFile(toolPath, "utf8");
}

If the tool is not declared in the manifest, the function throws a validation error before any file system access occurs, preventing execution of arbitrary paths.

Step 3: Wrapping for Safe Execution (wrapBrowserTool)

Before injection, the raw source must be wrapped to create an isolated execution context and bind arguments. The wrapBrowserTool function (lines 94–96 in src/learning/index.ts) constructs an async IIFE that assigns the tool source to a local constant and invokes it with serialized arguments.

// src/learning/index.ts – lines 94‑96
export function wrapBrowserTool(source, args: any = {}) {
  return `(async () => { const __egoBrowserTool = ${source};
            return await __egoBrowserTool(${JSON.stringify(args)}); })()`;
}

This wrapper ensures the tool executes as an anonymous async function, preventing global scope pollution and enabling top-level await patterns inside the tool code.

Step 4: CDP Evaluation in Page Context

The evaluate function imported from src/cdp-eval.ts transmits the wrapped JavaScript to the browser via ego.sendCDPMessage. This executes the code inside the current page's V8 context, granting the tool full access to document, window, and other browser APIs.

The promise returned by evaluate resolves to the tool's return value or rejects if the tool throws an exception, propagating errors back to the agent script as standard JavaScript promises.

Practical Implementation Example

The following example demonstrates discovering and invoking a learned tool for X.com:

// Example: post a tweet on X‑com using a learned browser tool
const ctx = await site.learnContext('https://x.com/home');
console.log('Available tools:', ctx.tools.map(t => t.toolName));

const result = await site.runBrowserTool('x-com', 'post_from_active_element', {
  text: 'Hello from Ego‑Browser!'
});
console.log('Tool result →', result);

Inside the site skill repository, the tool implementation exports a default async function that interacts with the DOM directly:

// Inside skills/ego-browser/learnings/x-com/tools/post_from_active_element.js
export default async function postFromActiveElement({ text }) {
  const active = document.activeElement;
  if (!active) throw new Error('No active element');
  active.value = text;
  const button = document.querySelector('button[data-testid="tweetButton"]');
  if (!button) throw new Error('Post button not found');
  button.click();
  return { status: 'sent', text };
}

Key Files and Responsibilities

File	Responsibility
src/helpers.ts	Facade exposing runSiteBrowserTool to agent scripts and coordinating the evaluation pipeline.
src/learning/index.ts	Core loading logic including loadBrowserToolSource, wrapBrowserTool, and manifest parsing.
src/cdp-eval.ts	CDP execution layer that transfers JavaScript into the browser page context and returns results.
skills/ego-browser/learnings/<site>/manifest.json	Declares available browserTools with paths, descriptions, and argument schemas.
skills/ego-browser/learnings/<site>/tools/*.js	Browser-side tool implementations exported as default async functions.

Summary

• Declarative Architecture: Site skills use manifest.json to declare browser tools, separating metadata from implementation and enabling runtime validation.
• Filesystem Resolution: The loadBrowserToolSource function in src/learning/index.ts resolves tool paths relative to the skill directory and validates them against the manifest schema.
• Secure Wrapping: wrapBrowserTool injects arguments by serializing them to JSON and wrapping the source in an async IIFE, preventing global scope leakage.
• CDP Execution: The evaluate function dispatches code to the browser via Chrome DevTools Protocol, executing tools with full DOM access while returning results to the agent process.
• Error Propagation: Exceptions thrown inside the browser context reject the promise returned to the agent, enabling standard try/catch error handling.

Frequently Asked Questions

Where does the ego-browser learning subsystem store site-specific tools?

Site-specific tools are stored under skills/ego-browser/learnings/<site-id>/. Each site directory contains a manifest.json file and a tools/ subdirectory containing the JavaScript implementations of browser tools.

How does the system validate that a requested browser tool exists before execution?

The loadBrowserToolSource function in src/learning/index.ts validates the requested tool name against the browserTools schema defined in the site's manifest. If the tool is not declared or the schema is invalid, it throws an error before attempting to read the file from disk.

What mechanism allows browser tools to receive parameters from the agent script?

Arguments are serialized to JSON and injected into the tool's execution context via the wrapBrowserTool function. This function constructs an async IIFE that deserializes the arguments and passes them as a single object parameter to the tool's default export function.

How are errors handled when a browser tool fails inside the page context?

Errors thrown within the browser tool's execution reject the promise returned by the evaluate function in src/cdp-eval.ts. This rejection propagates back through runSiteBrowserTool in src/helpers.ts, allowing the calling agent script to catch failures using standard JavaScript error handling patterns.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}