How the ClosedClaw Browser Tool Leverages Playwright for Web Automation Tasks

The ClosedClaw browser tool acts as a Playwright-driven HTTP/CLI façade that maintains a persistent Chrome DevTools Protocol (CDP) connection to Chromium, translating high-level commands like navigate, click, and screenshot into deterministic Playwright API calls.

The browser tool in the asafelobotomy/closedclaw repository implements a thin automation layer that bridges agent commands with Playwright's core capabilities. By exposing REST endpoints for browser actions while managing a long-lived CDP session in src/browser/pw-session.ts, the tool enables reliable web automation for LLM agents without exposing low-level complexity.

Architecture Overview

The implementation follows a client-server architecture where a local browser control server mediates between user commands and Playwright operations. The gateway starts a control service in src/browser/server.ts that exposes endpoints such as /navigate, /act, /screenshot, and /snapshot. When requests arrive, the server delegates to Playwright helpers defined in the pw-tools-core.* modules, which execute against a cached browser instance.

Establishing the Playwright CDP Connection

The foundation of the automation layer is a persistent Chrome DevTools Protocol (CDP) connection managed in src/browser/pw-session.ts. When the server initializes, it creates a single Playwright connection via chromium.connectOverCDP targeting a local Chromium instance.

The module caches the Browser instance using internal flags (cached, connecting) to prevent redundant connections. For each incoming request, the server calls getPageForTargetId, which resolves the Playwright Page object either by:

• Querying CDP sessions directly via Target.getTargetInfo when a targetId is provided
• Falling back to an HTTP /json/list fetch (findPageByTargetId) when Chrome extension relays block CDP APIs

This connection reuse pattern ensures that subsequent commands operate against the same browser context without re-establishing expensive network handshakes.

Executing Element Interactions

All user interactions—clicks, typing, hovering, and dragging—are implemented in src/browser/pw-tools-core.interactions.ts. These helpers receive a Page instance from getPageForTargetId, restore any cached role references (refs like e1, e2), and invoke Playwright locators.

For example, the clickViaPlaywright function handles both single and double clicks with modifier keys:

// src/browser/pw-tools-core.interactions.ts
export async function clickViaPlaywright(opts: {
  cdpUrl: string;
  targetId?: string;
  ref: string;
  doubleClick?: boolean;
  button?: "left" | "right" | "middle";
  modifiers?: Array<"Alt" | "Control" | "ControlOrMeta" | "Meta" | "Shift">;
  timeoutMs?: number;
}): Promise<void> {
  const page = await getPageForTargetId({
    cdpUrl: opts.cdpUrl,
    targetId: opts.targetId,
  });
  ensurePageState(page);
  restoreRoleRefsForTarget({ cdpUrl: opts.cdpUrl, targetId: opts.targetId, page });
  const ref = requireRef(opts.ref);
  const locator = refLocator(page, ref);
  const timeout = Math.max(500, Math.min(60_000, Math.floor(opts.timeoutMs ?? 8000)));

  try {
    if (opts.doubleClick) {
      await locator.dblclick({ timeout, button: opts.button, modifiers: opts.modifiers });
    } else {
      await locator.click({ timeout, button: opts.button, modifiers: opts.modifiers });
    }
  } catch (err) {
    throw toAIFriendlyError(err, ref);
  }
}

The refLocator function maps string references (e.g., e12) to Playwright Locator objects, enabling deterministic element addressing across multiple requests.

Navigation and Snapshot Generation

Page navigation and visual snapshots are handled in src/browser/pw-tools-core.snapshot.ts. The navigateViaPlaywright function wraps page.goto with configurable timeouts between 1,000ms and 120,000ms:

// src/browser/pw-tools-core.snapshot.ts
export async function navigateViaPlaywright(opts: {
  cdpUrl: string;
  targetId?: string;
  url: string;
  timeoutMs?: number;
}): Promise<{ url: string }> {
  const url = String(opts.url ?? "").trim();
  if (!url) throw new Error("url is required");

  const page = await getPageForTargetId(opts);
  ensurePageState(page);
  await page.goto(url, {
    timeout: Math.max(1000, Math.min(120_000, opts.timeoutMs ?? 20_000)),
  });
  return { url: page.url() };
}

For AI consumption, the tool provides two snapshot modes:

• AI Snapshots: Generated via _snapshotForAI, an internal Playwright method that returns a text-based representation of the page
• Role-Based Snapshots: Built using locator.ariaSnapshot() and stored with indexed references (e1, e2, etc.) in pageStates cache

These references persist across requests via storeRoleRefsForTarget and restoreRoleRefsForTarget, allowing agents to refer to specific elements consistently throughout a session.

Screenshot Capabilities

The screenshot functionality demonstrates the tool's flexibility in capturing both full-page and element-specific images. Implemented in src/browser/pw-tools-core.interactions.ts, the takeScreenshotViaPlaywright function accepts either a role reference or full-page flags:

// src/browser/pw-tools-core.interactions.ts (excerpt)
export async function takeScreenshotViaPlaywright(opts: {
  cdpUrl: string;
  targetId?: string;
  ref?: string;
  element?: string;
  fullPage?: boolean;
  type?: "png" | "jpeg";
}): Promise<{ buffer: Buffer }> {
  const page = await getPageForTargetId(opts);
  ensurePageState(page);
  restoreRoleRefsForTarget({ cdpUrl: opts.cdpUrl, targetId: opts.targetId, page });
  const type = opts.type ?? "png";

  if (opts.ref) {
    if (opts.fullPage) throw new Error("fullPage is not supported for element screenshots");
    const locator = refLocator(page, opts.ref);
    const buffer = await locator.screenshot({ type });
    return { buffer };
  }
  // …element‑based or full‑page screenshot omitted for brevity
}

When invoked via the CLI (ClosedClaw browser screenshot --ref e12), the command translates to an HTTP request that ultimately executes this Playwright wrapper.

CLI and Agent Integration

The high-level interface resides in src/browser/client-actions.ts, which marshals tool requests into HTTP calls to the control server. The browserAct function demonstrates this translation layer:

// src/browser/client-actions.ts
export async function browserAct(
  baseUrl: string | undefined,
  req: BrowserActRequest,
  opts?: { profile?: string },
): Promise<BrowserActResponse> {
  const q = buildProfileQuery(opts?.profile);
  return await fetchBrowserJson<BrowserActResponse>(withBaseUrl(baseUrl, `/act${q}`), {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify(req),
    timeoutMs: TIMEOUT_BROWSER_PAGE_MS,
  });
}

The CLI entry point (ClosedClaw browser ...) and the agent tool both utilize these wrappers, sending JSON payloads to endpoints like /act, which dispatch to the appropriate Playwright helper based on the action type.

Summary

• The ClosedClaw browser tool leverages Playwright's CDP connection (chromium.connectOverCDP) in src/browser/pw-session.ts to maintain a persistent session with Chromium, caching the Browser instance for performance.
• Page resolution occurs via getPageForTargetId, which supports both CDP session queries and fallback HTTP /json/list lookups when extensions block CDP APIs.
• All browser actions—navigation, clicking, typing, screenshots—are implemented in pw-tools-core.* modules using native Playwright APIs like page.goto, locator.click, and locator.screenshot.
• Role-based referencing enables deterministic element addressing across requests, with references stored in pageStates and restored via restoreRoleRefsForTarget.
• The architecture exposes Playwright functionality through a REST/CLI façade (src/browser/server.ts and client-actions.ts), wrapping errors in AI-friendly formats using toAIFriendlyError.

Frequently Asked Questions

What Chrome DevTools Protocol features does ClosedClaw use?

The tool primarily uses Target.getTargetInfo to resolve tab IDs to Playwright Page instances, along with chromium.connectOverCDP to establish the initial connection. When CDP APIs are blocked by Chrome extensions, it falls back to standard HTTP endpoints (/json/list) to locate targets.

How does ClosedClaw maintain element state between automation steps?

The tool caches role references (such as e1, e2) generated from ariaSnapshot() calls in a per-page state store (pageStates). When subsequent requests arrive, restoreRoleRefsForTarget rehydrates these mappings, allowing Playwright locators to resolve the same DOM elements across multiple HTTP requests.

Can the browser tool handle Chrome extensions that interfere with automation?

Yes. When the standard CDP session queries are blocked by extensions, the implementation in src/browser/pw-session.ts includes a fallback mechanism that fetches the target list via HTTP and locates pages using the findPageByTargetId helper, ensuring robust automation even in restricted environments.

What Playwright methods power the core browser interactions?

According to the source code, the tool uses page.goto for navigation, locator.click and locator.dblclick for interactions, locator.fill for text input, locator.screenshot for element captures, and page.screenshot or page.pdf for full-page documentation. AI snapshots utilize _snapshotForAI and locator.ariaSnapshot() for semantic page representation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}