# How to Integrate Webwright with OpenAI Codex as a Plugin: Complete Developer Guide

> Learn how to integrate Webwright with OpenAI Codex as a plugin. This guide shows you how to configure the manifest to enable browser automation skills for Codex.

- Repository: [Microsoft/Webwright](https://github.com/microsoft/Webwright)
- Tags: how-to-guide
- Published: 2026-06-25

---

**To integrate Webwright with OpenAI Codex, configure the [`.codex-plugin/plugin.json`](https://github.com/microsoft/Webwright/blob/main/.codex-plugin/plugin.json) manifest to expose the browser automation skills directory, allowing Codex to discover and invoke the `run` command that orchestrates Playwright execution through the OpenAI Responses API backend.**

The microsoft/Webwright repository ships with a Codex-compatible plugin that transforms OpenAI Codex into a state-of-the-art browser agent. By integrating Webwright with OpenAI Codex as a plugin, you enable the model to drive local Playwright browsers through natural language requests, capture screenshots, and self-verify results without leaving the conversational interface.

## Understanding the Plugin Architecture

The integration relies on three core components: a JSON manifest that describes the plugin capabilities, markdown-based skill definitions that Codex parses into function schemas, and a Python runtime that bridges the LLM with Playwright execution.

### The Plugin Manifest ([`.codex-plugin/plugin.json`](https://github.com/microsoft/Webwright/blob/main/.codex-plugin/plugin.json))

Codex discovers Webwright through the manifest located at [`.codex-plugin/plugin.json`](https://github.com/microsoft/Webwright/blob/main/.codex-plugin/plugin.json) in the repository root. This file declares the plugin as a "productivity" tool with read and write capabilities, pointing Codex to the skills directory where command definitions reside.

```json
{
  "name": "webwright",
  "version": "0.1.0",
  "description": "Turn your coding agent into a SOTA browser agent …",
  "interface": {
    "displayName": "Webwright",
    "shortDescription": "SOTA browser agent driven by your coding agent",
    "capabilities": ["Read", "Write"],
    "websiteURL": "https://github.com/microsoft/Webwright"
  },
  "skills": "./skills/"
}

```

The `interface.capabilities` array tells Codex that Webwright can both retrieve web content and persist files (screenshots, logs), while the `skills` field directs the plugin loader to parse the markdown catalog at `skills/webwright/`.

### Skill Definitions and the Run Command

Each skill is defined in human-readable markdown files that Codex converts into callable function schemas. The primary entry point is the `run` command, defined in [`skills/webwright/commands/run.md`](https://github.com/microsoft/Webwright/blob/main/skills/webwright/commands/run.md), which specifies the contract for executing one-shot web tasks. When Codex loads the plugin, it indexes these commands—such as `run` and `craft`—making them available as tools the model can invoke during conversations.

### The OpenAI Model Backend

Webwright uses [`src/webwright/models/openai_model.py`](https://github.com/microsoft/Webwright/blob/main/src/webwright/models/openai_model.py) to handle all LLM interactions during plugin execution. The `OpenAIModel` class extends the base model interface and specifically wraps the OpenAI Responses API. Key methods include:

- **`_serialize_response_input`**: Converts message histories into the Responses API format required by OpenAI
- **`_extract_response_text`**: Parses the JSON-schema-constrained payload to retrieve the LLM's textual output
- **`_usage_metrics_from_response_payload`**: Extracts token consumption data for monitoring and billing

This backend is what the plugin runtime calls when generating Playwright scripts or verifying execution results.

## Step-by-Step Integration Process

Integrating Webwright requires configuring the manifest, ensuring the runtime environment is available, and establishing the communication flow between Codex and the local browser environment.

1. **Clone and prepare the repository**: Ensure `microsoft/Webwright` is available on your host where the plugin will execute.

2. **Verify the manifest path**: Confirm that [`.codex-plugin/plugin.json`](https://github.com/microsoft/Webwright/blob/main/.codex-plugin/plugin.json) is accessible at the repository root, as Codex will fetch this to discover capabilities.

3. **Configure model settings**: Adjust [`src/webwright/config/model_openai.yaml`](https://github.com/microsoft/Webwright/blob/main/src/webwright/config/model_openai.yaml) to specify your OpenAI model endpoint, API keys, and token limits.

4. **Enable plugin loading in Codex**: Configure your Codex client to load the Webwright manifest URL, allowing the model to index the skill commands.

5. **Initialize execution environment**: Ensure Playwright browsers (Firefox/Chromium) are installed, as [`src/webwright/environments/local_browser.py`](https://github.com/microsoft/Webwright/blob/main/src/webwright/environments/local_browser.py) will spawn sandboxed browser sessions during task execution.

## Code Example: Triggering Webwright from Codex

To activate the plugin from an OpenAI Codex session, send a JSON payload that references the manifest and declares the `run` function. Below is a minimal request structure compatible with the OpenAI SDK or direct HTTP calls:

```json
{
  "model": "gpt-4o",
  "messages": [
    {
      "role": "system",
      "content": "You have access to the Webwright plugin."
    },
    {
      "role": "user",
      "content": "Find the current price of the Apple iPhone 15 on amazon.com and give me the exact figure."
    }
  ],
  "plugins": [
    {
      "type": "manifest",
      "url": "https://github.com/microsoft/Webwright/blob/main/.codex-plugin/plugin.json"
    }
  ],
  "functions": [
    {
      "name": "run",
      "description": "Execute a web-task using the Webwright workflow.",
      "parameters": {
        "type": "object",
        "properties": {
          "task": {
            "type": "string",
            "description": "Natural-language description of the web task."
          }
        },
        "required": ["task"]
      }
    }
  ],
  "function_call": "auto"
}

```

When Codex processes this request, it identifies the `run` function from the parsed [`run.md`](https://github.com/microsoft/Webwright/blob/main/run.md) skill definition, extracts the user's natural language query, and invokes the Webwright runtime with the task parameter.

## Runtime Execution Flow

Once Codex calls the `run` function, [`src/webwright/run/cli.py`](https://github.com/microsoft/Webwright/blob/main/src/webwright/run/cli.py) orchestrates the end-to-end workflow:

1. **Workspace creation**: The CLI generates a temporary directory at `final_runs/run_<id>/` to isolate the task execution.

2. **Plan generation**: Using `OpenAIModel`, the system writes a [`plan.md`](https://github.com/microsoft/Webwright/blob/main/plan.md) file based on the model's high-level outline for completing the task.

3. **Script instrumentation**: The runtime generates [`final_script.py`](https://github.com/microsoft/Webwright/blob/main/final_script.py), an instrumented Playwright script that implements the plan.

4. **Browser execution**: The script executes against the local browser environment defined in [`src/webwright/environments/local_browser.py`](https://github.com/microsoft/Webwright/blob/main/src/webwright/environments/local_browser.py), driving Firefox or Chromium through the planned steps.

5. **Artifact collection**: Screenshots and execution logs are captured to `final_runs/run_<id>/`, including [`final_script_log.txt`](https://github.com/microsoft/Webwright/blob/main/final_script_log.txt) and PNG files.

6. **Self-verification**: The [`src/webwright/agents/default.py`](https://github.com/microsoft/Webwright/blob/main/src/webwright/agents/default.py) agent feeds artifacts back to the OpenAI model to verify the extracted data before returning the final answer to Codex.

## Configuration and Environment Setup

The default agent behavior is controlled by [`src/webwright/agents/default.py`](https://github.com/microsoft/Webwright/blob/main/src/webwright/agents/default.py), which coordinates between the model backend and the browser environment. For OpenAI-specific settings, modify [`src/webwright/config/model_openai.yaml`](https://github.com/microsoft/Webwright/blob/main/src/webwright/config/model_openai.yaml) to adjust:

- Model name (e.g., `gpt-4o`, `gpt-4o-mini`)
- Maximum token limits
- Endpoint URLs
- Temperature and sampling parameters

Ensure the host running the plugin has network access to both the OpenAI API and the target websites, as the `OpenAIModel` class makes synchronous API calls during plan generation and verification phases.

## Summary

- **Load the manifest**: Point Codex to [`.codex-plugin/plugin.json`](https://github.com/microsoft/Webwright/blob/main/.codex-plugin/plugin.json) to discover Webwright's capabilities and skill commands.
- **Invoke skills naturally**: Once loaded, Codex can call the `run` function (defined in [`skills/webwright/commands/run.md`](https://github.com/microsoft/Webwright/blob/main/skills/webwright/commands/run.md)) to execute browser tasks.
- **Leverage the OpenAI backend**: The `OpenAIModel` class in [`src/webwright/models/openai_model.py`](https://github.com/microsoft/Webwright/blob/main/src/webwright/models/openai_model.py) handles Responses API serialization and token tracking.
- **Orchestrate via CLI**: [`src/webwright/run/cli.py`](https://github.com/microsoft/Webwright/blob/main/src/webwright/run/cli.py) manages workspace creation, Playwright execution, and artifact collection in `final_runs/run_<id>/` directories.
- **Verify results**: The default agent in [`src/webwright/agents/default.py`](https://github.com/microsoft/Webwright/blob/main/src/webwright/agents/default.py) enables self-correction by feeding screenshots and logs back to the model before returning final answers.

## Frequently Asked Questions

### What file does OpenAI Codex read to discover Webwright's capabilities?

Codex reads [`.codex-plugin/plugin.json`](https://github.com/microsoft/Webwright/blob/main/.codex-plugin/plugin.json) from the repository root. This JSON manifest declares the plugin name, version, capabilities (Read/Write), and the relative path to the skills directory where command definitions are stored.

### How does Webwright handle LLM interactions during plugin execution?

All LLM interactions route through [`src/webwright/models/openai_model.py`](https://github.com/microsoft/Webwright/blob/main/src/webwright/models/openai_model.py). The `OpenAIModel` class serializes conversation history into the OpenAI Responses API format, extracts structured output from JSON-schema-constrained responses, and tracks token usage via the `_usage_metrics_from_response_payload` method.

### Where does Webwright store screenshots and execution logs during a Codex plugin run?

The runtime creates isolated workspaces under `final_runs/run_<id>/` for each task. This directory contains [`plan.md`](https://github.com/microsoft/Webwright/blob/main/plan.md), [`final_script.py`](https://github.com/microsoft/Webwright/blob/main/final_script.py), [`final_script_log.txt`](https://github.com/microsoft/Webwright/blob/main/final_script_log.txt), and any screenshots captured during Playwright execution, which can be retrieved after the run completes.

### Can I use a different OpenAI model than the default for plugin tasks?

Yes. Modify [`src/webwright/config/model_openai.yaml`](https://github.com/microsoft/Webwright/blob/main/src/webwright/config/model_openai.yaml) to change the model name, token limits, and endpoint parameters. The `OpenAIModel` class reads this configuration at runtime, allowing you to specify alternative models like `gpt-4o-mini` or custom deployments without changing the core plugin code.