How Does the Model Interact with Plugin APIs: OpenAI's Function Calling Architecture

OpenAI's large language model interacts with plugin APIs through a manifest-driven function-calling workflow, where the model generates structured JSON payloads that are dispatched to plugin HTTP endpoints, and responses are fed back into the conversation context.

The openai/plugins repository demonstrates a contract-first architecture that enables secure, dynamic capability extension. Rather than hard-coding API logic, the model discovers callable functions through declarative manifests, treats each plugin as a remote function, and manages the full request-response lifecycle through standardized HTTP interactions.

The Manifest-Driven Contract

Every plugin in the repository lives under plugins/<name>/ and must include a .codex-plugin/plugin.json file. This manifest serves as the authoritative contract that describes the plugin's HTTP endpoint URL, authentication scheme, and the JSON schema for request/response payloads.

According to the openai/plugins source code, the manifest declares OpenAPI-style "functions" that the model can invoke. These declarations include parameter types, required fields, and descriptions that guide the model's argument generation. The contract ensures that the model never guesses at API structures—it strictly adheres to the schemas defined in the plugin configuration.

The Function Calling Workflow

Step 1: Schema Registration and Model Request

When a user submits a query requiring plugin assistance, the client includes a functions parameter in the openai.ChatCompletion.create call. This parameter mirrors the schemas defined in the plugin's plugin.json manifest and tells the model which capabilities are available.

The model analyzes the user input and, when appropriate, outputs a function call object containing the function name and a JSON-encoded arguments payload. Setting function_call="auto" allows the model to decide autonomously when invocation is necessary based on the conversation context.

Step 2: HTTP Dispatch to Plugin Endpoints

The OpenAI runtime extracts the target URL from the manifest's endpoint declaration and forwards the function call via HTTP POST. The request body contains the arguments JSON together with conversation messages for context preservation.

The dispatch mechanism validates that the outgoing payload matches the schema declared in plugins/<name>/.codex-plugin/plugin.json before transmission, ensuring type safety across the network boundary.

Step 3: Plugin Processing and Validation

Upon receiving the request, the plugin server—implemented in files like plugins/<name>/skills/*.py or *.ts—validates the incoming payload against its declared schema. The server then executes the requested business logic, whether querying databases, calling third-party services, or performing computations.

The plugin must return a JSON response adhering strictly to the output schema defined in its manifest. This contract ensures that the model can reliably parse the result without encountering unexpected data structures.

Step 4: Response Integration and Continuation

The OpenAI service receives the plugin's JSON response and injects it into the conversation history as a new message with role function. The model then processes this result in a subsequent ChatCompletion.create call, incorporating the data into its final answer or determining that additional plugin calls are necessary.

This architecture supports chained interactions where the model makes sequential calls to multiple plugins, using each response to inform the next request, all managed through the same manifest-defined contract.

Error Handling and Recovery

When a plugin returns an error status or malformed response, the model receives structured error information through the same function message channel. The model can surface user-friendly explanations or automatically retry with adjusted arguments, depending on the error type defined in the manifest's error schema.

Implementation Example

The following Python implementation demonstrates the complete interaction cycle between the model and a plugin API:

import openai

# 1️⃣ Define the functions the model may call (mirrors the plugin manifest)

functions = [
    {
        "name": "search_notebook",
        "description": "Search a user’s private notebook for relevant notes",
        "parameters": {
            "type": "object",
            "properties": {
                "query": {"type": "string", "description": "Search term"}
            },
            "required": ["query"]
        }
    }
]

# 2️⃣ Send a chat request; the model will decide whether to call the function

response = openai.ChatCompletion.create(
    model="gpt-4-0613",
    messages=[{"role": "user", "content": "Find my notes about the Q3 budget"}],
    functions=functions,
    function_call="auto"      # let the model decide when to call

)

# 3️⃣ If the model decided to call the function, forward it to the plugin endpoint

if response.choices[0].finish_reason == "function_call":
    function_name = response.choices[0].message.function_call.name
    arguments = json.loads(response.choices[0].message.function_call.arguments)

    # HTTP POST to the plugin (URL taken from the plugin’s plugin.json)

    plugin_response = requests.post(
        "https://my‑plugin.example.com/search_notebook",
        json=arguments,
        headers={"Authorization": "Bearer <api-key>"}
    ).json()

    # 4️⃣ Feed the plugin’s result back to the model

    continuation = openai.ChatCompletion.create(
        model="gpt-4-0613",
        messages=[
            {"role": "user", "content": "Find my notes about the Q3 budget"},
            {"role": "assistant", "function_call": response.choices[0].message.function_call},
            {"role": "function", "name": function_name, "content": json.dumps(plugin_response)}
        ]
    )
    print(continuation.choices[0].message.content)

In this flow, the finish_reason check confirms that the model opted to use the plugin, while the function_call.arguments extraction retrieves the structured payload derived from the manifest schema.

Critical Files in the Architecture

Understanding how the model interacts with plugin APIs requires familiarity with these repository locations:

• plugins/<name>/.codex-plugin/plugin.json — Declares the HTTP endpoint, authentication headers, and the complete JSON schema that governs how the model constructs function calls and interprets responses.

• README.md (repository root) — Documents the overarching plugin architecture and mandates that every plugin must include a valid manifest file to participate in the function-calling ecosystem.

• plugins/<name>/README.md (e.g., plugins/figma/README.md) — Provides concrete, domain-specific examples of how a particular plugin's API structure maps to the model's function invocation patterns.

• plugins/<name>/skills/*.py or *.ts — Contain the runtime implementations that receive HTTP requests, validate arguments against the manifest schema, execute business logic, and return standardized JSON results.

Summary

• Contract-First Design: The .codex-plugin/plugin.json manifest defines the complete interface between the model and plugin API, eliminating hard-coded integration logic.

• Function Calling Protocol: The model generates structured JSON arguments via ChatCompletion.create with functions parameters, which are dispatched to plugin HTTP endpoints.

• Bidirectional Flow: Plugin responses return through the conversation history as function role messages, enabling the model to synthesize results or chain multiple calls.

• Schema Enforcement: Both request and response payloads undergo validation against the OpenAPI-style schemas declared in the manifest, ensuring type-safe interactions.

Frequently Asked Questions

How does the model know which plugin API to call?

The model references the functions parameter provided in the API request, which contains schemas extracted from each plugin's plugin.json manifest. When the user query matches a described capability, the model generates a function call targeting that specific plugin's HTTP endpoint as declared in the manifest.

What authentication method secures the model-to-plugin connection?

Authentication credentials are declared in the plugins/<name>/.codex-plugin/plugin.json manifest, typically using Bearer tokens or API keys. The OpenAI runtime includes these headers when dispatching the HTTP POST request to the plugin endpoint, ensuring secure, authorized communication without exposing keys to the model itself.

Can the model execute multiple plugin calls in a single conversation?

Yes, the architecture supports call chaining. After receiving a plugin response and feeding it back via a function role message, the model can evaluate the new context and emit additional function_call objects. This enables complex workflows like retrieving data from one plugin and processing it through another within the same chat session.

What happens if a plugin API returns an error or invalid JSON?

The plugin should return a structured error response adhering to the schema defined in its manifest. The model receives this error as content in the function role message and can either surface a user-friendly explanation or adjust its arguments for a retry, depending on the error type and conversation context.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}