What Is the Flagship Skill in the NVIDIA Plugin?

The flagship skill in the NVIDIA plugin is Physical AI Neural Reconstruction, a production-ready workflow that executes large-scale neural-network inference on GPU-accelerated hardware and exposes 3D reconstruction capabilities through the OpenAI plugin interface.

The openai/plugins repository hosts the official NVIDIA plugin implementation, which extends OpenAI models with hardware-accelerated computing capabilities. The Physical AI Neural Reconstruction skill serves as the canonical demonstration of how the plugin orchestrates NVIDIA runtime containers, manages CUDA dependencies, and returns reconstructed assets through standard OpenAI API calls.

Architecture of the Flagship NVIDIA Skill

The Physical AI Neural Reconstruction skill is implemented in plugins/nvidia/skills/physical-ai-neural-reconstruction and follows a modular architecture that separates skill definition, runtime provisioning, and execution orchestration.

Skill Definition and Metadata

The skill declares its interface through a skill-card file located at skill-card.md. This document specifies the expected inputs, output formats, and usage patterns for the neural reconstruction workflow. It acts as the human-readable contract that defines how clients interact with the GPU-accelerated inference pipeline.

Runtime Dependencies and GPU Acceleration

Execution requires a dedicated NVIDIA runtime defined in nvidia-runtime.md within the skill's dependency references. This configuration ensures the correct Docker image, GPU drivers, and CUDA libraries are provisioned before the skill initializes. The runtime dependency management guarantees that neural network inference occurs on properly configured hardware with appropriate accelerator support.

Orchestration via OS-MO

The skill registers with the OpenAI Skill-Meta-Orchestrator (OS-MO) framework through the skill.oms.sig signature file. This machine-readable signature registers the skill with the plugin host, enabling the OpenAI API to route requests to the appropriate NVIDIA runtime container. The orchestration layer handles container lifecycle management and GPU resource allocation during execution.

Agents and Evaluation

An OpenAI-compatible agent definition in agents/openai.yaml provides the adapter that maps API calls to the skill's execution flow. Additionally, evals/evals.json contains the evaluation suite used to validate the skill's performance and correctness across various prompt scenarios, ensuring reliable behavior in production environments.

How to Invoke the Physical AI Neural Reconstruction Skill

Clients interact with the flagship skill through standard OpenAI API endpoints by specifying the NVIDIA plugin in their requests.

Python SDK Example

Use the openai library with the plugins parameter to activate the NVIDIA runtime:

import openai

resp = openai.ChatCompletion.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a plugin that can run the NVIDIA Physical AI Neural Reconstruction skill."},
        {"role": "user", "content": "Reconstruct a 3-D model from the following set of depth images."},
    ],
    plugins=["nvidia"]
)

print(resp.choices[0].message.content)

Direct HTTP API Call

Alternatively, invoke the skill via cURL by including the plugin identifier in the request payload:

curl https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "gpt-4o-mini",
        "messages": [
          {"role":"system","content":"You have access to the NVIDIA Physical AI Neural Reconstruction skill."},
          {"role":"user","content":"Generate a high-resolution 3-D mesh from these point clouds."}
        ],
        "plugins": ["nvidia"]
      }'

Both methods trigger the skill execution, which internally spins up the configured NVIDIA container, processes the neural inference workload, and returns the reconstructed 3D asset through the standard chat completion response.

Summary

• The Physical AI Neural Reconstruction skill is the flagship capability of the openai/plugins NVIDIA plugin, demonstrating large-scale neural inference on GPU hardware.
• The skill architecture separates concerns into skill definition (skill-card.md), runtime provisioning (nvidia-runtime.md), and orchestration (skill.oms.sig via OS-MO).
• Clients invoke the skill through standard OpenAI API calls by specifying plugins: ["nvidia"] in their requests.
• The implementation includes comprehensive evaluation suites (evals/evals.json) and agent adapters (agents/openai.yaml) for production deployment.

Frequently Asked Questions

What does the Physical AI Neural Reconstruction skill do?

The skill executes large-scale neural-network inference to reconstruct 3D models from input data such as depth images or point clouds. According to the openai/plugins source code, it runs on NVIDIA GPU hardware through containerized runtimes and returns high-resolution meshes or reconstructed assets via the OpenAI plugin interface.

How does the NVIDIA plugin handle GPU dependencies?

The plugin uses the nvidia-runtime.md configuration to declare Docker images, GPU drivers, and CUDA libraries required for execution. Before the skill runs, the OS-MO orchestrator provisions these dependencies, ensuring the neural inference workload executes on properly configured hardware with appropriate accelerator support.

Can I invoke the flagship skill without the OpenAI SDK?

Yes. You can invoke the Physical AI Neural Reconstruction skill directly via HTTP POST requests to the OpenAI API endpoint. Include the plugins: ["nvidia"] parameter in your JSON payload alongside your messages and model specification, as demonstrated in the cURL example using standard authentication headers.

Where is the skill evaluation logic defined?

The evaluation suite resides in evals/evals.json within the skill directory. This file contains test harnesses that validate the skill's performance and correctness across various prompt scenarios, ensuring the neural reconstruction pipeline behaves correctly under different input conditions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}