How to Validate OpenEnv Environment Configuration with `openenv validate`

The openenv validate command checks whether an OpenEnv environment directory is correctly structured for deployment and verifies that a running server complies with the runtime API contract.

OpenEnv, developed by Hugging Face, provides a command-line interface to ensure your environment configurations meet production standards. Whether you are preparing a local environment folder for multi-mode deployment or verifying a live server's health, the validation tools in src/openenv/cli/commands/validate.py enforce consistency across the ecosystem.

Understanding the openenv validate Command Architecture

The validation workflow splits into two distinct paths orchestrated by the CLI entry point in src/openenv/cli/commands/validate.py. The heavy lifting occurs in src/openenv/cli/_validation.py, which provides the core logic for both static analysis and runtime verification.

Local validation uses validate_multi_mode_deployment() to inspect files and metadata within an environment folder. Runtime validation uses validate_running_environment() to perform HTTP health checks against live endpoints. The CLI automatically detects which path to execute based on whether you provide a directory path or a URL argument detected by _looks_like_url().

Local Environment Validation

When you validate a local directory, the system performs comprehensive static analysis to ensure the environment can launch across multiple deployment modes. The validate_multi_mode_deployment() function returns a tuple (is_valid, issues) where issues lists human-readable error messages.

The validator checks for:

• Required configuration files: Presence of pyproject.toml and uv.lock
• Entry point configuration: A [project.scripts] entry named server pointing to a main function
• Application structure: The server/app.py file must contain a callable main() guarded by if __name__ == "__main__"
• Dependency verification: Required runtime dependencies (openenv or openenv-core) must be listed in pyproject.toml, or the Dockerfile must install them explicitly

Each check generates a criterion object using _make_criterion(), creating a uniform result dictionary containing the test ID, description, pass/fail status, and detailed diagnostics.

Runtime Server Validation

Runtime validation confirms that a deployed OpenEnv server adheres to the HTTP API contract. The validate_running_environment() function contacts the server and verifies critical endpoints return expected payloads.

The validator tests these endpoints:

• /openapi.json: Must be reachable and contain an info.version string
• /health: Must return {"status":"healthy"}
• /metadata: Must provide name and description fields
• /schema: Must return JSON objects defining action, observation, and state structures
• /mcp: Must respond with a valid JSON-RPC 2.0 payload
• Path compliance: The OpenAPI path set must match the expected mode contract including /reset, /step, and /state endpoints

Results aggregate into a structured report with a top-level passed flag, passed_count, and failed_criteria list, enabling automated decision-making in deployment pipelines.

Deployment Mode Detection

The get_deployment_modes() function in src/openenv/cli/_validation.py determines how an environment can be launched based on its structure and validation status.

The detection reports:

• docker: Available when a Dockerfile is present
• openenv_serve, uv_run, python_module: Set to True only when the environment passes all local validation checks, ensuring the environment can execute via the OpenEnv serve command, uv run, or direct Python module execution

Practical Usage Examples

Validating a Local Environment Directory

Run the command from within your environment root to verify local structure:

$ cd my_echo_env
$ openenv validate
[OK] echo: Ready for multi‑mode deployment

Supported deployment modes:
  [YES] docker
  [YES] openenv_serve
  [YES] uv_run
  [YES] python_module

Validate a specific directory by passing the path as an argument:

$ openenv validate envs/chess_env
[FAIL] chess: Not ready for multi‑mode deployment

Issues found:
  - Missing pyproject.toml
  - server/app.py missing main() function

Validating a Running OpenEnv Server

Point the validator at a live server URL to perform runtime checks:

$ openenv validate --url http://localhost:8000
{
  "target": "http://localhost:8000",
  "validation_type": "running_environment",
  "standard_version": "1.3.0",
  "standard_profile": "openenv-http/1.x",
  "mode": "simulation",
  "passed": true,
  "summary": {
    "passed_count": 5,
    "total_count": 5,
    "failed_criteria": [],
    "required_passed_count": 5,
    "required_total_count": 5
  },
  "criteria": [...]
}

JSON Output for CI/CD Pipelines

Use the --json flag to produce machine-readable reports for automated testing:

$ openenv validate --json > validation_report.json

# In a CI script

if jq -e '.passed' validation_report.json > /dev/null; then
  echo "Environment is valid"
else
  echo "Environment validation failed"
  cat validation_report.json
  exit 1
fi

Programmatic Validation in Python

Import the validation utilities directly to integrate checks into custom tooling:

from openenv.cli._validation import validate_multi_mode_deployment, get_deployment_modes
from pathlib import Path

env_path = Path("envs/echo_env")
is_valid, issues = validate_multi_mode_deployment(env_path)
print("Valid:", is_valid)
print("Issues:", issues)

print("Supported modes:", get_deployment_modes(env_path))

Summary

• The openenv validate command supports two distinct validation paths: local directory analysis and runtime server verification.
• Local validation in validate_multi_mode_deployment() ensures pyproject.toml, uv.lock, and server/app.py meet structural requirements.
• Runtime validation in validate_running_environment() confirms live servers respond correctly on /health, /metadata, /schema, /mcp, and /openapi.json endpoints.
• The CLI exits with non-zero status on failure, making it suitable for CI/CD integration.
• Deployment modes (docker, openenv_serve, uv_run, python_module) are automatically detected based on validation results.

Frequently Asked Questions

What files does openenv validate check in local mode?

The validator inspects pyproject.toml for project metadata and dependencies, uv.lock for dependency locking, and server/app.py for the required main() entry point. It also verifies the presence of a Dockerfile when checking for containerized deployment capabilities.

How does runtime validation verify an OpenEnv server?

Runtime validation performs HTTP requests to standard endpoints including /openapi.json, /health, /metadata, /schema, and /mcp. Each endpoint must return specific payload structures, such as {"status":"healthy"} for health checks and JSON-RPC 2.0 for the MCP endpoint, ensuring full API contract compliance.

Can I use openenv validate in CI/CD pipelines?

Yes. The command supports a --json flag that outputs structured validation reports suitable for automated parsing. The CLI returns a non-zero exit code when validation fails, allowing build scripts to halt deployment if environments do not meet requirements.

What is the difference between local and runtime validation?

Local validation analyzes static files on disk to ensure an environment can launch, checking for required configuration files and code structure. Runtime validation connects to a live server via HTTP to verify the running application responds correctly to API requests and maintains the expected runtime contract.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}