How to Monitor the Security of Your AI Infrastructure with AI-Infra-Guard

AI-Infra-Guard (A.I.G) is a modular red-team platform that continuously monitors AI services for configuration issues, known CVEs, and unsafe agent behaviors through a Go-based core engine, Python scanning modules, and a Vue.js web interface.

Maintaining visibility into the security posture of AI infrastructure requires specialized tooling that understands both traditional vulnerabilities and AI-specific risks. Tencent's open-source AI-Infra-Guard project provides a comprehensive monitoring solution that fingerprints AI services, detects configuration drift, and identifies unsafe agent behaviors in real time. This guide explains how to deploy and operate the platform to monitor the security of your AI infrastructure effectively.

Architecture Overview

The platform consists of three tightly coupled layers designed for continuous monitoring and assessment.

Core Engine (Go)

The backbone of the system is implemented in Go within cmd/cli/main.go, which exposes a WebSocket-based task manager (common/websocket/server.go) and a comprehensive REST API defined in docs/swagger.yaml. This engine orchestrates scan jobs, stores results in a lightweight SQLite database (pkg/database/model.go), and pushes real-time progress updates to connected clients.

Scanning Modules (Go & Python)

The AI-Infra scanner (pkg/vulstruct/scanner.go) fingerprints running AI services and matches them against YAML-based fingerprint rules (data/fingerprints/) and vulnerability rules (data/vuln/). Complementary Python sub-modules—skill-scan, mcp-scan, and agent-scan—provide specialized checks for AI-specific risks like prompt injection and unsafe tool usage.

User Interface (Vue + Go)

A single-page application (frontend/src/) consumes the API to display real-time scan progress, detailed component inventories, and CVE severity rankings. The interface is accessible at http://<host>:8088 by default.

Deploying AI-Infra-Guard for Continuous Monitoring

The fastest way to begin monitoring is via Docker Compose, which bundles the Go server, SQLite database, and web interface into a single deployable unit.

git clone https://github.com/Tencent/AI-Infra-Guard.git
cd AI-Infra-Guard
docker-compose -f docker-compose.images.yml up -d

Once deployed, the service listens on port 8088 and accepts connections from the CLI, web interface, or direct API calls.

Creating and Managing Scan Tasks

AI-Infra-Guard supports multiple entry points for initiating security scans depending on your automation requirements.

CLI-Based Scanning

The binary built from cmd/cli/main.go provides a straightforward interface for ad-hoc scanning. After building or downloading the release binary, target specific AI infrastructure endpoints:

# Build from source

go build -o aig ./cmd/cli/main.go

# Scan a local vLLM instance

./aig scan -t http://127.0.0.1:8000

# Scan an Ollama server on the network

./aig scan -t http://192.168.1.100:11434

The CLI forwards requests to the internal API and queues tasks via the task manager (common/websocket/task_manager.go).

API-Driven Automation

For programmatic monitoring, use the REST API defined in docs/swagger.yaml. To initiate an MCP (Model Context Protocol) server scan:

curl -X POST http://localhost:8088/api/v1/mcp/scan \
  -H "Content-Type: application/json" \
  -d '{"repo":"https://github.com/user/mcp-server"}'

Real-Time Security Monitoring

Once a scan initiates, the platform provides multiple channels for tracking progress and retrieving results.

WebSocket-Based Progress Tracking

The WebSocket server (common/websocket/server.go) streams live updates to the UI, displaying progress bars and execution logs. Programmatic consumers can subscribe to the same WebSocket channel or poll the REST endpoint:

# Poll for task status

curl http://localhost:8088/api/v1/tasks/{task-id}

Database Storage and Retrieval

Completed scan results persist in SQLite via the models defined in pkg/database/model.go. Retrieve comprehensive reports via the results endpoint:

curl http://localhost:8088/api/v1/tasks/{task-id}/result

The JSON payload includes:

• Detected component versions (e.g., vLLM, Ollama, ComfyUI)
• Matched CVE IDs with CVSS severity scores
• Skill-risk categories (T01-T09 classification)
• Direct remediation links

Automated Alerting and Integration

Integrate AI-Infra-Guard into existing observability pipelines by polling the API for new results and triggering alerts on high-severity findings.

# Get the latest task ID

latest=$(curl -s http://localhost:8088/api/v1/tasks | jq -r '.[0].id')

# Pull and evaluate the result

curl -s http://localhost:8088/api/v1/tasks/${latest}/result | jq '.vulns[] | select(.severity=="Critical")'

Configure cron jobs or webhooks to invoke these calls periodically, feeding outputs into Prometheus, Grafana, or your SIEM for centralized alerting.

Extending Coverage with Custom Rules

The scanner automatically reloads rules from the data/ directories on restart, enabling continuous improvement without code changes.

• Add fingerprint rules to data/fingerprints/ to recognize new AI components
• Append vulnerability definitions to data/vuln/ to detect emerging CVEs
• Place specialized Python checks in skill-scan/ or mcp-scan/ directories

This modular approach ensures your monitoring capabilities evolve alongside the threat landscape.

Summary

• AI-Infra-Guard provides continuous monitoring through a Go-based core, Python scanning modules, and a Vue.js dashboard
• Deploy instantly using docker-compose.images.yml for immediate visibility
• Initiate scans via CLI (cmd/cli/main.go), REST API (docs/swagger.yaml), or the web interface
• Track real-time progress through WebSocket connections (common/websocket/server.go)
• Store and query results from SQLite (pkg/database/model.go) via standardized endpoints
• Extend detection capabilities by adding YAML rules to data/fingerprints/ and data/vuln/

Frequently Asked Questions

What types of AI infrastructure can AI-Infra-Guard monitor?

AI-Infra-Guard monitors inference servers (vLLM, Ollama), model serving platforms (ComfyUI, Stable Diffusion WebUI), MCP servers, and AI agent frameworks. The fingerprint database in data/fingerprints/ identifies components by analyzing HTTP responses, headers, and endpoint behaviors, while the Python sub-modules evaluate application-layer risks in skills and agent configurations.

How does AI-Infra-Guard detect vulnerabilities?

The AI-Infra scanner (pkg/vulstruct/scanner.go) matches discovered component versions against a curated rule set in data/vuln/ containing over 2,000 CVE and GHSA entries. For Python-based checks, the skill-scan module analyzes code for unsafe patterns like hardcoded credentials, prompt injection vectors, and excessive permission scopes using static analysis and LLM-based evaluation.

Can I integrate AI-Infra-Guard with existing SIEM tools?

Yes. The REST API in docs/swagger.yaml exposes endpoints for task creation, status polling, and result retrieval. You can script periodic result extraction using standard HTTP clients, transform the JSON output, and forward it to Splunk, ELK Stack, or Prometheus. The WebSocket interface also enables real-time streaming of security events for immediate alerting.

What is the difference between the Go scanner and Python sub-modules?

The Go scanner (pkg/vulstruct/scanner.go) handles infrastructure-level fingerprinting and CVE matching with high performance and low resource overhead. The Python sub-modules (skill-scan, mcp-scan, agent-scan) provide specialized analysis of AI-specific artifacts like prompt templates, tool definitions, and agent conversation flows that require natural language processing and semantic understanding.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}