How to Monitor AI Models Using AI-Infra-Guard: A Complete Technical Guide

AI-Infra-Guard provides a task-based REST API architecture that lets you monitor AI model security scans by submitting tasks and polling the /status/{session_id} endpoint or subscribing to Server-Sent Events for real-time log streaming.

The Tencent/AI-Infra-Guard open-source framework implements a unified task management system to scan, protect, and monitor AI infrastructure. When you need to monitor AI models using AI-Infra-Guard, you interact with the TaskManager component through HTTP endpoints that track scan progress from submission to completion. This approach enables continuous monitoring of model security, prompt injection tests, and infrastructure vulnerability scans through a standardized REST interface.

Understanding the Monitoring Architecture

The monitoring system centers on a task lifecycle managed by three core components:

TaskManager orchestrates all scan operations. Implemented in [common/websocket/task_manager.go](https://github.com/Tencent/AI-Infra-Guard/blob/main/common/websocket/task_manager.go), this component stores task metadata, updates execution progress, and manages log aggregation for every AI model scan.

REST API Layer exposes the monitoring endpoints. The [common/websocket/api.go](https://github.com/Tencent/AI-Infra-Guard/blob/main/common/websocket/api.go) file defines the TaskStatusResponse struct and routes such as GET /status/{id} that return current task states【https://github.com/Tencent/AI-Infra-Guard/blob/main/common/websocket/api.go#L20-L28】.

Database Model Store persists configuration and credentials. Located in [pkg/database/model.go](https://github.com/Tencent/AI-Infra-Guard/blob/main/pkg/database/model.go), this layer maintains ModelParams records that link specific AI model configurations to their monitoring tasks.

The Monitoring Workflow

Submitting a Scan Task

To begin monitoring an AI model, submit a scan task via the tasks endpoint. The request body specifies the scan type (MCP, AI-Infra, or Prompt-Security), target model parameters, and concurrency settings.

curl -X POST http://127.0.0.1:8088/api/v1/app/taskapi/tasks \
  -H "Content-Type: application/json" \
  -d '{
        "type": "mcp_scan",
        "content": {
          "prompt": "Scan this repository for prompt injection vulnerabilities",
          "model": {
            "model": "gpt-4",
            "token": "YOUR_TOKEN",
            "base_url": "https://api.openai.com/v1"
          },
          "thread": 4,
          "language": "en"
        }
      }'

The API returns a unique session_id that serves as the monitoring handle for the task lifecycle.

Polling Task Status

Query the status endpoint using the session_id to retrieve current execution state, timestamps, and incremental logs. The endpoint is defined in [common/websocket/api.go](https://github.com/Tencent/AI-Infra-Guard/blob/main/common/websocket/api.go#L29-L30).

SESSION_ID="550e8400-e29b-41d4-a716-446655440000"

curl -s http://127.0.0.1:8088/api/v1/app/taskapi/status/$SESSION_ID \
  -H "Accept: application/json"

The TaskStatusResponse JSON includes fields for status (pending, running, completed, or failed), log (execution output), and Unix timestamps for created_at and updated_at.

Retrieving Final Results

Once the status returns completed, fetch the structured results through the result endpoint:

curl http://127.0.0.1:8088/api/v1/app/taskapi/result/$SESSION_ID \
  -H "Accept: application/json"

Real-Time Monitoring with SSE

For live monitoring without polling, open a Server-Sent Events connection to stream logs as the Agent component executes the scan. The SSE implementation resides in [common/websocket/sse_manager.go](https://github.com/Tencent/AI-Infra-Guard/blob/main/common/websocket/sse_manager.go).

curl http://127.0.0.1:8088/api/v1/app/taskapi/stream/$SESSION_ID

This connection pushes each new log line immediately, enabling real-time dashboard integration.

Key Source Files and Components

Component	Purpose	Source File
API Router	Defines REST endpoints including status and result routes	[common/websocket/api.go](https://github.com/Tencent/AI-Infra-Guard/blob/main/common/websocket/api.go)
TaskManager	Manages task lifecycle, state transitions, and log storage	[common/websocket/task_manager.go](https://github.com/Tencent/AI-Infra-Guard/blob/main/common/websocket/task_manager.go)
Model Storage	Persists AI model credentials and configuration parameters	[pkg/database/model.go](https://github.com/Tencent/AI-Infra-Guard/blob/main/pkg/database/model.go)
Agent Executor	Runs the actual scan logic against target AI models	[common/agent/agent.go](https://github.com/Tencent/AI-Infra-Guard/blob/main/common/agent/agent.go)
AI Runner	Interfaces with LLM APIs and captures model responses	[common/runner/ai.go](https://github.com/Tencent/AI-Infra-Guard/blob/main/common/runner/ai.go)
SSE Manager	Handles real-time log streaming via Server-Sent Events	[common/websocket/sse_manager.go](https://github.com/Tencent/AI-Infra-Guard/blob/main/common/websocket/sse_manager.go)

Summary

• AI-Infra-Guard monitors AI models through a task-centric REST API where each scan receives a unique session_id.
• The TaskManager in common/websocket/task_manager.go tracks task states (pending, running, completed, failed) and aggregates execution logs.
• Poll the GET /api/v1/app/taskapi/status/{session_id} endpoint to retrieve current task status and incremental logs.
• Use the SSE stream endpoint for real-time monitoring without polling overhead.
• Model credentials are stored securely in the database layer defined in pkg/database/model.go, ensuring each task runs with proper authentication.

Frequently Asked Questions

What endpoints does AI-Infra-Guard expose for monitoring tasks?

The framework exposes three primary monitoring endpoints: POST /tasks to create scans, GET /status/{session_id} to check execution state, and GET /result/{session_id} to fetch completed outputs. These routes are implemented in [common/websocket/api.go](https://github.com/Tencent/AI-Infra-Guard/blob/main/common/websocket/api.go), with the status endpoint specifically returning JSON containing the task state and current log buffer.

How does the TaskManager track the status of AI model scans?

The TaskManager maintains an in-memory or database-backed registry of active tasks, updating records after each execution phase. As the Agent component runs scans defined in [common/agent/agent.go](https://github.com/Tencent/AI-Infra-Guard/blob/main/common/agent/agent.go), it reports progress back to the TaskManager, which persists timestamps and log entries that the status API subsequently serves to clients.

Can I monitor tasks in real-time instead of polling?

Yes. AI-Infra-Guard supports Server-Sent Events (SSE) through the /api/v1/app/taskapi/stream/{session_id} endpoint. By connecting to this endpoint, clients receive immediate push notifications for each new log line generated by the [common/runner/ai.go](https://github.com/Tencent/AI-Infra-Guard/blob/main/common/runner/ai.go) execution layer, eliminating the latency and overhead of periodic polling.

Where are model credentials stored when running monitoring tasks?

Model credentials and configuration parameters are stored as ModelParams records in the database layer defined in [pkg/database/model.go](https://github.com/Tencent/AI-Infra-Guard/blob/main/pkg/database/model.go). The TaskManager retrieves these credentials when initiating a scan, ensuring that authentication tokens and base URLs remain persisted across monitoring sessions while keeping sensitive data out of client-side polling requests.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}