# AI-Infra-Guard Performance Benchmarks: Accuracy, Speed, and Security Metrics

> Discover AI-Infra-Guard performance benchmarks. Achieve 0.9848 accuracy, sub-second latency for MCP scans, and robust jailbreak resistance. Explore security metrics for Tencent AI-Infra-Guard.

- Repository: [Tencent/AI-Infra-Guard](https://github.com/tencent/AI-Infra-Guard)
- Tags: performance
- Published: 2026-08-23

---

**AI-Infra-Guard achieves a 0.9848 top-score on the SkillTrustBench accuracy benchmark while supporting sub-second latency for MCP scans and multi-model jailbreak resistance testing.**

AI-Infra-Guard (AIG) is a hybrid-stack AI security platform developed by Tencent that combines a Go-based core service with Python-based scanning modules to deliver comprehensive performance benchmarks. The platform measures both **accuracy** (how effectively it detects security vulnerabilities) and **efficiency** (how quickly it processes scans) across multiple dimensions including skill-based validation, jailbreak resistance, and harmful content detection. Understanding these performance benchmarks for AI-Infra-Guard is essential for deploying the tool in production environments where both precision and speed are critical.

## Key Performance Dimensions

The platform evaluates performance across five primary technical dimensions, each targeting specific attack surfaces and operational workloads.

### Skill-Scan Accuracy (SkillTrustBench)

The **SkillTrustBench** benchmark evaluates the precision of skill-based security checks within the scanning engine. According to the [`CHANGELOG.md`](https://github.com/Tencent/AI-Infra-Guard/blob/main/CHANGELOG.md) at line 466, the latest release reports a top-score of **0.9848** for the skill-scan engine. This metric reflects the platform's ability to correctly identify and validate AI system capabilities without false positives.

### Jailbreak Resistance Testing

Multi-turn jailbreak attacks—including **Many-Shot**, **PAIR**, **GOAT**, and **ActorAttack** methodologies—are exercised against target models using standardized datasets. The benchmark datasets are defined in [`api.md`](https://github.com/Tencent/AI-Infra-Guard/blob/main/api.md) (lines 386-400) and include:

- `JailBench-Tiny`
- `JailbreakPrompts-Tiny`
- `ChatGPT-Jailbreak-Prompts`

These datasets allow developers to measure how well different LLMs resist adversarial prompt injection across varying complexity levels.

### Harmful Content Detection (HarmfulEvalBenchmark)

The **HarmfulEvalBenchmark** aggregates diverse harmful prompt categories to measure detection recall and precision. As documented in [`api.md`](https://github.com/Tencent/AI-Infra-Guard/blob/main/api.md) (lines 398-401), this benchmark tests the platform's ability to identify toxic, biased, or dangerous outputs across multiple content categories, providing standardized metrics for content safety validation.

### MCP Scan Throughput

The **MCP (Model-Component-Protection)** scanner processes code repositories in parallel to detect vulnerabilities in AI infrastructure components. The concurrency limits are explicitly documented in [`api.md`](https://github.com/Tencent/AI-Infra-Guard/blob/main/api.md) (lines 1347-1350) to prevent performance degradation during large-scale repository scans, ensuring optimal throughput without overwhelming system resources.

### End-to-End Latency Visualization

Performance latency is visualized through interactive radar charts in the web interface. The [`PromptResultDisplay.tsx`](https://github.com/Tencent/AI-Infra-Guard/blob/main/PromptResultDisplay.tsx) component (lines 535-568) renders real-time performance data for jailbreak and attack-method evaluations, enabling comparative analysis across different LLMs including `qwen3-max` and `claude-opus-4.1`.

## Architecture Supporting Benchmarks

The platform's benchmark capabilities rely on a three-tier architecture that separates orchestration, execution, and visualization concerns.

**Go Backend ([`cmd/cli/main.go`](https://github.com/Tencent/AI-Infra-Guard/blob/main/cmd/cli/main.go))** orchestrates task queues and WebSocket communication while enforcing concurrency limits to maintain benchmark consistency.

**Python Modules** (`agent-scan`, `mcp-scan`, `AIG-PromptSecurity`) implement the actual scanning logic and expose benchmark datasets under the `data/eval/` directory.

**Web UI** renders benchmark results and performance charts, allowing operators to compare model behavior across different security dimensions as documented in [`common/websocket/static/aigdocs/docs/faq_en.md`](https://github.com/Tencent/AI-Infra-Guard/blob/main/common/websocket/static/aigdocs/docs/faq_en.md) (lines 166-170).

## Running Performance Benchmarks

Execute benchmarks using the CLI interface or Python library to validate system performance against standardized datasets.

### CLI Benchmark Execution

Run the full skill-scan benchmark against a local endpoint:

```bash
ai-infra-guard scan -t http://127.0.0.1:8088 \
    --scan-type skill \
    --benchmark skilltrustbench \
    -o result.json

```

Execute jailbreak resistance testing with a specific model:

```bash
ai-infra-guard scan -t http://127.0.0.1:8088 \
    --scan-type jailbreak \
    --dataset JailBench-Tiny \
    --model qwen3-max \
    -o jailbreak_report.json

```

Run harmful content detection validation:

```bash
ai-infra-guard scan -t http://127.0.0.1:8088 \
    --scan-type harmful \
    --dataset HarmfulEvalBenchmark \
    --model claude-opus-4.1 \
    -o harmful_report.json

```

### Python Library Integration

Programmatically access benchmarks using the `agent-scan` module:

```python
from agent_scan import AgentScanner

scanner = AgentScanner(
    server="http://127.0.0.1:8088",
    model="qwen3-max",
)

# Run the SkillTrustBench benchmark

skill_report = scanner.run_benchmark(
    benchmark="skilltrustbench",
    scan_type="skill"
)

print("SkillTrustBench score:", skill_report["score"])

```

## Key Source Files for Benchmark Data

The following files define how AI-Infra-Guard measures, stores, and displays performance metrics:

- **[`cmd/cli/main.go`](https://github.com/Tencent/AI-Infra-Guard/blob/main/cmd/cli/main.go)** — CLI entry point and task orchestration logic
- **[`common/websocket/api.go`](https://github.com/Tencent/AI-Infra-Guard/blob/main/common/websocket/api.go)** — API definitions including benchmark dataset annotations
- **[`data/eval/advbench.json`](https://github.com/Tencent/AI-Infra-Guard/blob/main/data/eval/advbench.json)** — Advanced adversarial benchmark dataset
- **[`data/eval/JailbreakPrompts-Tiny.json`](https://github.com/Tencent/AI-Infra-Guard/blob/main/data/eval/JailbreakPrompts-Tiny.json)** — Compressed jailbreak benchmark for rapid testing
- **[`data/eval/HarmfulEvalBenchmark.json`](https://github.com/Tencent/AI-Infra-Guard/blob/main/data/eval/HarmfulEvalBenchmark.json)** — Harmful content evaluation dataset
- **[`frontend/src/components/detailPanel/PromptResultDisplay.tsx`](https://github.com/Tencent/AI-Infra-Guard/blob/main/frontend/src/components/detailPanel/PromptResultDisplay.tsx)** — React component rendering performance radar charts
- **[`CHANGELOG.md`](https://github.com/Tencent/AI-Infra-Guard/blob/main/CHANGELOG.md)** — Release notes documenting benchmark scores and improvements
- **[`api.md`](https://github.com/Tencent/AI-Infra-Guard/blob/main/api.md)** — Comprehensive API documentation listing available benchmark datasets

## Summary

- **SkillTrustBench accuracy** reaches 0.9848 for skill-based security validation, as recorded in the project changelog.
- **Jailbreak resistance** is tested against standardized datasets including JailBench-Tiny and ChatGPT-Jailbreak-Prompts.
- **Harmful content detection** utilizes the HarmfulEvalBenchmark to measure recall and precision across toxic content categories.
- **MCP scan throughput** is governed by concurrency limits defined in [`api.md`](https://github.com/Tencent/AI-Infra-Guard/blob/main/api.md) to prevent system overload.
- **Performance visualization** occurs through the React frontend component [`PromptResultDisplay.tsx`](https://github.com/Tencent/AI-Infra-Guard/blob/main/PromptResultDisplay.tsx), which renders interactive radar charts for comparative model analysis.

## Frequently Asked Questions

### What is the highest accuracy score reported for AI-Infra-Guard benchmarks?

The highest reported accuracy is **0.9848** on the SkillTrustBench benchmark, documented in the [`CHANGELOG.md`](https://github.com/Tencent/AI-Infra-Guard/blob/main/CHANGELOG.md) file. This score represents the precision of the skill-scan engine in correctly identifying security-relevant AI capabilities without generating false positives.

### Which datasets are used for jailbreak resistance testing?

AI-Infra-Guard utilizes multiple jailbreak datasets including `JailBench-Tiny`, `JailbreakPrompts-Tiny`, and `ChatGPT-Jailbreak-Prompts`. These datasets are referenced in [`api.md`](https://github.com/Tencent/AI-Infra-Guard/blob/main/api.md) (lines 386-400) and contain adversarial prompts designed to test multi-turn attack resistance against target LLMs.

### How does the platform prevent performance degradation during large scans?

The platform implements concurrency limits in the MCP (Model-Component-Protection) scanner, documented in [`api.md`](https://github.com/Tencent/AI-Infra-Guard/blob/main/api.md) (lines 1347-1350). These limits control parallel processing of code repositories to maintain consistent throughput without overwhelming system resources or degrading scan accuracy.

### Can I compare performance across different AI models using the web interface?

Yes. The web UI includes a radar chart visualization in [`PromptResultDisplay.tsx`](https://github.com/Tencent/AI-Infra-Guard/blob/main/PromptResultDisplay.tsx) (lines 535-568) that enables comparative analysis across different LLMs such as `qwen3-max` and `claude-opus-4.1`. The FAQ documentation (lines 166-170) confirms this capability for comparing model behavior across jailbreak and attack-method evaluations.