Configuring AnnexIVDocument Export for EU AI Act Compliance Reporting

The Agent Governance Toolkit (AGT) provides a TechnicalDocumentationExporter class that automatically aggregates runtime governance artefacts into a structured AnnexIVDocument, generating both Markdown dossiers and JSON manifests that satisfy EU AI Act Article 11 and Annex IV evidence requirements.

The Microsoft Agent Governance Toolkit (AGT) ships with a dedicated Annex IV stack designed to transform runtime governance artefacts into the structured technical documentation mandated by EU AI Act Article 11 and Annex IV. By leveraging the TechnicalDocumentationExporter and EvidencePipeline components, development teams can automate the generation of compliance reports that map directly to the five mandatory Annex IV sections. This article explains how to configure and execute the export process using the toolkit's Python SDK and CLI interfaces.

Core Architecture of the Annex IV Stack

TechnicalDocumentationExporter

Located in agent-governance-python/agent-mesh/src/agentmesh/governance/annex_iv.py (lines 4-13), this high-level façade aggregates governance artefacts including policy definitions, audit-log entries, compliance scores, and SLO/SLI metrics into a single AnnexIVDocument model. The exporter inserts placeholder sections that deployers must complete with system-design information that cannot be auto-generated.

AnnexIVDocument and AnnexIVSection Models

Also in annex_iv.py (lines 30-70), these Pydantic models mirror the five mandatory Annex IV sections: General description, Development process, Monitoring & control, Risk management, and Accuracy/robustness/cybersecurity. Each section contains generated content, placeholders for manual input, and a list of source artefacts that fed the content.

EvidencePipeline

Implemented in agent-governance-python/agent-mesh/src/agentmesh/governance/evidence_pipeline.py (lines 4-18), this CLI-driven pipeline discovers artefacts on-disk—including policy YAMLs, audit-log JSONL, compliance-report JSON, and SLO JSON—and feeds them to the exporter. It produces a Markdown dossier and a JSON manifest that together satisfy the EU AI Act evidence-collection requirements.

EvidenceReport

Defined in evidence_pipeline.py (lines 77-88), this wrapper bundles the generated AnnexIVDocument with EvidenceSource metadata including SHA-256 hashes, file paths, and record counts. The report can be persisted as annex-iv-report.md and annex-iv-manifest.json for regulatory submission and auditor verification.

Compliance Framework Integration

The exporter specifically filters for ComplianceFramework.EU_AI_ACT reports (lines 95-104 in annex_iv.py), ensuring the Annex IV dossier reflects the same evidence that powers the internal audit trail and maintaining consistency between runtime compliance monitoring and regulatory documentation.

Data Flow and Export Process

The export process follows five distinct stages from runtime execution to final documentation:

1. Runtime Collection – Agents execute under the AGT policy engine; every tool call is logged to the audit log and SLO metrics are collected by Agent SRE.

2. Compliance Scanning – The compliance engine produces ComplianceReport objects for the EU AI Act framework, as documented in docs/compliance/eu-ai-act-checklist.md.

3. Evidence Discovery – EvidencePipeline.run() scans policies/ for YAML policy files, reads audit.jsonl, loads EU AI Act ComplianceReport JSON files, and optionally ingests SLO/SLI JSON.

4. Document Generation – TechnicalDocumentationExporter.export() constructs a fully-populated AnnexIVDocument, inserting placeholders marked _PLACEHOLDER or "DEPLOYER ACTION REQUIRED" where manual documentation is required for system design, risk registers, or robustness testing artefacts.

5. Persistence – EvidenceReport.save_markdown() writes the human-readable Annex IV dossier, while save_manifest() creates a machine-readable evidence inventory with cryptographic hashes for downstream auditors.

Implementation Examples

Programmatic Export with TechnicalDocumentationExporter

For embedded compliance workflows, instantiate the exporter directly and populate it with runtime artefacts:

from agentmesh.governance import TechnicalDocumentationExporter
from agentmesh.governance.compliance import ComplianceReport, ComplianceFramework
from agentmesh.governance.policy import Policy
from agentmesh.governance.audit import AuditEntry
from datetime import datetime, timezone

# Gather artefacts from runtime

compliance = ComplianceReport(
    framework=ComplianceFramework.EU_AI_ACT,
    compliance_score=92.5,
    total_controls=120,
    controls_met=115,
    controls_failed=5,
    period_start=datetime(2024, 1, 1, tzinfo=timezone.utc),
    period_end=datetime(2024, 6, 30, tzinfo=timezone.utc),
    violations=[],
    recommendations=[],
)

policy = Policy.from_yaml("""\
apiVersion: governance.toolkit/v1
name: high-risk-policy
default_action: deny
rules:
  - name: block-dangerous-tools
    condition: "action.type in ['shell_exec', 'delete_file']"
    action: deny
""")

audit = AuditEntry(
    event_type="tool_call",
    outcome="allowed",
    timestamp=datetime.now(timezone.utc),
    details={"tool": "web_search", "query": "latest AI news"},
)

# Configure exporter

exporter = TechnicalDocumentationExporter(
    system_name="Contoso Trading Agent",
    provider="Contoso Financial Inc.",
    system_description="Automated market-making for regulated securities.",
    system_version="1.4.2",
)

exporter.add_compliance_report(compliance)
exporter.add_policies([policy])
exporter.add_audit_entries([audit])

# Generate Annex IV document

doc = exporter.export()
print(doc.to_markdown())    # Human-readable

print(doc.json(indent=2))   # Machine-readable JSON

This approach utilizes the core implementation in annex_iv.py to construct compliant documentation without CLI dependencies.

CLI-Driven Evidence Pipeline

For CI/CD integration, use the EvidencePipeline to automatically discover and package artefacts:

from pathlib import Path
from agentmesh.governance.evidence_pipeline import EvidencePipeline

pipeline = EvidencePipeline(
    system_name="Contoso Trading Agent",
    provider="Contoso Financial Inc.",
    policies_dir=Path("policies/"),
    audit_log_path=Path("logs/audit.jsonl"),
    compliance_reports_dir=Path("reports/"),
    slo_data_path=Path("slo/metrics.json"),
)

report = pipeline.run()
report.save_markdown(Path("annex-iv-report.md"))
report.save_manifest(Path("annex-iv-manifest.json"))

The pipeline implementation in evidence_pipeline.py produces two artefacts: a Markdown dossier ready for regulatory submission and a JSON manifest containing SHA-256 digests and timestamps for tamper-evident audit trails.

Using the Built-in AGT CLI

If the agent-governance-copilot-cli package is installed, execute the export directly from the shell:

agt export-annex-iv \
    --system-name "Contoso Trading Agent" \
    --provider "Contoso Financial Inc." \
    --policies-dir policies/ \
    --audit-log logs/audit.jsonl \
    --compliance-dir reports/ \
    --slo-file slo/metrics.json \
    --out annex-iv-report.md

The CLI wrapper forwards arguments to EvidencePipeline and handles path resolution automatically, as documented in the CLI README.

Mapping Annex IV Requirements to AGT Components

The toolkit explicitly addresses each mandatory Annex IV section:

• System Description (Section 1) – Auto-generated from runtime metadata plus deployer placeholders for architecture diagrams and deployment context.
• Design and Development (Section 2) – Aggregates all governance policies from the policies directory; includes placeholders for technical specifications and development methodologies.
• Monitoring and Functioning (Section 3) – Compiles audit-trail statistics, SLO metrics, and compliance findings from the evidence pipeline to demonstrate operational monitoring.
• Risk Management (Section 4) – Extracts risk-related policy violations and mitigations; placeholders indicate where to attach the full risk register and risk treatment plans.
• Accuracy, Robustness, Cybersecurity (Section 5) – Pulls accuracy-related SLOs and security-event summaries; placeholders request testing artefacts and penetration test results.

The pipeline emits gap warnings when mandatory evidence is missing—such as absent policy files or missing EU AI Act reports—enabling teams to resolve compliance gaps before regulatory review.

Summary

• The TechnicalDocumentationExporter class in annex_iv.py serves as the primary interface for converting governance artefacts into Annex IV documentation.
• The EvidencePipeline automates discovery of policies, audit logs, compliance reports, and SLO metrics from the filesystem.
• Output formats include Markdown for human review and JSON manifest for machine-readable evidence inventory with cryptographic hashes.
• The system inserts placeholders for manual documentation that cannot be auto-generated, clearly marked for deployer attention.
• All components filter specifically for ComplianceFramework.EU_AI_ACT to ensure regulatory alignment.

Frequently Asked Questions

What file formats does the AnnexIVDocument export support?

The export generates two primary formats: a Markdown dossier via AnnexIVDocument.to_markdown() suitable for human-readable submission, and a JSON representation via the Pydantic model's .json() method. Additionally, EvidenceReport.save_manifest() produces a JSON manifest enumerating all evidence sources with SHA-256 hashes and metadata, creating a tamper-evident record for auditors.

How does the toolkit handle missing compliance evidence?

The EvidencePipeline and TechnicalDocumentationExporter emit gap warnings when required artefacts are absent—for example, if no EU AI Act compliance reports exist in the specified directory or if policy YAML files are missing. These warnings appear in the console output and within the generated EvidenceReport, allowing teams to identify and remediate documentation deficiencies before final submission.

Can I customize the Annex IV sections beyond the auto-generated content?

Yes. While the AnnexIVDocument model auto-populates sections based on discovered artefacts, it explicitly includes placeholder fields marked _PLACEHOLDER or "DEPLOYER ACTION REQUIRED" for system design specifications, risk registers, and testing documentation. Developers can programmatically modify these sections before calling to_markdown(), or manually edit the generated Markdown output while preserving the structured JSON manifest for audit purposes.

Where is the EvidencePipeline implementation located in the source code?

The EvidencePipeline class is implemented in agent-governance-python/agent-mesh/src/agentmesh/governance/evidence_pipeline.py (lines 4-18 for the class definition, lines 77-88 for EvidenceReport). This module orchestrates the discovery of governance artefacts and coordinates with TechnicalDocumentationExporter in agent-governance-python/agent-mesh/src/agentmesh/governance/annex_iv.py to produce the final compliance documentation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}