Running Release Readiness Analysis with AI Agents Using the AWS Agent Toolkit

The AWS Agent Toolkit for AWS provides a plugin-based ecosystem that enables AI coding agents to execute end-to-end release readiness assessments, scoring AWS workloads on a 0-100 scale across seven dimensions using declarative markdown skills and MCP Server tools.

The AWS Agent Toolkit for AWS (aws/agent-toolkit-for-aws) is an open-source repository that supplies AI agents with authenticated, sandboxed access to AWS services. It enables running release readiness analysis with AI agents through a declarative skill system that automates cost, security, compliance, and operational checks without requiring custom code in your project.

Architecture of the Release Readiness System

The Toolkit implements a four-layer architecture that separates authentication, orchestration, business logic, and validation.

MCP Server Layer

The foundation is the Model Context Protocol (MCP) Server, which exposes four critical low-level tools: call_aws, run_script, search_documentation, and retrieve_skill. These tools provide authenticated AWS API access and sandboxed execution environments for agent-driven scripts.

Plugins and Skills

The aws-agents-for-devsecops plugin (located in plugins/aws-agents-for-devsecops/) bundles the release readiness skill set. Skills are markdown-based instruction packages stored under skills/ directories that define assessment workflows and reference documents like readiness-rubric.md.

Validation Infrastructure

Before deployment, tools/validate.py ensures every plugin manifest and skill front-matter conforms to the required schema. This validator enforces kebab-case naming conventions, description length limits, and directory-to-name consistency.

How the Release Readiness Assessment Works

The assessment follows a five-stage workflow defined in skills/specialized-skills/analytics-skills/amazon-opensearch-service/references/assessment-workflow.md:

1. Skill Discovery – The agent invokes retrieve_skill to load the release-readiness skill from the devsecops plugin.
2. Data Gathering – The skill executes call_aws to poll service configurations via APIs like lambda:GetFunctionConfiguration and cloudwatch:GetMetricData. It may also run sandboxed Python scripts via run_script.
3. Scoring – Using the canonical 7-dimension rubric defined in readiness-rubric.md, the skill aggregates metrics and applies risk-blocker overrides.
4. Tier Assignment – The final numeric score (0-100) maps to a GREEN/YELLOW/RED tier.
5. Remediation Linking – The agent optionally chains to remediation skills like aws-agents-for-devsecops:remediate-issues.

Installing the DevSecOps Plugin

Install the plugin for your specific AI agent to enable release readiness capabilities.

For Claude Code:

/plugin install aws-agents-for-devsecops@claude-plugins-official
/plugin reload-plugins

For Codex:

codex plugin marketplace add aws/agent-toolkit-for-aws
codex plugins install aws-agents-for-devsecops

For generic skill pack installation:

npx skills add aws/agent-toolkit-for-aws/skills

Triggering a Readiness Assessment

Invoke the skill from your agent's CLI using natural language. For Claude Code:

/ask "run release-readiness on my Lambda function arn:aws:lambda:us-west-2:123456789012:function:my-app"

Behind the scenes, the skill executes sandboxed collection scripts like collect_metrics.py (located in plugins/aws-agents-for-devsecops/skills/.../scripts/):

# collect_metrics.py – run inside the MCP sandbox

import json, os
from mcp import call_aws  # MCP-provided helper

def get_lambda_config(arn):
    resp = call_aws("lambda", "GetFunctionConfiguration", {"FunctionName": arn})
    return resp

def get_error_rate(arn):
    # Pull CloudWatch metrics for the function's error count

    query = {
        "Namespace": "AWS/Lambda",
        "MetricName": "Errors",
        "Dimensions": [{"Name": "FunctionName", "Value": arn.split(":")[-1]}],
        "StartTime": "2024-07-01T00:00:00Z",
        "EndTime": "2024-07-01T23:59:59Z",
        "Period": 300,
        "Statistics": ["Sum"],
    }
    return call_aws("cloudwatch", "GetMetricData", {"MetricDataQueries": [query]})

if __name__ == "__main__":
    arn = os.getenv("TARGET_ARN")
    cfg = get_lambda_config(arn)
    err = get_error_rate(arn)
    print(json.dumps({"config": cfg, "errors": err}))

Understanding the Readiness Score

The readiness rubric in skills/specialized-skills/analytics-skills/amazon-opensearch-service/references/readiness-rubric.md evaluates seven dimensions:

• Operational readiness – On-call coverage and monitoring status
• Compatibility – API version conflicts and dependency validation
• Cost confidence – Budget variance against estimates
• Security posture – IAM policy and encryption compliance
• Compliance – Regulatory requirement adherence
• Performance – Latency and throughput baselines
• Reliability – Error rates and automated recovery capabilities

The skill aggregates these into a final score mapped to release tiers:

• GREEN (80-100): Proceed to release
• YELLOW (60-79): Address recommendations before release
• RED (0-59): Resolve blocking issues

Summary

• The AWS Agent Toolkit exposes MCP Server tools (call_aws, run_script, retrieve_skill) that provide AI agents with authenticated AWS access.
• The aws-agents-for-devsecops plugin packages the release readiness skill set, with manifests validated by tools/validate.py.
• Assessments follow the declarative workflow in assessment-workflow.md, scoring against the seven-dimension rubric in readiness-rubric.md.
• Integration requires only plugin installation and natural language prompts—no custom infrastructure code is necessary.

Frequently Asked Questions

What AI agents are compatible with the AWS Agent Toolkit?

The Toolkit supports Claude Code, Codex, Cursor, Kiro, and any agent implementing the Model Context Protocol. Each agent consumes the same skill manifests located in the skills/ directory, though installation commands vary by platform.

How is the readiness score calculated?

The skill uses the canonical 7-dimension rubric defined in readiness-rubric.md. It polls AWS services via call_aws, aggregates metrics across operational, security, cost, and compatibility dimensions, applies risk-blocker weights, and emits a normalized score from 0-100 mapped to GREEN/YELLOW/RED tiers.

Can I customize the readiness assessment for my organization?

Yes. The skill system is declarative and markdown-based. You can fork the aws-agents-for-devsecops plugin, modify the reference documents in the skills/ directory, and update scoring logic. Run tools/validate.py before deployment to ensure schema compliance.

What AWS permissions does the agent require?

The MCP Server requires IAM permissions for the specific services being assessed. For Lambda readiness checks, the agent needs lambda:GetFunctionConfiguration and cloudwatch:GetMetricData. The Toolkit uses temporary credentials and sandboxed execution to minimize privilege escalation risks.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}