# How to Configure Logging for Agents Built with the AWS Agent Toolkit

> Easily configure logging for agents built with the AWS Agent Toolkit. Leverage SDK-level wire debugging and automatic CloudWatch provisioning for robust monitoring.

- Repository: [Amazon Web Services/agent-toolkit-for-aws](https://github.com/aws/agent-toolkit-for-aws)
- Tags: how-to-guide
- Published: 2026-06-28

---

**The AWS Agent Toolkit enables comprehensive logging through two integrated layers: SDK-level wire debugging via Boto3 configuration and automatic CloudWatch provisioning for Lambda and API Gateway agents through specialized skills in the `plugins/aws-core/skills/` directory.**

The AWS Agent Toolkit provides an opinionated observability framework for agents deployed across AWS services. Whether you are debugging API calls during development or monitoring production workloads in Amazon CloudWatch, the toolkit standardizes logging configuration through concrete skill implementations. This guide references the actual source code in the `aws/agent-toolkit-for-aws` repository to demonstrate how to configure both Boto3 SDK diagnostics and application-level CloudWatch logging.

## SDK-Level Logging with Boto3 and Botocore

The AWS SDK for Python (Boto3) and its underlying Botocore library use the standard Python `logging` module to emit wire-level request and response details. The toolkit documents these patterns in [`plugins/aws-core/skills/aws-sdk-python-usage/SKILL.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/plugins/aws-core/skills/aws-sdk-python-usage/SKILL.md), providing specific helpers for different debugging scenarios.

### Stream Logging for Debug Output

To quickly dump all Botocore traffic to `stderr` during development, use the `set_stream_logger` method. This approach is ideal for troubleshooting permission errors or API latency issues in real time.

```python
import boto3

# Enable logging for all SDK components

boto3.set_stream_logger("")

# Or target only botocore internals

boto3.set_stream_logger("botocore")

```

### File Logging for Persistent Diagnostics

For long-running agents where you need to review logs after execution, the `set_file_logger` method writes debug output to a specified file path. This implementation is documented in the SDK usage skill and prevents log loss in ephemeral compute environments.

```python
from botocore.session import Session
import logging

session = Session()
session.set_file_logger(logging.DEBUG, "/tmp/botocore.log")

```

### Programmatic Configuration with botocore.config.Config

You can combine logging configuration with retry and timeout policies by using the `botocore.config.Config` class. This method allows you to register event hooks that adjust log levels dynamically for specific client instances.

```python
from botocore.config import Config
import boto3
import logging

cfg = Config(
    retries={"total_max_attempts": 2, "mode": "adaptive"},
    connect_timeout=5,
    read_timeout=10,
)

client = boto3.client("s3", config=cfg)

# Enable debug logging when requests are created

client.meta.events.register(
    "request-created", 
    lambda **_: logging.getLogger("botocore").setLevel(logging.DEBUG)
)

```

## Application-Level Logging to Amazon CloudWatch

When agents run as Lambda functions, Fargate tasks, or behind API Gateway, the toolkit automatically provisions the necessary CloudWatch infrastructure. The relevant skills handle IAM role attachment, log group creation, and retention policy configuration.

### Lambda Function Logging with aws-lambda-managed-instances

The `aws-lambda-managed-instances` skill automates CloudWatch setup for Lambda-based agents. According to [`plugins/aws-core/skills/aws-lambda-managed-instances/SKILL.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/plugins/aws-core/skills/aws-lambda-managed-instances/SKILL.md), this skill creates the log group `/aws/lambda/<function-name>`, attaches the `AWSLambdaBasicExecutionRole` (which includes `logs:CreateLogGroup`, `logs:CreateLogStream`, and `logs:PutLogEvents`), and configures a default 30-day retention period.

The skill accepts a `log_retention` parameter that allows you to customize how long logs persist before automatic deletion.

### API Gateway Access and Execution Logging

For agents exposed via REST APIs, the `creating-api-gateway-stage` skill configures both access logs and execution logs. As documented in [`skills/specialized-skills/serverless-skills/creating-api-gateway-stage/references/create-api-gateway-stage.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/specialized-skills/serverless-skills/creating-api-gateway-stage/references/create-api-gateway-stage.md), this skill emits the exact CLI commands required to enable CloudWatch logging and define structured JSON log formats for API requests.

### IAM Permissions and Log Retention Policies

All CloudWatch logging implementations in the toolkit follow a consistent three-step pattern:

1. **Create or validate the CloudWatch log group** using `aws logs create-log-group`
2. **Attach IAM permissions** via the `AWSLambdaBasicExecutionRole` or a custom policy with CloudWatch Logs permissions
3. **Set the retention period** (commonly 30 days) to control storage costs

The `debugging-lambda-timeouts` skill in [`skills/specialized-skills/serverless-skills/debugging-lambda-timeouts/SKILL.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/specialized-skills/serverless-skills/debugging-lambda-timeouts/SKILL.md) explicitly reminds developers to verify that the Lambda function's log group exists and that logging is not disabled, ensuring observability during troubleshooting sessions.

## End-to-End Logging Configuration Example

Below is a complete implementation that combines SDK-level debugging with CloudWatch application logging for a Lambda-based agent. The toolkit handles the CloudWatch infrastructure, so you only need to configure the Python logger and Boto3 debug settings.

```python
import logging
import boto3
from botocore.config import Config

# SDK-level logging: capture wire traffic to stderr

boto3.set_stream_logger("botocore")

# Configure client with retry logic

cfg = Config(
    retries={"total_max_attempts": 2, "mode": "adaptive"},
    connect_timeout=5,
    read_timeout=10,
)
s3_client = boto3.client("s3", config=cfg)

def lambda_handler(event, context):
    """
    Handler function with CloudWatch logging enabled.
    The aws-lambda-managed-instances skill automatically creates:
    - Log group: /aws/lambda/<function-name>
    - IAM role with CloudWatch permissions
    - 30-day retention policy
    """
    logger = logging.getLogger()
    logger.setLevel(logging.INFO)
    
    logger.info("Agent processing started", extra={"event_id": event.get("id")})
    
    try:
        response = s3_client.list_buckets()
        logger.debug("Retrieved %d buckets", len(response.get("Buckets", [])))
        return {"status": "success", "bucket_count": len(response["Buckets"])}
    except Exception as exc:
        logger.error("S3 operation failed: %s", exc, exc_info=True)
        raise

```

## Summary

- **SDK-level logging** is configured via `boto3.set_stream_logger()` or `set_file_logger()` and documented in [`plugins/aws-core/skills/aws-sdk-python-usage/SKILL.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/plugins/aws-core/skills/aws-sdk-python-usage/SKILL.md).
- **CloudWatch logging** for Lambda agents is automated by the `aws-lambda-managed-instances` skill, which provisions log groups, IAM roles, and retention policies.
- **API Gateway logging** is handled by the `creating-api-gateway-stage` skill, which configures access and execution logs through standardized CLI commands.
- All logging configurations require the `AWSLambdaBasicExecutionRole` or equivalent IAM permissions to write to CloudWatch Logs.
- The toolkit standardizes log retention (default 30 days) to balance observability needs with storage costs.

## Frequently Asked Questions

### How do I enable debug logging for Boto3 calls only?

Use `boto3.set_stream_logger("botocore")` to limit output to Botocore internals, or register an event hook on a specific client instance via `client.meta.events.register()` as shown in [`plugins/aws-core/skills/aws-sdk-python-usage/SKILL.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/plugins/aws-core/skills/aws-sdk-python-usage/SKILL.md). This targets only the AWS SDK rather than your entire application.

### What IAM permissions are required for CloudWatch logging?

The toolkit automatically attaches the `AWSLambdaBasicExecutionRole` managed policy, which includes `logs:CreateLogGroup`, `logs:CreateLogStream`, and `logs:PutLogEvents`. For custom roles, ensure these permissions are granted to allow the agent to write to CloudWatch Logs.

### How do I configure log retention for Lambda functions?

When using the `aws-lambda-managed-instances` skill, pass the `log_retention` parameter (in days) to override the default 30-day retention period. The skill executes `aws logs put-retention-policy` automatically during deployment.

### Can I log API Gateway requests to CloudWatch?

Yes. The `creating-api-gateway-stage` skill configures both access logging (full request/response details) and execution logging (API Gateway internal operations) by creating dedicated CloudWatch log groups and setting the appropriate stage variables, as documented in the skill's reference file.