How to Audit AWS Operations Using CloudTrail: A Complete Operational Guide

CloudTrail records every AWS API call in your account, creating an immutable audit trail that shows who did what, when, and where by capturing management events, data events, and optional Insight events for downstream analysis.

In the aws/agent-toolkit-for-aws repository, CloudTrail is configured specifically for operational auditing rather than security threat detection, combining management-event logs with optional data-event selectors and analytics pipelines using Amazon Athena or CloudWatch Alarms. This guide walks through the exact implementation patterns found in the toolkit's observability skills, showing you how to set up trails, query activity, and monitor critical changes in real time.

Setting Up CloudTrail for Operational Auditing

Creating a Multi-Region Trail

Start by establishing a single source of truth for API activity across all regions. According to the setting-up-cloudtrail-multi-region skill, you should enable log file validation to ensure log integrity:

aws cloudtrail create-trail \
  --name org-trail \
  --s3-bucket-name org-cloudtrail-logs \
  --is-multi-region-trail \
  --include-global-service-events \
  --enable-log-file-validation \
  --region us-east-1

aws cloudtrail start-logging --name org-trail --region us-east-1

This configuration captures global service events (like IAM or Route 53) alongside regional activity, writing all logs to the specified S3 bucket with cryptographic validation enabled.

Enabling Data Event Selectors

By default, CloudTrail only logs management events (control-plane actions). To audit data-plane operations—such as S3 object reads or Lambda invocations—you must explicitly configure data event selectors as documented in skills/core-skills/aws-observability/references/cloudtrail.md:

aws cloudtrail put-event-selectors \
  --trail-name org-trail \
  --event-selectors '[{
    "ReadWriteType":"All",
    "IncludeManagementEvents":true,
    "DataResources":[
      {"Type":"AWS::S3::Object","Values":["arn:aws:s3"]},
      {"Type":"AWS::Lambda::Function","Values":["arn:aws:lambda"]}
    ]
  }]'

This selector captures all read and write operations on S3 objects and Lambda function invocations, providing the granular visibility needed to answer questions like "Who accessed this specific object?"

Querying CloudTrail Logs for Investigations

Real-Time Lookups with AWS CLI

For immediate operational debugging, use the lookup-events API to search recent activity without waiting for S3 delivery. As shown in the CloudTrail reference file, filter by specific event names to track destructive operations:

aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventName,AttributeValue=DeleteBucket \
  --start-time 2026-04-20T00:00:00Z

This returns the 50 most recent matching events within the specified timeframe, including user identity, source IP address, and request parameters—ideal for quick incident response.

SQL Analysis with Amazon Athena

For historical analysis across large volumes, Athena queries the raw JSON logs stored in S3 directly. The cloudtrail.md reference provides this optimized query structure for auditing bucket deletions:

SELECT eventTime, userIdentity.arn, sourceIPAddress, eventName
FROM cloudtrail_logs
WHERE eventName = 'DeleteBucket'
  AND eventTime > '2026-04-20'
ORDER BY eventTime DESC
LIMIT 100;

This approach scales to terabytes of logs without loading data into a separate database, using standard SQL to correlate events across time ranges and services.

Real-Time Monitoring with CloudWatch Alarms

Configuring Metric Filters

Transform CloudTrail logs into actionable metrics by streaming the trail to CloudWatch Logs and creating metric filters. This pattern from the cloudtrail.md reference file monitors for specific API calls:

aws logs put-metric-filter \
  --log-group-name /aws/cloudtrail/org-trail \
  --filter-name DeleteBucketFilter \
  --filter-pattern '{ $.eventName = "DeleteBucket" }' \
  --metric-transformations metricName=DeleteBucketCount,metricNamespace=CloudTrail,metricValue=1

The filter pattern uses JSON-path syntax to match the eventName field, incrementing a custom metric each time a matching event occurs.

Setting Up Notifications

Attach alarms to these metrics for immediate alerting via SNS when critical operations occur:

aws cloudwatch put-metric-alarm \
  --alarm-name DeleteBucketAlarm \
  --metric-name DeleteBucketCount \
  --namespace CloudTrail \
  --threshold 1 \
  --evaluation-periods 1 \
  --comparison-operator GreaterThanOrEqualToThreshold \
  --alarm-actions arn:aws:sns:us-east-1:123456789012:OpsAlerts

This creates a real-time notification pipeline that triggers within minutes of an API call, bridging the gap between audit logging and operational response.

Required IAM Permissions

The toolkit's iam-permissions.md file specifies minimal permissions for CloudTrail auditing:

• cloudtrail:LookupEvents – For CLI lookups and console investigations
• cloudtrail:DescribeTrails – To verify trail configuration
• cloudtrail:GetEventSelectors – To audit current data event coverage
• cloudtrail:PutEventSelectors – To modify event capture settings

Additionally, querying S3-stored logs requires s3:GetObject on the trail bucket, and Athena queries need athena:StartQueryExecution and glue:GetTable permissions.

Summary

• Enable multi-region trails with log file validation to ensure complete, tamper-evident coverage of all API activity across your AWS account.
• Configure data event selectors when you need to audit data-plane operations like S3 object access or Lambda invocations beyond the default management events.
• Use lookup-events for immediate investigations and Athena for complex historical analysis across large log volumes stored in S3.
• Implement CloudWatch metric filters on CloudTrail logs to convert audit events into real-time alarms for operational monitoring.
• Apply least-privilege IAM policies using the specific actions listed in the toolkit's permission matrix to secure access to audit data.

Frequently Asked Questions

What is the difference between management events and data events in CloudTrail?

Management events capture control-plane operations like creating EC2 instances or modifying IAM policies, and are logged by default. Data events record data-plane activity such as S3 object reads or DynamoDB item modifications, and must be explicitly enabled via event selectors because they generate significantly higher volumes of log data.

How long does CloudTrail retain logs in S3?

CloudTrail delivers events to S3 within approximately 5 minutes of the API call, but retention is determined by your S3 bucket lifecycle policies, not CloudTrail itself. As implemented in the aws/agent-toolkit-for-aws patterns, you should configure S3 lifecycle rules to transition logs to Glacier after 90 days and expire them based on your compliance requirements.

Can I use CloudTrail for security threat detection?

While CloudTrail provides the raw event data needed for security analysis, the Agent Toolkit specifically positions CloudTrail for operational auditing rather than threat detection. For security-grade monitoring, you should enable GuardDuty and IAM Access Analyzer alongside CloudTrail, using the audit logs as forensic data sources rather than real-time threat indicators.

What is the most efficient way to query large CloudTrail log volumes?

For ad-hoc queries across terabytes of historical data, Amazon Athena is the most efficient approach because it scans JSON files directly in S3 using standard SQL without requiring data ingestion. For frequent, repetitive queries, consider using CloudWatch Logs Insights on a trail configured to send logs to CloudWatch, which offers faster query performance for recent data at a higher cost per gigabyte.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}