How to Implement Custom Metrics Using EMF in CloudWatch

You implement custom CloudWatch metrics using the Embedded Metric Format (EMF) by writing structured JSON log events containing a top-level _aws block; the CloudWatch Logs agent automatically extracts these as metrics without requiring explicit PutMetricData API calls.

The aws/agent-toolkit-for-aws repository provides reference implementations that demonstrate how to implement custom metrics using EMF in CloudWatch through AWS Lambda Powertools. This serverless-native approach enables high-resolution metric publishing via asynchronous log ingestion rather than synchronous API calls, reducing latency and operational complexity.

Understanding the Embedded Metric Format (EMF)

EMF is a JSON specification that embeds metric definitions directly within log events. When your application writes a log entry containing a specific _aws metadata block, CloudWatch Logs automatically parses the event and creates corresponding metrics in the specified namespace.

According to the reference documentation in skills/core-skills/aws-observability/references/metrics.md, the EMF payload requires:

• A top-level _aws object containing the Timestamp and CloudWatchMetrics array
• Dimension definitions that map to key-value pairs in the log event
• Metric definitions specifying Name, Unit, and optional StorageResolution

Implementing EMF Metrics in Python Lambda Functions

The toolkit provides a concrete implementation in skills/core-skills/aws-serverless/assets/powertools-handler.py that demonstrates the production-ready pattern for emitting EMF metrics.

Using the Metrics Decorator

The AWS Lambda Powertools library provides a Metrics class that automatically serializes custom metrics into EMF format when the function completes. The handler imports Logger, Tracer, and Metrics from aws_lambda_powertools and applies the @metrics.log_metrics() decorator:

# File: skills/core-skills/aws-serverless/assets/powertools-handler.py

from aws_lambda_powertools import Logger, Metrics, Tracer
from aws_lambda_powertools.metrics import MetricUnit
from aws_lambda_powertools.utilities.typing import LambdaContext

logger = Logger()
tracer = Tracer()
metrics = Metrics()

@logger.inject_lambda_context()
@tracer.capture_lambda_handler
@metrics.log_metrics(capture_cold_start_metric=True)
def handler(event: dict, context: LambdaContext) -> dict:
    logger.info("Processing request")
    metrics.add_metric(name="RequestsProcessed",
                       unit=MetricUnit.Count,
                       value=1)
    return {"statusCode": 200, "body": "OK"}

The @metrics.log_metrics(capture_cold_start_metric=True) decorator accepts a capture_cold_start_metric parameter that automatically tracks initialization overhead. When the handler returns, Powertools flushes the metrics as a single EMF-formatted log line.

Understanding the Generated EMF Payload

Under the hood, the metrics.add_metric() call generates JSON that CloudWatch recognizes as metric definitions. The resulting log entry contains:

{
  "_aws": {
    "Timestamp": 1622547800000,
    "CloudWatchMetrics": [
      {
        "Namespace": "MyService",
        "Dimensions": [["ServiceName", "Environment"]],
        "Metrics": [
          { "Name": "Latency", "Unit": "Milliseconds", "StorageResolution": 60 },
          { "Name": "RequestCount", "Unit": "Count" }
        ]
      }
    ]
  },
  "ServiceName": "OrderService",
  "Environment": "Production",
  "Latency": 120,
  "RequestCount": 1
}

CloudWatch extracts the Latency and RequestCount values, associating them with the ServiceName and Environment dimensions in the MyService namespace.

Production Best Practices for EMF Metrics

When implementing custom metrics using EMF in production workloads, follow these guidelines from the aws/agent-toolkit-for-aws observability references:

Configure Flush Intervals: Set the flush interval to ≤ 5 seconds to ensure that CloudWatch alarms evaluate fresh data promptly. Powertools automatically batches metrics and flushes them at the end of each invocation or when the buffer size limit is reached.

Limit Dimension Cardinality: Keep dimensions low-cardinality (e.g., ServiceName, Environment, Region) to prevent metric explosion. High-cardinality values like user IDs or request IDs in dimensions can exponentially increase the number of unique metric series, impacting both costs and query performance.

Monitor Retention Policies: EMF metrics follow standard CloudWatch retention schedules—high-resolution data (1-second granularity) is retained for 3 hours before down-sampling to 1-minute aggregates, which are stored for 15 months.

Troubleshooting EMF Parsing Errors

CloudWatch exposes operational metrics that indicate when EMF payloads fail validation. Monitor the AWS/Logs namespace for the following:

• EMFValidationErrors: Counts payloads that violate the EMF schema (e.g., missing required fields)
• EMFParsingErrors: Counts JSON syntax errors or malformed structures

Query these metrics using the AWS CLI to verify your implementation health:

aws cloudwatch get-metric-statistics \
    --namespace AWS/Logs \
    --metric-name EMFParsingErrors \
    --statistics Sum \
    --period 60 \
    --start-time $(date -u -d '5 minutes ago' +%Y-%m-%dT%H:%M:%SZ) \
    --end-time $(date -u +%Y-%m-%dT%H:%M:%SZ)

A non-zero sum indicates that your Lambda function is emitting malformed JSON, which requires investigation of the log payload structure.

Summary

• EMF eliminates API calls: Implement custom metrics by writing structured logs rather than using PutMetricData, reducing latency and costs.
• Use Powertools for automation: The Metrics class in skills/core-skills/aws-serverless/assets/powertools-handler.py handles EMF serialization automatically via the @metrics.log_metrics() decorator.
• Structure matters: Valid EMF requires a top-level _aws block containing Timestamp, Namespace, Dimensions, and Metrics definitions.
• Monitor parsing health: Track EMFParsingErrors and EMFValidationErrors in the AWS/Logs namespace to detect malformed payloads.
• Control dimensions: Keep cardinality low to avoid exponential metric growth and unexpected CloudWatch costs.

Frequently Asked Questions

What is the difference between EMF and standard PutMetricData API calls?

EMF publishes metrics asynchronously through CloudWatch Logs, while PutMetricData requires synchronous HTTP requests to the CloudWatch API. EMF reduces latency because your code only writes to stdout, and the CloudWatch Logs agent handles the metric extraction. This approach also correlates metrics with the original log event for easier debugging.

How do I set custom dimensions when using AWS Lambda Powertools?

Use the metrics.add_dimension() method before calling add_metric(), or configure default dimensions during Metrics initialization. The toolkit handler demonstrates that dimensions must match the keys defined in your CloudWatchMetrics dimension sets. For example, if you define [["ServiceName", "Environment"]] in your EMF structure, both keys must exist as top-level JSON properties.

Can I use EMF with containerized applications or only Lambda?

While the aws/agent-toolkit-for-aws repository focuses on Lambda implementations in powertools-handler.py, EMF works with any workload that writes to CloudWatch Logs. Containerized applications using the CloudWatch agent or Fluent Bit can emit EMF-formatted logs. However, the Powertools library specifically targets Lambda runtimes and provides the @metrics.log_metrics() decorator for automatic flushing.

What causes EMFParsingErrors and how do I fix them?

EMFParsingErrors occur when CloudWatch cannot parse the JSON structure—typically due to malformed syntax, missing required fields like Timestamp or Namespace, or invalid metric units. To fix these, validate your JSON payload against the EMF specification documented in skills/core-skills/aws-observability/references/metrics.md, and ensure your logging library isn't adding extra formatting or timestamps that corrupt the JSON structure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}