# How to Implement Custom Metrics Using EMF in CloudWatch

> Learn to implement custom CloudWatch metrics with EMF. Write structured JSON logs; the agent automatically extracts them as metrics. Simplify metric collection with AWS Agent Toolkit.

- Repository: [Amazon Web Services/agent-toolkit-for-aws](https://github.com/aws/agent-toolkit-for-aws)
- Tags: how-to-guide
- Published: 2026-07-03

---

**You implement custom CloudWatch metrics using the Embedded Metric Format (EMF) by writing structured JSON log events containing a top-level `_aws` block; the CloudWatch Logs agent automatically extracts these as metrics without requiring explicit `PutMetricData` API calls.**

The aws/agent-toolkit-for-aws repository provides reference implementations that demonstrate how to implement custom metrics using EMF in CloudWatch through AWS Lambda Powertools. This serverless-native approach enables high-resolution metric publishing via asynchronous log ingestion rather than synchronous API calls, reducing latency and operational complexity.

## Understanding the Embedded Metric Format (EMF)

EMF is a JSON specification that embeds metric definitions directly within log events. When your application writes a log entry containing a specific `_aws` metadata block, CloudWatch Logs automatically parses the event and creates corresponding metrics in the specified namespace.

According to the reference documentation in [`skills/core-skills/aws-observability/references/metrics.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/references/metrics.md), the EMF payload requires:

- A top-level `_aws` object containing the `Timestamp` and `CloudWatchMetrics` array
- Dimension definitions that map to key-value pairs in the log event
- Metric definitions specifying `Name`, `Unit`, and optional `StorageResolution`

## Implementing EMF Metrics in Python Lambda Functions

The toolkit provides a concrete implementation in [`skills/core-skills/aws-serverless/assets/powertools-handler.py`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-serverless/assets/powertools-handler.py) that demonstrates the production-ready pattern for emitting EMF metrics.

### Using the Metrics Decorator

The AWS Lambda Powertools library provides a `Metrics` class that automatically serializes custom metrics into EMF format when the function completes. The handler imports `Logger`, `Tracer`, and `Metrics` from `aws_lambda_powertools` and applies the `@metrics.log_metrics()` decorator:

```python

# File: skills/core-skills/aws-serverless/assets/powertools-handler.py

from aws_lambda_powertools import Logger, Metrics, Tracer
from aws_lambda_powertools.metrics import MetricUnit
from aws_lambda_powertools.utilities.typing import LambdaContext

logger = Logger()
tracer = Tracer()
metrics = Metrics()

@logger.inject_lambda_context()
@tracer.capture_lambda_handler
@metrics.log_metrics(capture_cold_start_metric=True)
def handler(event: dict, context: LambdaContext) -> dict:
    logger.info("Processing request")
    metrics.add_metric(name="RequestsProcessed",
                       unit=MetricUnit.Count,
                       value=1)
    return {"statusCode": 200, "body": "OK"}

```

The `@metrics.log_metrics(capture_cold_start_metric=True)` decorator accepts a `capture_cold_start_metric` parameter that automatically tracks initialization overhead. When the handler returns, Powertools flushes the metrics as a single EMF-formatted log line.

### Understanding the Generated EMF Payload

Under the hood, the `metrics.add_metric()` call generates JSON that CloudWatch recognizes as metric definitions. The resulting log entry contains:

```json
{
  "_aws": {
    "Timestamp": 1622547800000,
    "CloudWatchMetrics": [
      {
        "Namespace": "MyService",
        "Dimensions": [["ServiceName", "Environment"]],
        "Metrics": [
          { "Name": "Latency", "Unit": "Milliseconds", "StorageResolution": 60 },
          { "Name": "RequestCount", "Unit": "Count" }
        ]
      }
    ]
  },
  "ServiceName": "OrderService",
  "Environment": "Production",
  "Latency": 120,
  "RequestCount": 1
}

```

CloudWatch extracts the `Latency` and `RequestCount` values, associating them with the `ServiceName` and `Environment` dimensions in the `MyService` namespace.

## Production Best Practices for EMF Metrics

When implementing custom metrics using EMF in production workloads, follow these guidelines from the aws/agent-toolkit-for-aws observability references:

**Configure Flush Intervals**: Set the flush interval to **≤ 5 seconds** to ensure that CloudWatch alarms evaluate fresh data promptly. Powertools automatically batches metrics and flushes them at the end of each invocation or when the buffer size limit is reached.

**Limit Dimension Cardinality**: Keep dimensions low-cardinality (e.g., `ServiceName`, `Environment`, `Region`) to prevent metric explosion. High-cardinality values like user IDs or request IDs in dimensions can exponentially increase the number of unique metric series, impacting both costs and query performance.

**Monitor Retention Policies**: EMF metrics follow standard CloudWatch retention schedules—high-resolution data (1-second granularity) is retained for 3 hours before down-sampling to 1-minute aggregates, which are stored for 15 months.

## Troubleshooting EMF Parsing Errors

CloudWatch exposes operational metrics that indicate when EMF payloads fail validation. Monitor the `AWS/Logs` namespace for the following:

- **`EMFValidationErrors`**: Counts payloads that violate the EMF schema (e.g., missing required fields)
- **`EMFParsingErrors`**: Counts JSON syntax errors or malformed structures

Query these metrics using the AWS CLI to verify your implementation health:

```bash
aws cloudwatch get-metric-statistics \
    --namespace AWS/Logs \
    --metric-name EMFParsingErrors \
    --statistics Sum \
    --period 60 \
    --start-time $(date -u -d '5 minutes ago' +%Y-%m-%dT%H:%M:%SZ) \
    --end-time $(date -u +%Y-%m-%dT%H:%M:%SZ)

```

A non-zero sum indicates that your Lambda function is emitting malformed JSON, which requires investigation of the log payload structure.

## Summary

- **EMF eliminates API calls**: Implement custom metrics by writing structured logs rather than using `PutMetricData`, reducing latency and costs.
- **Use Powertools for automation**: The `Metrics` class in [`skills/core-skills/aws-serverless/assets/powertools-handler.py`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-serverless/assets/powertools-handler.py) handles EMF serialization automatically via the `@metrics.log_metrics()` decorator.
- **Structure matters**: Valid EMF requires a top-level `_aws` block containing `Timestamp`, `Namespace`, `Dimensions`, and `Metrics` definitions.
- **Monitor parsing health**: Track `EMFParsingErrors` and `EMFValidationErrors` in the `AWS/Logs` namespace to detect malformed payloads.
- **Control dimensions**: Keep cardinality low to avoid exponential metric growth and unexpected CloudWatch costs.

## Frequently Asked Questions

### What is the difference between EMF and standard PutMetricData API calls?

EMF publishes metrics asynchronously through CloudWatch Logs, while `PutMetricData` requires synchronous HTTP requests to the CloudWatch API. EMF reduces latency because your code only writes to stdout, and the CloudWatch Logs agent handles the metric extraction. This approach also correlates metrics with the original log event for easier debugging.

### How do I set custom dimensions when using AWS Lambda Powertools?

Use the `metrics.add_dimension()` method before calling `add_metric()`, or configure default dimensions during `Metrics` initialization. The toolkit handler demonstrates that dimensions must match the keys defined in your `CloudWatchMetrics` dimension sets. For example, if you define `[["ServiceName", "Environment"]]` in your EMF structure, both keys must exist as top-level JSON properties.

### Can I use EMF with containerized applications or only Lambda?

While the aws/agent-toolkit-for-aws repository focuses on Lambda implementations in [`powertools-handler.py`](https://github.com/aws/agent-toolkit-for-aws/blob/main/powertools-handler.py), EMF works with any workload that writes to CloudWatch Logs. Containerized applications using the CloudWatch agent or Fluent Bit can emit EMF-formatted logs. However, the Powertools library specifically targets Lambda runtimes and provides the `@metrics.log_metrics()` decorator for automatic flushing.

### What causes EMFParsingErrors and how do I fix them?

`EMFParsingErrors` occur when CloudWatch cannot parse the JSON structure—typically due to malformed syntax, missing required fields like `Timestamp` or `Namespace`, or invalid metric units. To fix these, validate your JSON payload against the EMF specification documented in [`skills/core-skills/aws-observability/references/metrics.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/references/metrics.md), and ensure your logging library isn't adding extra formatting or timestamps that corrupt the JSON structure.