# How to Publish Custom CloudWatch Metrics Using the EMF Format

> Easily publish custom CloudWatch metrics with EMF format. Send logs to CloudWatch for automatic metric extraction without API calls. Learn more now.

- Repository: [Amazon Web Services/agent-toolkit-for-aws](https://github.com/aws/agent-toolkit-for-aws)
- Tags: how-to-guide
- Published: 2026-06-26

---

**Write a JSON object containing an `_aws` field with a `CloudWatchMetrics` array to standard output, allowing CloudWatch Logs to automatically extract custom metrics without API calls.**

The aws/agent-toolkit-for-aws repository defines the Embedded Metric Format (EMF) as the recommended approach for publishing custom CloudWatch metrics from Lambda functions and containerized workloads. According to the [`skills/core-skills/aws-observability/SKILL.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/SKILL.md) file, EMF lets you emit metrics through log entries rather than direct API calls. This approach reduces overhead while maintaining compatibility with CloudWatch alarms and dashboards.

## Understanding the Embedded Metric Format

The Embedded Metric Format is a JSON specification that you write to stdout (or your container's log stream). The CloudWatch Logs agent parses these entries and creates corresponding metrics in your specified namespace.

### How EMF Works

When you publish custom CloudWatch metrics using the EMF format, three components work together:

1. **Log Event Payload** – Your application writes a JSON object containing an `_aws` section. This section includes a `CloudWatchMetrics` array that defines the namespace, dimensions, and metric metadata, plus a `Timestamp` field expressed in milliseconds since the epoch.

2. **CloudWatch Logs Agent** – The service validates the JSON against the EMF schema and creates the metric. If validation fails, CloudWatch emits `EMFValidationErrors` or `EMFParsingErrors` metrics to the `AWS/Logs` namespace, which you can monitor to detect malformed submissions.

3. **Flushing Mechanism** – In Lambda, libraries like Powertools automatically flush EMF payloads after every invocation (within 5 seconds). For containers, you must configure the log driver to flush at least every 5 seconds to ensure reliable alarm evaluations.

### EMF Constraints and Limits

The [`skills/core-skills/aws-observability/references/metrics.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/references/metrics.md) file defines specific constraints that your EMF payloads must satisfy:

- **Namespace**: Must be a valid CloudWatch namespace (e.g., `MyApp`).
- **Metric Name**: Maximum 255 characters, alphanumeric and underscores only.
- **Dimensions**: Maximum 10 per metric; dimension keys limited to 250 characters, values to 1024 characters.
- **Timestamp**: Required; expressed in milliseconds. If omitted, CloudWatch uses ingestion time, which may cause clock-skew issues.
- **Payload Size**: Maximum 256 KB per log event.
- **Flush Interval**: Must be less than or equal to 5 seconds for reliable alarm evaluation.

## Publishing EMF Metrics from AWS Lambda

For Lambda functions, use the AWS Lambda Powertools (Node.js) or aws-embedded-metrics (Python) libraries to handle the JSON construction automatically.

### Node.js with Lambda Powertools

```javascript
import { Metrics, MetricUnits } from '@aws-lambda-powertools/metrics';

const metrics = new Metrics({ namespace: 'MyApp', serviceName: 'OrderService' });

export const handler = async (event) => {
  metrics.addMetric('OrderCount', MetricUnits.Count, 1);
  metrics.addDimension('Region', 'us-east-1');
  
  await metrics.flush();
  
  return { statusCode: 200 };
};

```

The Powertools library builds the `_aws` block, sets the timestamp, and writes the EMF JSON to the Lambda log stream automatically. This eliminates manual JSON construction and ensures compliance with the 256 KB payload limit.

### Python with aws-embedded-metrics

```python
from aws_embedded_metrics import metric_scope, MetricUnit

@metric_scope
def handler(event, context, metrics):
    metrics.set_namespace("MyApp")
    metrics.put_dimensions({"Region": "us-east-1"})
    metrics.put_metric("OrderCount", 1, MetricUnit.Count)
    
    return {"statusCode": 200}

```

No explicit flush is required; the library writes the EMF payload on function exit. The decorator handles the `_aws` metadata construction and ensures the millisecond timestamp is correctly formatted.

## Publishing EMF Metrics from Containerized Workloads

For containers or custom runtimes, construct the EMF JSON manually and write it to stdout.

### Manual EMF JSON Structure

```json
{
  "_aws": {
    "Timestamp": 1729974000000,
    "CloudWatchMetrics": [
      {
        "Namespace": "MyApp",
        "Dimensions": [["Region"]],
        "Metrics": [
          {"Name": "OrderCount", "Unit": "Count"}
        ]
      }
    ]
  },
  "OrderCount": 1,
  "Region": "us-east-1"
}

```

Write this JSON string to your container's standard output. The CloudWatch Logs service extracts the metric automatically.

### ADOT Collector Configuration

For containerized environments using the AWS Distro for OpenTelemetry (ADOT) collector, the [`skills/core-skills/aws-observability/assets/otel-config.yaml`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/assets/otel-config.yaml) file provides a configuration that forwards EMF-derived metrics to CloudWatch.

## EMF vs PutMetricData: Choosing the Right Approach

According to the [`metrics.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/metrics.md) reference file in the aws/agent-toolkit-for-aws repository, you should prefer EMF for Lambda and container workloads, while reserving `PutMetricData` API calls for batch jobs requiring synchronous confirmation. This recommendation appears in the observability skill documentation alongside specific constraints for metric names, dimensions, and payload sizes.

**EMF advantages:**
- **Low overhead** – One log write replaces multiple `PutMetricData` API calls.
- **Automatic dimension handling** – Libraries enforce the 10-dimension limit and character constraints.
- **Built-in aggregation** – CloudWatch aggregates metrics across invocations, enabling standard alarms and dashboards.

Use `PutMetricData` only when you need immediate confirmation of metric delivery or are running batch processes outside of Lambda or container environments.

## Monitoring EMF Validation Errors

When you publish custom CloudWatch metrics using the EMF format, validation failures do not throw runtime errors in your application. Instead, CloudWatch Logs emits `EMFValidationErrors` and `EMFParsingErrors` metrics to the `AWS/Logs` namespace. Create alarms on these metrics to detect malformed payloads, namespace violations, or dimension limit breaches in production.

## Summary

- **Write JSON to stdout** – EMF requires a specific JSON structure with an `_aws` section containing `CloudWatchMetrics` metadata and a millisecond timestamp.
- **Respect limits** – Adhere to the 256 KB payload size, 10-dimension maximum, and 5-second flush intervals defined in [`skills/core-skills/aws-observability/references/metrics.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/references/metrics.md).
- **Use libraries for Lambda** – Leverage Lambda Powertools (Node.js) or aws-embedded-metrics (Python) to handle JSON construction and flushing automatically.
- **Monitor validation** – Watch `EMFValidationErrors` in the `AWS/Logs` namespace to catch malformed metric submissions.
- **Prefer over PutMetricData** – Choose EMF for Lambda and containers; reserve `PutMetricData` for synchronous batch job requirements.

## Frequently Asked Questions

### What is the maximum payload size for EMF log events?

The maximum payload size for a single EMF log event is **256 KB**. Exceeding this limit causes CloudWatch to reject the log entry entirely, preventing metric extraction. For high-cardinality data, aggregate metrics in your application before emitting them rather than sending individual records.

### How do I troubleshoot EMF metrics that are not appearing in CloudWatch?

Check the `EMFValidationErrors` and `EMFParsingErrors` metrics in the `AWS/Logs` namespace. These metrics increment when your JSON structure violates the EMF schema, such as missing the `_aws` section, malformed timestamps (which must be in milliseconds), or invalid namespace names. Additionally, verify that your log driver flushes within the required 5-second window.

### Can I use EMF with programming languages other than Node.js and Python?

Yes. While Node.js and Python have official libraries (`@aws-lambda-powertools/metrics` and `aws-embedded-metrics`), you can implement EMF in any language by manually constructing the JSON payload and writing it to standard output. Ensure your JSON includes the `_aws` field with a valid `CloudWatchMetrics` array and a millisecond-precision timestamp.

### When should I use PutMetricData instead of EMF?

Use `PutMetricData` only for batch jobs or scenarios requiring synchronous confirmation of metric delivery. According to the aws/agent-toolkit-for-aws observability documentation, EMF is the recommended default for Lambda functions and containerized workloads because it reduces API costs and handles aggregation automatically, while `PutMetricData` incurs API charges and requires manual batching for efficiency.