# Publishing Custom Metrics Using AWS Embedded Metrics Format (EMF): A Complete Guide

> Publish custom CloudWatch metrics with AWS Embedded Metrics Format EMF. Embed metric data in logs, reduce costs, and eliminate per-metric API calls. Your complete guide.

- Repository: [Amazon Web Services/agent-toolkit-for-aws](https://github.com/aws/agent-toolkit-for-aws)
- Tags: how-to-guide
- Published: 2026-07-02

---

**AWS Embedded Metrics Format (EMF) lets you publish custom CloudWatch metrics by embedding metric data directly inside structured log events, eliminating per-metric API calls and reducing costs.**

The Agent Toolkit for AWS provides a comprehensive reference implementation that defines the EMF JSON schema, operational limits, and best-practice constraints for asynchronous metric publishing. This approach enables Lambda and container workloads to emit metrics via CloudWatch Logs without blocking on network requests.

## How EMF Works in the Agent Toolkit

EMF operates by injecting a specialized metadata block into standard JSON log events. According to the Toolkit's reference documentation in [`skills/core-skills/aws-observability/references/metrics.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/references/metrics.md), the mechanism relies on four key components:

- **Log Event** – A JSON line written to CloudWatch Logs containing an `_aws` metadata block that describes one or more **MetricDirectives** (namespace, dimensions, and metric definitions).
- **`_aws.CloudWatchMetrics`** – An array of metric directives where each directive supports up to **100 metrics** and **30 dimensions** per set.
- **`Timestamp`** – Required field representing milliseconds since epoch. If omitted, CloudWatch uses log ingestion time, but explicit timestamps prevent clock-skew issues.
- **Helper Libraries** – Language-specific implementations like `aws-embedded-metrics` and Lambda Powertools that manage JSON serialization, dimension cardinality enforcement, and automatic flushing at ≤5-second intervals.

When CloudWatch Logs receives an EMF-formatted event, it parses the `_aws` block, extracts the metric datapoints, and surfaces them in your specified custom namespace. Parsing failures are automatically emitted as `EMFParsingErrors` and `EMFValidationErrors` metrics in the `AWS/Logs` namespace, enabling immediate troubleshooting visibility.

## Architectural Flow

The metric publishing process follows this asynchronous pipeline:

1. **Application code** invokes a library method such as `metrics.put_metric()` or `metrics.add_metric()`.
2. The library constructs the EMF JSON, injects the required `_aws` metadata block, and writes the structured line to stdout or a configured logger.
3. The CloudWatch Logs agent (or Lambda runtime) forwards the log line to the designated **Log Group**.
4. CloudWatch extracts the `_aws` payload and stores the metric datapoints in your **custom metric namespace**.
5. CloudWatch Alarms, dashboards, and Metric Insights query the published metrics using standard CloudWatch APIs.

## Implementation Examples

The Agent Toolkit recommends specific libraries for Node.js and Python that handle the EMF JSON structure, enforce dimensional limits, and manage flush intervals automatically.

### Node.js with aws-embedded-metrics

The `aws-embedded-metrics` library manages the `_aws` block construction and enforces the 100-metric limit per batch. In Lambda environments, flushing occurs automatically when the function exits; container workloads may call `metrics.flush()` explicitly.

```javascript
// Install: npm install aws-embedded-metrics
const { metricScope, Unit } = require('aws-embedded-metrics');

exports.handler = metricScope(metrics => async (event) => {
  // Set low-cardinality dimensions (max 30 per set)
  metrics.setDimensions({ ServiceName: 'OrderService', Environment: 'Prod' });

  // Record custom metrics
  metrics.putMetric('Latency', 120, Unit.Milliseconds);
  metrics.putMetric('RequestCount', 1, Unit.Count);

  console.log('Order processed successfully');
});

```

### Python with Lambda Powertools

Lambda Powertools automatically includes the required `Timestamp` field and emits the EMF payload when the Lambda function returns. This implementation is documented in the Toolkit's metrics reference alongside the Node.js approach.

```python

# pip install aws-lambda-powertools

from aws_lambda_powertools import Metrics
from aws_lambda_powertools.metrics import MetricUnit

metrics = Metrics(namespace="MyService", service="order")

def lambda_handler(event, context):
    # Configure dimensions (respects 30 dimension limit)

    metrics.set_dimensions({"Environment": "Prod"})
    
    # Add metrics (enforces 100 metric limit per event)

    metrics.add_metric(name="Latency", unit=MetricUnit.Milliseconds, value=85)
    metrics.add_metric(name="RequestCount", unit=MetricUnit.Count, value=1)
    
    # Automatic EMF emission on function exit

    return {"statusCode": 200}

```

## EMF Limits and Constraints

The reference file [`skills/core-skills/aws-observability/references/metrics.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/references/metrics.md) specifies strict boundaries that prevent CloudWatch API throttling and ensure consistent parsing:

- **Maximum 100 metrics** per `MetricDirective` and **30 dimensions** per `DimensionSet`.
- **Dimension key** length ≤ 250 characters; **value** length ≤ 1,024 characters.
- **Flush interval** must remain ≤ 5 seconds; longer intervals risk alarm evaluation delays.
- **Cost structure** incurs only log ingestion fees, with zero per-metric API charges.

## Troubleshooting EMF Failures

When EMF events contain malformed JSON or exceed dimensional limits, CloudWatch Logs emits diagnostic metrics rather than failing silently. Monitor the `AWS/Logs` namespace for `EMFParsingErrors` (invalid JSON structure) and `EMFValidationErrors` (schema violations such as missing namespaces or invalid dimension types).

For detailed debugging guidance, consult [`skills/core-skills/aws-observability/references/troubleshooting.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/references/troubleshooting.md) in the Agent Toolkit repository. Containerized workloads utilizing the AWS Distro for OpenTelemetry (ADOT) can reference the collector configuration at [`plugins/aws-core/skills/aws-observability/assets/otel-config.yaml`](https://github.com/aws/agent-toolkit-for-aws/blob/main/plugins/aws-core/skills/aws-observability/assets/otel-config.yaml) for EMF forwarding setup.

## Summary

- **AWS Embedded Metrics Format (EMF)** publishes custom CloudWatch metrics by embedding them in structured log events, enabling asynchronous, zero-latency metric emission.
- Each EMF log event supports up to **100 metrics** and **30 dimensions**, with a required `_aws` metadata block containing the `CloudWatchMetrics` directive.
- **Helper libraries** like `aws-embedded-metrics` and Lambda Powertools automatically handle JSON serialization, dimensional constraints, and the ≤5-second flush interval.
- **Parsing failures** surface as `EMFParsingErrors` and `EMFValidationErrors` in the `AWS/Logs` namespace, providing immediate visibility into configuration issues.
- **Cost optimization** is achieved by eliminating `PutMetricData` API calls, paying only for CloudWatch Logs ingestion.

## Frequently Asked Questions

### What is the maximum number of metrics per EMF log event?

Each EMF log event supports a maximum of **100 metrics** per `MetricDirective`, as defined in the Agent Toolkit's reference documentation. Additionally, you may define up to **30 dimensions** per `DimensionSet`. Exceeding these limits results in `EMFValidationErrors` and rejected metric ingestion.

### How does EMF differ from the CloudWatch PutMetricData API?

EMF embeds metric data within log events written to CloudWatch Logs, while `PutMetricData` requires synchronous HTTP API calls. According to the Toolkit's metrics reference, EMF is the default recommendation for Lambda and container workloads because it operates asynchronously, co-locates metrics with logs for correlation, and incurs only log ingestion costs rather than per-metric API charges.

### Why are my EMF metrics not appearing in CloudWatch?

Missing metrics typically indicate parsing failures or validation errors. Check the `AWS/Logs` namespace for `EMFParsingErrors` (malformed JSON) or `EMFValidationErrors` (schema violations like missing namespaces or oversized dimension values). Ensure your log events include the required `_aws` metadata block with a valid `Timestamp` field in milliseconds since epoch.

### What is the recommended flush interval for EMF metrics?

The Agent Toolkit specifies a **maximum flush interval of 5 seconds**. Exceeding this interval risks missing data points in CloudWatch Alarms, as alarm evaluation windows may complete before metrics arrive. Language-specific libraries like `aws-embedded-metrics` and Lambda Powertools automatically enforce this constraint.