# Publishing Custom CloudWatch Metrics Using EMF: A Complete Implementation Guide

> Learn to publish custom CloudWatch metrics using EMF. This guide shows how to write structured JSON to standard output for automatic extraction and processing by CloudWatch Logs without PutMetricData calls.

- Repository: [Amazon Web Services/agent-toolkit-for-aws](https://github.com/aws/agent-toolkit-for-aws)
- Tags: how-to-guide
- Published: 2026-06-29

---

**The Embedded Metric Format (EMF) enables asynchronous publication of custom CloudWatch metrics by writing a structured JSON log line to standard output, which CloudWatch Logs automatically extracts and processes without requiring synchronous `PutMetricData` API calls.**

Publishing custom CloudWatch metrics using EMF is the recommended pattern for Lambda functions and containerized workloads documented in the `aws/agent-toolkit-for-aws` repository. According to [`skills/core-skills/aws-observability/SKILL.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/SKILL.md) and the detailed specification in [`skills/core-skills/aws-observability/references/metrics.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/references/metrics.md), this format decouples metric emission from ingestion by embedding definitions within log events, reducing latency and API costs compared to traditional methods.

## How EMF Works

The EMF workflow transforms structured log output into queryable CloudWatch metrics through three distinct phases validated by the CloudWatch Logs service.

### Log Event Payload Structure

Your application writes a JSON object containing an `_aws` section to **standard output**. This section must include a `CloudWatchMetrics` array defining the **namespace**, **metric names**, **dimensions**, and **units**, alongside a `Timestamp` field expressed in **milliseconds** since the epoch.

### CloudWatch Logs Agent Processing

The CloudWatch Logs service parses each JSON payload and validates it against the EMF schema. As implemented in the toolkit's observability module, validation failures generate the `EMFValidationErrors` and `EMFParsingErrors` metrics in the `AWS/Logs` namespace, enabling proactive monitoring of malformed submissions through standard CloudWatch alarms.

### Flushing Requirements

In Lambda environments, libraries automatically flush the EMF payload after every invocation within **5 seconds**. For containerized workloads running on ECS or EKS, you must configure the log driver to flush at least every 5 seconds to ensure reliable alarm evaluation and prevent data gaps during metric aggregation.

## EMF Constraints and Limits

The `aws/agent-toolkit-for-aws` documentation enforces strict validation rules defined in [`skills/core-skills/aws-observability/references/metrics.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/references/metrics.md). These constraints prevent ingestion failures and ensure optimal query performance:

- **Namespace**: Must be a valid CloudWatch namespace string (e.g., `MyApp`).
- **Metric Name**: Maximum 255 characters, supporting alphanumeric characters and underscores only.
- **Dimensions**: Maximum 10 dimensions per metric; dimension keys limited to 250 characters and values to 1024 characters.
- **Timestamp**: Required field in milliseconds since epoch; if omitted, CloudWatch falls back to ingestion time, potentially causing clock-skew issues that affect alarm accuracy.
- **Payload Size**: Maximum 256 KB per log event.
- **Flush Interval**: Must not exceed 5 seconds for reliable alarm evaluation.

## Implementation Examples

The repository provides production-ready patterns for emitting EMF-compliant logs across multiple runtimes.

### Node.js with Lambda Powertools

The Lambda Powertools library constructs the `_aws` block automatically, manages timestamp generation, and handles dimension serialization:

```javascript
import { Metrics, MetricUnits } from '@aws-lambda-powertools/metrics';

const metrics = new Metrics({ namespace: 'MyApp', serviceName: 'OrderService' });

export const handler = async (event) => {
  metrics.addMetric('OrderCount', MetricUnits.Count, 1);
  metrics.addDimension('Region', 'us-east-1');
  
  await metrics.flush();
  
  return { statusCode: 200 };
};

```

### Python with aws-embedded-metrics

For Python workloads running in Lambda or containers, the decorator pattern handles EMF serialization and automatic flushing:

```python
from aws_embedded_metrics import metric_scope, MetricUnit

@metric_scope
def handler(event, context, metrics):
    metrics.set_namespace("MyApp")
    metrics.put_dimensions({"Region": "us-east-1"})
    metrics.put_metric("OrderCount", 1, MetricUnit.Count)
    
    return {"statusCode": 200}

```

### Manual EMF JSON Construction

For languages without native libraries, construct the payload manually following the schema defined in the toolkit's reference documentation:

```json
{
  "_aws": {
    "Timestamp": 1729974000000,
    "CloudWatchMetrics": [
      {
        "Namespace": "MyApp",
        "Dimensions": [["Region"]],
        "Metrics": [
          {"Name": "OrderCount", "Unit": "Count"}
        ]
      }
    ]
  },
  "OrderCount": 1,
  "Region": "us-east-1"
}

```

Write this JSON string to `stdout` or the container's log driver to trigger automatic metric extraction.

## EMF vs PutMetricData

The observability skill documentation recommends distinct patterns based on workload architecture:

- **EMF**: Preferred for Lambda functions and long-running containerized services due to lower overhead, automatic aggregation, and asynchronous processing.
- **PutMetricData**: Reserved for batch jobs requiring synchronous confirmation of metric delivery or explicit API response handling.

## Integration with ADOT Collector

For containerized environments, the repository includes an AWS Distro for OpenTelemetry (ADOT) collector configuration in [`skills/core-skills/aws-observability/assets/otel-config.yaml`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/assets/otel-config.yaml). This configuration forwards EMF-derived metrics from container logs to CloudWatch, enabling unified observability across hybrid serverless and container architectures.

## Summary

- Publishing custom CloudWatch metrics using EMF requires writing a JSON payload with an `_aws` section containing the `CloudWatchMetrics` array and a millisecond-precision timestamp.
- Hard limits include a 256 KB payload size, maximum 10 dimensions per metric, and mandatory 5-second flush intervals for reliable alerting.
- Use Lambda Powertools for Node.js or `aws-embedded-metrics` for Python to automate payload construction and enforce validation rules.
- Monitor `EMFValidationErrors` and `EMFParsingErrors` in the `AWS/Logs` namespace to detect malformed submissions and constraint violations.
- Reference [`skills/core-skills/aws-observability/references/metrics.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/references/metrics.md) for the complete specification and constraint matrix.
- Containerized workloads can leverage the ADOT configuration in [`skills/core-skills/aws-observability/assets/otel-config.yaml`](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/assets/otel-config.yaml) for centralized metric forwarding.

## Frequently Asked Questions

### What is the maximum payload size for EMF log events?

EMF supports a maximum payload size of **256 KB** per log event. Exceeding this limit results in validation failures tracked by the `EMFValidationErrors` metric in the `AWS/Logs` namespace.

### How does EMF handle timestamp precision?

EMF requires the `Timestamp` field in the `_aws` section to be expressed in **milliseconds since the epoch**. If omitted, CloudWatch defaults to the log ingestion time, which may introduce clock-skew issues and affect alarm accuracy.

### When should I use PutMetricData instead of EMF?

Use `PutMetricData` for batch processing workloads that require synchronous confirmation of metric delivery or when you need explicit API error handling. EMF is optimized for high-throughput, asynchronous scenarios like Lambda invocations and containerized microservices where minimizing latency is critical.

### How can I detect malformed EMF submissions?

CloudWatch Logs automatically emits `EMFValidationErrors` and `EMFParsingErrors` metrics to the `AWS/Logs` namespace when it encounters invalid EMF payloads. Configure alarms on these metrics to identify formatting errors, constraint violations, or schema mismatches in your application logs.