# How to Write Log Insights Queries for CloudWatch Log Analysis: A Complete Guide > Master CloudWatch Logs Insights queries to analyze and visualize your AWS log data. Learn to use fields, filter, parse, stats, sort, and limit clauses for effective log analysis. - Repository: [Amazon Web Services/agent-toolkit-for-aws](https://github.com/aws/agent-toolkit-for-aws) - Tags: how-to-guide - Published: 2026-07-02 --- **Use CloudWatch Logs Insights' pipe-delimited query language—combining `fields`, `filter`, `parse`, `stats`, `sort`, and `limit` clauses—to extract, aggregate, and visualize log data directly from your AWS environment.** The `aws/agent-toolkit-for-aws` repository provides a definitive reference for building these queries, documenting every command and pattern you need to analyze logs efficiently. This guide walks through the core syntax, practical examples, and troubleshooting techniques found in the toolkit's observability skills. ## Understanding the Core Query Syntax CloudWatch Logs Insights uses a specialized SQL-like syntax that processes log data through a pipeline of commands. Each command is separated by a pipe character (`|`), allowing you to transform raw log entries into structured insights. ### Selecting Fields with `fields` The `fields` command defines which attributes appear in your output. You can reference system fields like `@timestamp`, `@message`, and `@logStream`, or extract custom fields from your log format. ```sql fields @timestamp, @message, @logStream | limit 20 ``` As detailed in the [log-insights.md](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/references/log-insights.md) reference, always specify your desired fields first to reduce data transfer and improve query performance. ### Filtering Data with `filter` The `filter` command applies boolean predicates to narrow your result set. You can use comparison operators (`=`, `!=`, `<`, `>`), pattern matching (`like`, `=~`), and logical operators (`and`, `or`). ```sql fields @timestamp, @message | filter @message like /ERROR/ and @timestamp > 1686816000 | sort @timestamp desc ``` ### Parsing Unstructured Data with `parse` When logs contain embedded JSON or custom text formats, the `parse` command extracts structured fields using grok-style patterns or regular expressions. ```sql fields @message | parse @message "* * *" as ip, user, action | filter action = "DELETE" | stats count() by ip ``` This pattern creates new temporary fields (`ip`, `user`, `action`) that can be referenced in subsequent pipeline stages. ### Aggregating with `stats` The `stats` command performs calculations across your dataset, supporting functions like `count()`, `sum()`, `avg()`, `min()`, `max()`, and `percentile()`. Group results using the `by` clause. ```sql fields durationMs, endpoint | filter durationMs > 0 | stats avg(durationMs) as avgLatency, count() as requestCount by endpoint | sort avgLatency desc ``` ## Practical Log Insights Query Examples The following patterns demonstrate how to solve common operational questions using the exact syntax validated in the agent toolkit's reference files. ### 1. List the Latest Error Messages Retrieve the most recent 20 error entries to start incident investigation. ```sql fields @timestamp, @message | filter @message like /ERROR/ | sort @timestamp desc | limit 20 ``` ### 2. Count Occurrences of Each Error Type Extract the error level and message body, then aggregate to find frequency distribution. ```sql fields @message | parse @message "* *" as level, rest | filter level = "ERROR" | stats count() as errorCount by rest | sort errorCount desc ``` ### 3. Calculate Average Latency per API Endpoint Analyze performance by computing average response times grouped by endpoint. ```sql fields @timestamp, endpoint, durationMs | filter durationMs > 0 | stats avg(durationMs) as avgLatency by endpoint | sort avgLatency desc ``` ### 4. Identify Top Traffic Sources by IP Address Parse Apache-style access logs to find the five most active IP addresses. ```sql fields @message | parse @message "* * *" as ip, user, action | stats count() as hits by ip | sort hits desc | limit 5 ``` ### 5. Join Request and Response Logs to Measure Latency Use subqueries to correlate events across different log streams and calculate end-to-end latency. ```sql # Subquery: get request timestamps subquery request_logs | fields @timestamp as reqTime, requestId | filter @message like /REQUEST/ | stats min(@timestamp) as startTime by requestId # Main query: correlate with responses fields @timestamp, requestId, @message | filter @message like /RESPONSE/ | stats max(@timestamp) as endTime by requestId | join request_logs on requestId | stats (endTime - startTime) / 1000 as latencySec by requestId | sort latencySec desc | limit 10 ``` ## Advanced Query Patterns Beyond basic filtering, CloudWatch Logs Insights supports complex analytical operations documented in the toolkit's advanced sections. ### Joining Log Streams with `join` The `join` command merges results from multiple log groups or subqueries based on common fields, enabling correlation analysis across distributed systems. As implemented in the agent toolkit examples, always ensure join keys are indexed or high-cardinality fields to prevent memory exhaustion. ### Pattern Matching and Subqueries Pattern commands (`pattern`) automatically cluster similar log messages, while subqueries allow you to nest queries for multi-stage analysis. These features are detailed in the [SKILL.md](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/SKILL.md) file, which explains how to invoke these patterns within automated workflows. ## Troubleshooting and Optimization When queries return timeout errors or excessive results, consult the [troubleshooting.md](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/references/troubleshooting.md) reference. Key optimization strategies include: - **Time-range scoping**: Always specify the narrowest `@timestamp` range in your `filter` clause to reduce scanned data volume. - **Field pre-filtering**: Use `fields` immediately after the implicit log source to drop unnecessary columns before aggregation. - **Limit early testing**: Add `limit 10` during query development to validate syntax before running full scans. For visualization best practices, the [dashboards.md](https://github.com/aws/agent-toolkit-for-aws/blob/main/skills/core-skills/aws-observability/references/dashboards.md) file explains how to embed these queries into CloudWatch dashboards for continuous monitoring. ## Summary - **CloudWatch Logs Insights** uses a pipe-delimited syntax processed sequentially from `fields` through `limit`. - **Essential commands** include `fields` for projection, `filter` for boolean filtering, `parse` for regex extraction, and `stats` for aggregation. - **Complex analysis** is supported via `join` operations and subqueries to correlate events across log streams. - **Performance optimization** requires strict time-range filtering and early field selection, as documented in the toolkit's troubleshooting guides. - **Source files** [`log-insights.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/log-insights.md), [`SKILL.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/SKILL.md), [`troubleshooting.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/troubleshooting.md), and [`dashboards.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/dashboards.md) in the `aws/agent-toolkit-for-aws` repository provide the authoritative reference for query syntax and integration patterns. ## Frequently Asked Questions ### What is the maximum execution time for a CloudWatch Logs Insights query? CloudWatch Logs Insights queries time out after **15 minutes** of execution. For large datasets, optimize by narrowing the time range in your `filter` clause or using sampled queries. The troubleshooting guide in the agent toolkit recommends testing with `limit` clauses first to validate performance before full scans. ### How does the `parse` command differ from the `filter` command? The **`parse`** command extracts new fields from existing log text using pattern matching, creating structure from unstructured data. The **`filter`** command removes rows from the result set based on boolean conditions. According to the [`log-insights.md`](https://github.com/aws/agent-toolkit-for-aws/blob/main/log-insights.md) reference, `parse` should precede `filter` when you need to filter based on extracted values. ### Can I query multiple log groups in a single CloudWatch Logs Insights query? Yes, you can select multiple log groups in the CloudWatch console or API, but each query executes against the union of those groups. For cross-log-group correlation, use the **`join`** command with subqueries, as demonstrated in the agent toolkit's latency analysis example, which matches `requestId` values across separate request and response logs. ### Why does my query show "No results found" when I can see matching logs in the console? This typically occurs when the **time range** selected in the query window does not overlap with the `@timestamp` values in your filter, or when field names are case-sensitive. The troubleshooting documentation advises verifying that your `filter` clause uses exact field names and that the query time range includes the log events.