# How to Process JSON at the Command Line with jq > Learn to process JSON at the command line with jq. Discover how this powerful tool integrates with Unix-style I/O for reliable filtering and piping without temp files. - Repository: [Joshua Levy/the-art-of-command-line](https://github.com/jlevy/the-art-of-command-line) - Tags: how-to-guide - Published: 2026-02-24 --- **TLDR:** The `jlevy/the-art-of-command-line` repository designates **jq** as the definitive tool for processing JSON at the command line, utilizing its canonicalization features and Unix-style I/O to enable reliable diffing, filtering, and pipeline integration without temporary files. The `jlevy/the-art-of-command-line` repository treats JSON as a first-class data format for shell workflows, explicitly recommending **jq** for all non-interactive JSON processing. According to the guide's "Processing files and data" section in [`README.md`](https://github.com/jlevy/the-art-of-command-line/blob/main/README.md) [2†L227-L228], jq embodies the Unix philosophy: it is a small, portable, composable tool that reads from *stdin* and writes to *stdout*, allowing seamless chaining with `grep`, `awk`, `sed`, `diff`, and `colordiff`. ## Canonicalization and Diffing JSON Files A core strength of jq is its ability to produce **canonical JSON**—output with deterministic key ordering and normalized whitespace—making it ideal for version control and diffing. In [`README.md`](https://github.com/jlevy/the-art-of-command-line/blob/main/README.md) lines 367-370 [3†L367-L370], the guide demonstrates how to compare two JSON files without creating temporary files using **process substitution** and the `--sort-keys` flag. To normalize JSON for reliable comparison: ```bash jq --sort-keys . data.json > canonical.json ``` This command guarantees reproducible object key order, removing formatting variations that would otherwise cause false positives in diffs. For direct file comparison, combine process substitution with `diff`: ```bash diff <(jq --sort-keys . file1.json) <(jq --sort-keys . file2.json) | colordiff | less -R ``` Here, `<( )` feeds the canonicalized output directly into `diff` as file descriptors, while `colordiff` adds color highlighting and `less -R` preserves the color codes in the pager. ## Essential jq Operations for Data Extraction jq uses a path-based query language to extract and transform JSON structures. Below are the fundamental patterns endorsed by the repository for daily command-line work. ### Pretty-Printing and Formatting The simplest jq operation formats compact or minified JSON into human-readable indentation: ```bash jq . data.json | less -R ``` The identity filter `.` reformats the input with proper indentation. Piping to `less -R` preserves color codes if jq is invoked with `--color-output` in your shell alias. ### Extracting Specific Fields To stream a single field from every object in an array, use the iterator operator `.[]` combined with property access: ```bash jq '.[] | .id' data.json ``` This outputs each `id` value on a new line, suitable for piping into `xargs` or other line-oriented tools. ### Filtering Objects by Condition The `select()` function filters arrays based on boolean expressions: ```bash jq '.[] | select(.status == "active")' data.json ``` Only objects where the `status` field equals `"active"` are emitted to *stdout*. ### Counting and Aggregation To aggregate data before output, wrap expressions in array constructors `[]` and apply operators like `length`: ```bash jq '[.[] | .id] | length' data.json ``` This constructs a temporary array of all IDs, then returns the count, effectively giving you the number of items without external tools like `wc`. ## Integrating jq with Shell Pipelines Because jq outputs valid JSON (or raw text with the `-r` flag), downstream tools that understand JSON—including Python one-liners, language-specific libraries, or subsequent jq invocations—can immediately consume the results. The repository highlights this composability by showing jq integrated with `curl` and `xargs`: ```bash curl -s https://api.github.com/repos/jlevy/the-art-of-command-line/releases | jq -r '.[0].tag_name' | xargs -I{} echo "Latest tag: {}" ``` This pipeline fetches API data, extracts the latest release tag as a raw string (`-r` removes JSON quotes), and passes it to `xargs` for formatted output. This pattern exemplifies how jq serves as the bridge between web APIs and standard Unix text processing tools. ## Interactive Alternatives to jq While jq excels in scripts and non-interactive pipelines, the repository notes that exploratory work benefits from interactive tools. For manual data exploration, the guide references **jid** and **jiq** in lines 27-28 [2†L27-L28] of the README. These utilities provide terminal-based interfaces for real-time JSON querying, though jq remains the go-to utility for production scripts and automated workflows. ## Summary - The `jlevy/the-art-of-command-line` repository explicitly recommends jq in [`README.md`](https://github.com/jlevy/the-art-of-command-line/blob/main/README.md) [2†L227-L228] as the standard command-line JSON processor. - **Canonicalization** via `jq --sort-keys .` enables deterministic diffing and version control by normalizing key order and whitespace. - **Process substitution** (`<( )`) allows `diff` to compare JSON files directly without temporary intermediate files, as shown in [`README.md`](https://github.com/jlevy/the-art-of-command-line/blob/main/README.md) [3†L367-L370]. - jq queries combine iterators (`.[]`), filters (`select()`), and aggregators (`length`) to transform JSON without external dependencies. - Output from jq integrates with standard Unix tools like `curl`, `xargs`, `grep`, and `colordiff`, maintaining the composability of shell pipelines. - For interactive exploration, **jid** and **jiq** provide alternatives, but jq remains the preferred tool for automated processing. ## Frequently Asked Questions ### What makes jq the preferred tool for processing JSON at the command line? jq is a lightweight, portable processor that adheres to the Unix philosophy of reading from *stdin* and writing to *stdout*, allowing it to chain with standard shell tools like `grep`, `awk`, and `diff`. According to the `jlevy/the-art-of-command-line` source code, it is the recommended utility for all non-interactive JSON work because it outputs valid JSON that any downstream consumer can parse. ### How do I reliably diff two JSON files that have different formatting or key ordering? Use jq's `--sort-keys` flag to canonicalize both files, ensuring object keys appear in alphabetical order and whitespace is normalized. Then use process substitution to feed both outputs directly to `diff`: `diff <(jq --sort-keys . file1.json) <(jq --sort-keys . file2.json)`. This technique, documented in [`README.md`](https://github.com/jlevy/the-art-of-command-line/blob/main/README.md) [3†L367-L370], eliminates false positives caused by formatting differences. ### Can jq handle streaming or very large JSON files? Yes. jq processes input as a stream and can operate on newline-delimited JSON (NDJSON) or large arrays using the iterator operator `.[]`, which emits one object at a time rather than loading the entire structure into memory. This streaming capability makes it suitable for processing logs or API responses that would overwhelm tools that require complete memory allocation. ### What are the interactive alternatives to jq for exploring JSON data? The repository mentions **jid** (JSON incremental digger) and **jiq** as interactive alternatives for exploratory work where you need real-time feedback on queries [2†L27-L28]. These tools provide terminal-based interfaces for constructing jq-like queries interactively, though for scripts and automated pipelines, the standard jq utility remains the authoritative tool.