# Architecture of the jsoncsv Python Tool: A 4-Layer Deep Dive > Explore the modular four-layer architecture of the jsoncsv Python tool. Understand its CLI dispatcher, JSON processor, dump engine, and utility layer for efficient data conversion. - Repository: [alingse/jsoncsv](https://github.com/alingse/jsoncsv) - Tags: architecture - Published: 2026-02-24 --- **The jsoncsv Python tool follows a modular four-layer architecture comprising a CLI dispatcher, a core JSON processor for flattening and restoring nested data, a pluggable dump engine for CSV/XLS serialization, and a utility layer for type safety and key encoding.** The jsoncsv package is a lightweight open-source command-line utility that transforms nested JSON into flat CSV or Excel tables and vice versa. As implemented in the `alingse/jsoncsv` repository, its architecture demonstrates clean separation of concerns across four distinct logical layers. Understanding this structure helps developers extend the tool with new output formats or integrate its functionality into data processing pipelines. ## CLI Layer: Command Dispatch ([`jsoncsv/main.py`](https://github.com/alingse/jsoncsv/blob/main/jsoncsv/main.py)) The entry point defines two Click commands that parse arguments and dispatch work to the appropriate internal functions. ### The `jsoncsv` Command This command handles JSON expansion and restoration. It validates options like `--expand`, `--restore`, `--safe`, and `--array`, then calls `jsontool.convert_json` with the selected processing function. The command explicitly closes input and output streams to ensure deterministic I/O behavior. ### The `mkexcel` Command This command creates CSV or XLS files from JSON streams. It selects the appropriate dumper class—either `DumpCSV` or `DumpXLS`—and invokes `dumptool.dump_excel` to handle the serialization. Both commands reside in [`jsoncsv/main.py`](https://github.com/alingse/jsoncsv/blob/main/jsoncsv/main.py) and serve as thin wrappers around the core processing logic. ## Core JSON Processor: Flattening Logic ([`jsoncsv/jsontool.py`](https://github.com/alingse/jsoncsv/blob/main/jsoncsv/jsontool.py)) The [`jsoncsv/jsontool.py`](https://github.com/alingse/jsoncsv/blob/main/jsoncsv/jsontool.py) module contains the tree-walking algorithms that convert nested JSON into flat key-value maps and back again. ### Key Processing Functions **`gen_leaf`** recursively traverses JSON objects and yields `(path, value)` tuples for every leaf node. **`is_array_index`** detects whether a collection of path segments represents a list by checking if all indices are numeric. **`expand`** flattens a JSON object into a dictionary where keys are dot-separated paths (e.g., `user.address.zip`). **`restore`** performs the inverse operation, reconstructing nested structures from flat representations using **`from_leaf`**, which reassembles objects from path tuples. **`convert_json`** streams JSON lines from an input file, applies either `expand` or `restore` to each object, and writes the results to an output file. It handles the `--array` flag for processing entire JSON arrays instead of line-delimited objects. ## Dump Engine: CSV and XLS Output ([`jsoncsv/dumptool.py`](https://github.com/alingse/jsoncsv/blob/main/jsoncsv/dumptool.py)) The dump subsystem in [`jsoncsv/dumptool.py`](https://github.com/alingse/jsoncsv/blob/main/jsoncsv/dumptool.py) uses an abstract-base-class pattern to support multiple output formats through a common interface. ### Class Hierarchy **`Dump`** serves as the abstract skeleton defining `initialize`, `prepare`, `dump_file`, and `on_finish` methods. **`ReadHeadersMixin`** provides static utility methods that scan the first N rows to collect all possible keys and optionally sort them. **`DumpExcel`** combines the abstract `Dump` class with `ReadHeadersMixin`, storing headers and data before writing. Concrete implementations **`DumpCSV`** and **`DumpXLS`** inherit from this hierarchy. `DumpCSV` writes data using Python’s `csv.DictWriter`, while `DumpXLS` creates spreadsheets via `xlwt`. The **`dump_excel`** function validates the selected dumper class and executes the generic workflow: instantiate the dumper, prepare headers, and stream rows to the output file. ## Utility Layer: Safe Key Handling ([`jsoncsv/utils.py`](https://github.com/alingse/jsoncsv/blob/main/jsoncsv/utils.py)) The [`jsoncsv/utils.py`](https://github.com/alingse/jsoncsv/blob/main/jsoncsv/utils.py) module provides shared infrastructure for type safety and key encoding. **Type aliases** like `JsonType`, `PathType`, and `LeafType` enable static typing throughout the codebase. When users enable `--safe` mode to handle keys containing the separator character, the module's **`encode_safe_key`** and **`decode_safe_key`** functions escape dots with backslashes (e.g., `key\.name`). This prevents key collisions when flattening JSON objects where field names themselves contain dot characters. ## Practical Code Examples ### Expanding Nested JSON to Flat Structure ```bash python -m jsoncsv jsoncsv data.json expanded.json ``` This processes [`data.json`](https://github.com/alingse/jsoncsv/blob/main/data.json) line-by-line, converting nested objects into flat dictionaries with dot-separated keys like `address.street`. ### Restoring Flat JSON to Nested Structure ```bash python -m jsoncsv jsoncsv --restore expanded.json restored.json ``` This reverses the flattening operation, reconstructing the original hierarchical JSON using the `restore` function. ### Using Safe Mode for Keys with Separators ```bash python -m jsoncsv jsoncsv --safe data.json safe_expanded.json ``` Keys containing `.` are escaped as `\.` using `encode_safe_key`, preventing ambiguity during the flattening process. ### Converting JSON to CSV ```bash python -m jsoncsv mkexcel data.json table.csv ``` Behind the scenes, `DumpCSV` reads the first lines to infer headers via `ReadHeadersMixin`, then streams each object as a CSV row. ### Creating Excel Files with Sorted Headers ```bash python -m jsoncsv mkexcel -t xls --sort data.json table.xls ``` `DumpXLS` creates an `xlwt.Workbook`, writes sorted headers in the first row, and fills subsequent rows with data. ## Summary The architecture of the jsoncsv Python tool separates concerns into four distinct layers: - **CLI Layer** ([`jsoncsv/main.py`](https://github.com/alingse/jsoncsv/blob/main/jsoncsv/main.py)): Parses command-line arguments via Click and dispatches to core functions - **Core Processor** ([`jsoncsv/jsontool.py`](https://github.com/alingse/jsoncsv/blob/main/jsoncsv/jsontool.py)): Implements tree-walking algorithms (`gen_leaf`, `expand`, `restore`) for JSON flattening and reconstruction - **Dump Engine** ([`jsoncsv/dumptool.py`](https://github.com/alingse/jsoncsv/blob/main/jsoncsv/dumptool.py)): Provides an extensible framework with abstract `Dump` classes and concrete `DumpCSV`/`DumpXLS` implementations - **Utility Layer** ([`jsoncsv/utils.py`](https://github.com/alingse/jsoncsv/blob/main/jsoncsv/utils.py)): Supplies type definitions and safe-key encoding to handle edge cases in key names This modular design allows developers to add new output formats by extending the `Dump` class hierarchy, while keeping the command-line interface lightweight and predictable. ## Frequently Asked Questions ### How does jsoncsv handle deeply nested JSON objects? The `gen_leaf` function in [`jsoncsv/jsontool.py`](https://github.com/alingse/jsoncsv/blob/main/jsoncsv/jsontool.py) recursively traverses the JSON tree, yielding `(path, value)` tuples for every leaf node regardless of nesting depth. The `expand` function accumulates these paths into dot-separated keys, preserving the full hierarchical context in the flat output. ### What is the difference between the `expand` and `restore` functions in jsoncsv? **`expand`** flattens a nested JSON object into a dictionary with dot-separated keys (e.g., `{"user": {"name": "John"}}` becomes `{"user.name": "John"}`). **`restore`** performs the inverse operation, using `from_leaf` to rebuild nested structures from flat key-value pairs by splitting paths and reconstructing objects and arrays. ### How can I add a new output format to jsoncsv? Create a new class inheriting from `Dump` (or `DumpExcel` if you need header reading) in [`jsoncsv/dumptool.py`](https://github.com/alingse/jsoncsv/blob/main/jsoncsv/dumptool.py). Implement the `write_headers` and `write_obj` methods, then register your class in the CLI's `mkexcel` command in [`jsoncsv/main.py`](https://github.com/alingse/jsoncsv/blob/main/jsoncsv/main.py). The `dump_excel` function handles the generic workflow, so your implementation only needs to manage format-specific serialization. ### What does the `--safe` mode do in jsoncsv? The `--safe` flag activates key encoding in [`jsoncsv/utils.py`](https://github.com/alingse/jsoncsv/blob/main/jsoncsv/utils.py). When enabled, `encode_safe_key` escapes separator characters (dots) with backslashes, allowing JSON keys that contain dots to be flattened unambiguously. The `decode_safe_key` function reverses this process during restoration, ensuring data integrity for keys like `version.1.0`.