How to Use Pre‑Read Rows for Header Detection in jsoncsv

jsoncsv determines CSV/XLS column headers by scanning the first N JSON objects in the input stream, controlled via the -r / --row CLI option or the read_row argument in the Python API.

The alingse/jsoncsv library converts JSON streams to tabular formats. When working with large files or inconsistent schemas, you can limit header detection to a representative sample of initial records rather than scanning the entire dataset.

How Header Detection Works

According to the jsoncsv source code, header detection is implemented in the ReadHeadersMixin class within jsoncsv/dumptool.py. The load_headers method (lines 40-64) accepts a read_row parameter that controls how many JSON objects are inspected before writing the output header.

When processing begins, the mixin reads up to read_row lines from the input stream. Each JSON object is parsed, and its keys are added to a Python set to collect unique field names. After the pre-read loop completes, the set is sorted to produce a deterministic column order.

Using the CLI with the -r Flag

The mkexcel command defined in jsoncsv/main.py (lines 65-66) exposes the -r / --row option to control pre-read behavior from the command line.

Converting JSON to CSV

To generate a CSV while inspecting only the first 3 JSON objects for headers:

jsoncsv mkexcel -r 3 input.json output.csv

The -r 3 argument tells mkexcel to pass read_row=3 to dump_excel, which forwards it to the DumpCSV class via ReadHeadersMixin.load_headers.

Exporting to Excel Format

For Excel output, combine the -t xls flag with -r:

jsoncsv mkexcel -t xls -r 5 input.json output.xls

This selects the DumpXLS dumper class while limiting header detection to the first 5 rows.

Sorting Detected Headers

Combine row limits with the --sort (-s) flag to alphabetically sort the detected headers:

jsoncsv mkexcel -r 4 -s input.json output.csv

Using the Python API

When using jsoncsv programmatically, pass the read_row argument directly to dump_excel in jsoncsv/dumptool.py:

from jsoncsv.dumptool import dump_excel, DumpCSV

with open("input.json", "r", encoding="utf-8") as fin, \
     open("output.csv", "w", encoding="utf-8", newline="") as fout:
    # Inspect first 10 objects to determine headers

    dump_excel(fin, fout, DumpCSV, read_row=10, sort_type=False)

The dump_excel helper (lines 51-61) validates the dumper class and initiates the process. The read_row value propagates through DumpExcel.prepare (lines 73-76) to ReadHeadersMixin.load_headers, which executes the pre-read loop.

Key Implementation Files

• jsoncsv/main.py (lines 65-66): Defines the -r/--row CLI argument and passes it to the dump functions.
• jsoncsv/dumptool.py (lines 40-64): Contains ReadHeadersMixin.load_headers, which implements the pre-read logic, header collection via set operations, and sorting.
• jsoncsv/dumptool.py (lines 73-76): The DumpExcel.prepare method orchestrates header loading by calling load_headers with the supplied read_row value.

Summary

• Use the -r N or --row N flag with mkexcel to limit header detection to the first N JSON objects.
• In Python code, pass read_row=N to dump_excel() to achieve the same behavior.
• The ReadHeadersMixin.load_headers method in jsoncsv/dumptool.py implements the logic, collecting keys into a set and sorting them deterministically.
• This feature is essential for large files or when later records contain fields that should not appear as columns.

Frequently Asked Questions

What happens if I omit the -r flag?

If you do not specify -r or read_row, jsoncsv reads the entire input file to collect all possible header names. This ensures complete schema coverage but may impact performance on large datasets.

Can I use pre-read rows with Excel (XLS) output?

Yes. The -r flag works with both CSV and XLS formats. When using -t xls, the DumpXLS class receives the read_row parameter and uses the same ReadHeadersMixin logic as CSV exports.

How does jsoncsv handle headers that appear in later rows but not in the pre-read set?

Fields that appear only after the pre-read limit are ignored in the output. Only keys found within the first N JSON objects (where N equals the read_row value) are included as columns in the final CSV or Excel file.

Is the header order deterministic when using pre-read rows?

Yes. The load_headers method sorts the collected keys before writing the header row, ensuring consistent column ordering across runs regardless of the order in which keys appeared in the pre-read objects.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}