JSON vs CSV Export Formats in Twinkle Eval: Key Differences and Use Cases

JSONExporter preserves the complete nested hierarchy of evaluation results with full environment metadata, while CSVExporter flattens the data into tabular rows optimized for spreadsheet analysis.

The ai-twinkle/eval repository provides a flexible results export framework that supports multiple output formats through a common interface. Understanding the differences between JSON and CSV export formats in Twinkle Eval helps you choose the right format for archival storage, API exchange, or quick data analysis in Excel.

Architectural Overview

Both exporters inherit from the abstract ResultsExporter base class defined in twinkle_eval/results_exporters.py. The factory pattern in ResultsExporterFactory instantiates the appropriate exporter based on the requested format, ensuring consistent interface behavior while allowing format-specific implementations.

Key Differences Between JSON and CSV Exporters

Data Structure and Hierarchy

JSONExporter writes the complete nested results dictionary, preserving the hierarchy of datasets, files, and detailed metrics. This structure mirrors the internal evaluation data model exactly, making it ideal for programmatic consumption.

CSVExporter calls _flatten_results() to transform the nested dictionary into a flat table where each row represents a single file result. This denormalization makes the data spreadsheet-friendly but loses the original nesting structure.

Environment Metadata Handling

In twinkle_eval/results_exporters.py, the JSONExporter guarantees an environment block through _enhance_with_environment(), inserting default "N/A" placeholders for missing data to ensure schema completeness (lines 38-44).

The CSVExporter embeds a subset of environment fields—such as GPU model, TP/PP size, and framework version—directly into each row via _flatten_results() (lines 101-117). This repetition enables filtering by hardware configuration in Excel but increases file size.

File Extensions and Output Formatting

As implemented in the source code:

• JSONExporter.get_file_extension() returns .json (lines 30-32)
• CSVExporter.get_file_extension() returns .csv (lines 78-80)

The JSON output uses json.dump(..., indent=4) for pretty-printing with 4-space indentation, making it human-readable while maintaining machine parseability. The CSV output uses Python's csv.DictWriter with a standard header row, optimized for data analysis tools.

Handling of Missing Fields

JSONExporter ensures data completeness by inserting "N/A" for missing environment values, preventing schema validation errors in downstream consumers.

CSVExporter leaves cells empty when fields are absent, relying on the first flattened record to define the column schema. This can result in sparse columns when certain metrics are optional.

Practical Code Examples

Exporting to JSON Format

from twinkle_eval.results_exporters import ResultsExporterFactory

# results is the dictionary returned by an evaluation run

results = evaluator.run_evaluation()

output_path = "eval_output"

# Export to JSON only

json_files = ResultsExporterFactory.export_results(
    results,
    output_path,
    formats=["json"]
)
print(f"JSON written to: {json_files[0]}")

This creates eval_output.json containing the full nested structure with the guaranteed environment block.

Exporting to CSV Format

from twinkle_eval.results_exporters import ResultsExporterFactory

output_path = "eval_output"

# Export to CSV only

csv_files = ResultsExporterFactory.export_results(
    results,
    output_path,
    formats=["csv"]
)
print(f"CSV written to: {csv_files[0]}")

This generates eval_output.csv with one row per evaluated file, including flattened environment metadata columns.

Direct Exporter Usage

For advanced customization, instantiate exporters directly:

from twinkle_eval.results_exporters import JSONExporter, CSVExporter

# JSON export with custom filename

json_exporter = JSONExporter()
json_path = json_exporter.export(results, "full_results.json")

# CSV export with custom filename  

csv_exporter = CSVExporter()
csv_path = csv_exporter.export(results, "summary_results.csv")

When to Use Each Format

Choose JSON when you need:

• Complete data preservation for archival storage
• Hierarchical data access for custom reporting tools
• API integration requiring full environment metadata
• Input for the HTML report generator

Choose CSV when you need:

• Quick analysis in Excel, Google Sheets, or LibreOffice
• Statistical processing in R, pandas, or SQL databases
• Sharing results with non-technical stakeholders
• Filtering by specific hardware configurations (GPU, TP/PP size)

Summary

• JSONExporter in twinkle_eval/results_exporters.py preserves the complete nested hierarchy of evaluation results and guarantees an environment metadata block via _enhance_with_environment().
• CSVExporter flattens the nested data structure into tabular rows using _flatten_results(), embedding environment fields in each row for spreadsheet analysis.
• JSON uses .json extension with pretty-printed 4-space indentation; CSV uses .csv with standard csv.DictWriter output.
• JSON is ideal for archival storage and API exchange; CSV is optimized for data analysis and business user consumption.

Frequently Asked Questions

Can I export to both JSON and CSV simultaneously?

Yes. Pass both formats to the factory method: formats=["json", "csv"]. The ResultsExporterFactory.export_results() method in twinkle_eval/results_exporters.py iterates through the requested formats and returns a list of output file paths for each format generated.

Does the CSV format include all nested metrics from the evaluation?

No. The CSV format only includes fields that are explicitly flattened by the _flatten_results() method. While it captures per-file metrics and key environment fields (GPU type, tensor parallelism size, framework version), deeply nested structures or custom metadata may be omitted unless the flattening logic is extended.

How does JSONExporter handle missing environment data?

The JSONExporter calls _enhance_with_environment() before writing output, which inserts default "N/A" placeholders for any missing environment fields. This ensures the JSON schema remains consistent and valid for downstream consumers, preventing KeyError exceptions in parsing tools.

Which format should I use for generating HTML reports?

Use the JSON format. The HTML report generator in Twinkle Eval consumes the complete nested structure provided by JSONExporter, including the full environment block and hierarchical dataset information. The CSV format lacks the necessary nesting to support rich report generation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}