# Data Formats Compatible with ExoJAX's Database and Data Modules: A Complete Guide

> Explore data formats ExoJAX supports including ExoMol HITRAN VALD and more. Our guide details native spectroscopic formats compatible with ExoJAX modules.

- Repository: [Hajime Kawahara/exojax](https://github.com/hajimekawahara/exojax)
- Tags: how-to-guide
- Published: 2026-03-03

---

**ExoJAX supports native spectroscopic formats including ExoMol (.trans.bz2, .states.bz2), HITRAN/HITEMP (.par), VALD 3 (ASCII), Kurucz (.dat), CIA (.cia), and Mie-scattering grids (.npz), automatically converting them to lazy-IO HDF5/vaex representations for JAX-compatible high-performance computing.**

ExoJAX provides a unified, RADIS-based API for molecular and atomic spectroscopic databases used in exoplanet atmospheric modeling. Understanding what data formats are compatible with ExoJAX's database modules is essential for loading line lists, opacity data, and scattering parameters efficiently.

## Supported Native Data Formats

ExoJAX's database layer abstracts multiple spectroscopic sources through dedicated reader classes. Each class ingests source-specific files and optionally caches them in a high-performance format.

### ExoMol Molecular Databases

**ExoMol** delivers transition data as `.trans.bz2` (transitions) and `.states.bz2` (energy levels) files, accompanied by optional `.def` and `.broad` definition files. The `MdbExomol` class in [`src/exojax/database/exomol/api.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/database/exomol/api.py) reads these via the RADIS C-API and loads them into a **vaex** `DataFrame` when `engine="vaex"` is specified (lines 34-44). On first load, the raw text converts to an HDF5/vaex cache; subsequent loads read the cached `.h5` file for significant speed improvements (lines 92-93).

### HITRAN and HITEMP Line Lists

**HITRAN** and **HITEMP** databases provide `.par` parameter files, `.states` files, and optional `.broad` or `.cia` files. The `MdbHitran` class inherits from `HITRANDatabaseManager` and parses `.par` files via RADIS, supporting both **vaex** and **pytables** engines (selectable via the `engine` parameter). Initialization arguments and engine selection are defined in [`src/exojax/database/hitran/api.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/database/hitran/api.py) (lines 30-44). Like ExoMol, HITRAN data automatically caches to HDF5/vaex on first load.

### VALD 3 Atomic Line Lists

**VALD 3** "Long format" ASCII line lists—extracted via `Extract All`, `Extract Stellar`, or `Extract Element` queries—are parsed by `read_ExAll` in [`src/exojax/database/core_atom/io.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/database/core_atom/io.py). This function reads the CSV-like file into a `pandas.DataFrame` and, when `engine="vaex"` is specified, converts it to a lazy-IO `vaex.DataFrame` (lines 136-154). If a matching `.hdf5` cache exists, the function opens it directly to avoid re-parsing (lines 111-114).

### Kurucz Atomic Data

Classic **Kurucz** line list files (`*.dat` style) are handled by `read_kurucz` in the same [`core_atom/io.py`](https://github.com/hajimekawahara/exojax/blob/main/core_atom/io.py) module. This function reads fixed-width ASCII files into a `pandas.DataFrame`, with optional conversion to vaex for memory-efficient operations (lines 262-270).

### Collision-Induced Absorption (CIA)

Plain text **.cia** files (e.g., `H2-H2_2011.cia`) contain tabulated absorption coefficients for molecular collisions. The `read_cia` function in [`src/exojax/database/cia/io.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/database/cia/io.py) reads header information and data tables, returning wavenumber, temperature, and coefficient arrays suitable for opacity calculations (lines 29-73).

### Mie-Scattering Grids

For atmospheric cloud modeling, **Mie-scattering** data is stored as binary `*.npz` files generated by *pymiescatt*. The `read_miegrid` function in [`src/exojax/database/mie.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/database/mie.py) loads these NumPy archives and returns the grid, particle radii, and width arrays (lines 86-95).

## Automatic Conversion to JAX-Compatible Formats

All database classes share a common API surface defined in [`src/exojax/database/_common/commonapi.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/database/_common/commonapi.py) (lines 42-48). This ensures uniform access to attributes such as `nu_lines`, `A`, `elower`, `gpp`, and `gamma_self` regardless of the underlying source format. The automatic conversion to **HDF5/vaex** creates lazy-IO structures that integrate seamlessly with JAX's just-in-time compilation requirements, eliminating memory bottlenecks during high-resolution spectral synthesis.

## Loading Data from ExoMol

To load an ExoMol database with automatic caching to HDF5/vaex:

```python
from exojax.database.exomol import api as exomol_api

mdb = exomol_api.MdbExomol(
    path="/path/to/12C-16O/Li2015",   # directory containing .trans/.states files

    nurange=[2000, 2500],            # wavenumber range (cm‑1)

    crit=1e-30,                      # line‑strength cutoff

    engine="vaex",                   # use lazy‑IO vaex (default)

)

print(mdb.nu_lines[:5])   # first five line centre wavenumbers

print(mdb.A[:5])          # Einstein‑A coefficients

```

The constructor signature and supported arguments are implemented in [`src/exojax/database/exomol/api.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/database/exomol/api.py) (lines 34-44).

## Working with VALD Atomic Data

To read a VALD "Long format" line list and convert it to a lazy DataFrame:

```python
from exojax.database.core_atom.io import read_ExAll

df = read_ExAll(
    allf="/path/to/Fe1_extract_all.gz",
    engine="vaex",      # lazily load as vaex DataFrame

)

print(df.head())

```

The conversion logic and vaex integration are defined in [`src/exojax/database/core_atom/io.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/database/core_atom/io.py) (lines 136-154).

## Reading CIA and Mie-Scattering Files

For collision-induced absorption data used in high-temperature atmospheres:

```python
from exojax.database.cia.io import read_cia

nus, nue = 400.0, 5000.0          # requested wavenumber window

nucia, tcia, ac = read_cia(
    filename="/path/to/H2-H2_2011.cia",
    nus=nus,
    nue=nue,
)

print(nucia.shape, ac.shape)    # arrays ready for opacity calculations

```

Implementation details for the CIA reader are in [`src/exojax/database/cia/io.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/database/cia/io.py) (lines 29-73).

For cloud microphysics using pre-computed Mie scattering:

```python
from exojax.database.mie import read_miegrid

grid, rg_arr, sigmag_arr = read_miegrid("example_miegrid.npz")
print(grid.shape, rg_arr[:3])

```

The Mie grid loader resides in [`src/exojax/database/mie.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/database/mie.py) (lines 86-95).

## Summary

- **ExoJAX** ingests native formats from **ExoMol** (`.trans.bz2`, `.states.bz2`), **HITRAN/HITEMP** (`.par`), **VALD 3** (ASCII), **Kurucz** (`.dat`), **CIA** (`.cia`), and **Mie grids** (`.npz`).
- All databases automatically convert to **HDF5/vaex** lazy-IO formats on first load, caching to `.h5` files for subsequent access.
- The unified API in [`src/exojax/database/_common/commonapi.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/database/_common/commonapi.py) ensures consistent attribute access across molecular and atomic data sources.
- Selectable engines (`vaex` or `pytables`) allow optimization for memory constraints and computational requirements.

## Frequently Asked Questions

### Can ExoJAX read HITRAN data directly without conversion?

No, but the conversion is automatic and transparent. When you instantiate `MdbHitran` with a `.par` file, ExoJAX parses it via RADIS and creates an HDF5/vaex cache. Subsequent initializations read the cached file directly, providing fast, JAX-compatible access without manual preprocessing.

### What is the advantage of using vaex over pandas for spectroscopic data?

**vaex** provides lazy, out-of-core DataFrames that handle billions of lines without loading everything into RAM. This is critical for high-temperature HITEMP databases or large ExoMol molecules where `pandas` would exhaust memory. The `engine="vaex"` parameter (default in most loaders) enables this optimization.

### How does ExoJAX handle updates when source database files change?

ExoJAX checks for existing HDF5 cache files before loading. If the cache exists, it loads directly from `.h5` (see [`src/exojax/database/core_atom/io.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/database/core_atom/io.py) lines 111-114). To force re-conversion after updating source files, manually delete the cached `.hdf5` files in the database directory, or modify the cache detection logic in the respective [`api.py`](https://github.com/hajimekawahara/exojax/blob/main/api.py) modules.

### Are there any limitations when reading VALD "Extract Element" versus "Extract All" formats?

Both formats use the same `read_ExAll` parser in [`src/exojax/database/core_atom/io.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/database/core_atom/io.py). The function expects the standard VALD "Long format" columns (wavelength, log gf, excitation potential, etc.). As long as the file follows VALD's ASCII output specification, the specific extraction type (All, Stellar, or Element) does not affect compatibility.