Data Formats Compatible with ExoJAX's Database and Data Modules: A Complete Guide

ExoJAX supports native spectroscopic formats including ExoMol (.trans.bz2, .states.bz2), HITRAN/HITEMP (.par), VALD 3 (ASCII), Kurucz (.dat), CIA (.cia), and Mie-scattering grids (.npz), automatically converting them to lazy-IO HDF5/vaex representations for JAX-compatible high-performance computing.

ExoJAX provides a unified, RADIS-based API for molecular and atomic spectroscopic databases used in exoplanet atmospheric modeling. Understanding what data formats are compatible with ExoJAX's database modules is essential for loading line lists, opacity data, and scattering parameters efficiently.

Supported Native Data Formats

ExoJAX's database layer abstracts multiple spectroscopic sources through dedicated reader classes. Each class ingests source-specific files and optionally caches them in a high-performance format.

ExoMol Molecular Databases

ExoMol delivers transition data as .trans.bz2 (transitions) and .states.bz2 (energy levels) files, accompanied by optional .def and .broad definition files. The MdbExomol class in src/exojax/database/exomol/api.py reads these via the RADIS C-API and loads them into a vaex DataFrame when engine="vaex" is specified (lines 34-44). On first load, the raw text converts to an HDF5/vaex cache; subsequent loads read the cached .h5 file for significant speed improvements (lines 92-93).

HITRAN and HITEMP Line Lists

HITRAN and HITEMP databases provide .par parameter files, .states files, and optional .broad or .cia files. The MdbHitran class inherits from HITRANDatabaseManager and parses .par files via RADIS, supporting both vaex and pytables engines (selectable via the engine parameter). Initialization arguments and engine selection are defined in src/exojax/database/hitran/api.py (lines 30-44). Like ExoMol, HITRAN data automatically caches to HDF5/vaex on first load.

VALD 3 Atomic Line Lists

VALD 3 "Long format" ASCII line lists—extracted via Extract All, Extract Stellar, or Extract Element queries—are parsed by read_ExAll in src/exojax/database/core_atom/io.py. This function reads the CSV-like file into a pandas.DataFrame and, when engine="vaex" is specified, converts it to a lazy-IO vaex.DataFrame (lines 136-154). If a matching .hdf5 cache exists, the function opens it directly to avoid re-parsing (lines 111-114).

Kurucz Atomic Data

Classic Kurucz line list files (*.dat style) are handled by read_kurucz in the same core_atom/io.py module. This function reads fixed-width ASCII files into a pandas.DataFrame, with optional conversion to vaex for memory-efficient operations (lines 262-270).

Collision-Induced Absorption (CIA)

Plain text .cia files (e.g., H2-H2_2011.cia) contain tabulated absorption coefficients for molecular collisions. The read_cia function in src/exojax/database/cia/io.py reads header information and data tables, returning wavenumber, temperature, and coefficient arrays suitable for opacity calculations (lines 29-73).

Mie-Scattering Grids

For atmospheric cloud modeling, Mie-scattering data is stored as binary *.npz files generated by pymiescatt. The read_miegrid function in src/exojax/database/mie.py loads these NumPy archives and returns the grid, particle radii, and width arrays (lines 86-95).

Automatic Conversion to JAX-Compatible Formats

All database classes share a common API surface defined in src/exojax/database/_common/commonapi.py (lines 42-48). This ensures uniform access to attributes such as nu_lines, A, elower, gpp, and gamma_self regardless of the underlying source format. The automatic conversion to HDF5/vaex creates lazy-IO structures that integrate seamlessly with JAX's just-in-time compilation requirements, eliminating memory bottlenecks during high-resolution spectral synthesis.

Loading Data from ExoMol

To load an ExoMol database with automatic caching to HDF5/vaex:

from exojax.database.exomol import api as exomol_api

mdb = exomol_api.MdbExomol(
    path="/path/to/12C-16O/Li2015",   # directory containing .trans/.states files

    nurange=[2000, 2500],            # wavenumber range (cm‑1)

    crit=1e-30,                      # line‑strength cutoff

    engine="vaex",                   # use lazy‑IO vaex (default)

)

print(mdb.nu_lines[:5])   # first five line centre wavenumbers

print(mdb.A[:5])          # Einstein‑A coefficients

The constructor signature and supported arguments are implemented in src/exojax/database/exomol/api.py (lines 34-44).

Working with VALD Atomic Data

To read a VALD "Long format" line list and convert it to a lazy DataFrame:

from exojax.database.core_atom.io import read_ExAll

df = read_ExAll(
    allf="/path/to/Fe1_extract_all.gz",
    engine="vaex",      # lazily load as vaex DataFrame

)

print(df.head())

The conversion logic and vaex integration are defined in src/exojax/database/core_atom/io.py (lines 136-154).

Reading CIA and Mie-Scattering Files

For collision-induced absorption data used in high-temperature atmospheres:

from exojax.database.cia.io import read_cia

nus, nue = 400.0, 5000.0          # requested wavenumber window

nucia, tcia, ac = read_cia(
    filename="/path/to/H2-H2_2011.cia",
    nus=nus,
    nue=nue,
)

print(nucia.shape, ac.shape)    # arrays ready for opacity calculations

Implementation details for the CIA reader are in src/exojax/database/cia/io.py (lines 29-73).

For cloud microphysics using pre-computed Mie scattering:

from exojax.database.mie import read_miegrid

grid, rg_arr, sigmag_arr = read_miegrid("example_miegrid.npz")
print(grid.shape, rg_arr[:3])

The Mie grid loader resides in src/exojax/database/mie.py (lines 86-95).

Summary

• ExoJAX ingests native formats from ExoMol (.trans.bz2, .states.bz2), HITRAN/HITEMP (.par), VALD 3 (ASCII), Kurucz (.dat), CIA (.cia), and Mie grids (.npz).
• All databases automatically convert to HDF5/vaex lazy-IO formats on first load, caching to .h5 files for subsequent access.
• The unified API in src/exojax/database/_common/commonapi.py ensures consistent attribute access across molecular and atomic data sources.
• Selectable engines (vaex or pytables) allow optimization for memory constraints and computational requirements.

Frequently Asked Questions

Can ExoJAX read HITRAN data directly without conversion?

No, but the conversion is automatic and transparent. When you instantiate MdbHitran with a .par file, ExoJAX parses it via RADIS and creates an HDF5/vaex cache. Subsequent initializations read the cached file directly, providing fast, JAX-compatible access without manual preprocessing.

What is the advantage of using vaex over pandas for spectroscopic data?

vaex provides lazy, out-of-core DataFrames that handle billions of lines without loading everything into RAM. This is critical for high-temperature HITEMP databases or large ExoMol molecules where pandas would exhaust memory. The engine="vaex" parameter (default in most loaders) enables this optimization.

How does ExoJAX handle updates when source database files change?

ExoJAX checks for existing HDF5 cache files before loading. If the cache exists, it loads directly from .h5 (see src/exojax/database/core_atom/io.py lines 111-114). To force re-conversion after updating source files, manually delete the cached .hdf5 files in the database directory, or modify the cache detection logic in the respective api.py modules.

Are there any limitations when reading VALD "Extract Element" versus "Extract All" formats?

Both formats use the same read_ExAll parser in src/exojax/database/core_atom/io.py. The function expects the standard VALD "Long format" columns (wavelength, log gf, excitation potential, etc.). As long as the file follows VALD's ASCII output specification, the specific extraction type (All, Stellar, or Element) does not affect compatibility.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}