How ExoJAX Supports HMC-NUTS and SVI for Bayesian Atmospheric Retrieval

ExoJAX supports HMC-NUTS and SVI by providing a fully differentiable JAX-based radiative transfer pipeline that integrates seamlessly with NumPyro and Blackjax, enabling both exact MCMC sampling and fast variational inference on exoplanet spectra.

ExoJAX is an open-source Python library for exoplanet atmospheric retrieval built entirely on JAX. By implementing radiative transfer, opacity calculation, and instrumental convolution as differentiable functions, ExoJAX enables gradient-based Bayesian inference techniques including Hamiltonian Monte Carlo with No-U-Turn Sampling (HMC-NUTS) and Stochastic Variational Inference (SVI) through native integration with probabilistic programming frameworks.

JAX-Based Architecture for Differentiable Radiative Transfer

The foundation of ExoJAX's inference capabilities lies in its fully differentiable forward model. All components—from the radiative transfer solver to opacity grids—are written using jax.numpy, allowing automatic differentiation via jax.grad and just-in-time compilation via jax.jit.

The core radiative transfer engine is implemented in [src/exojax/rt/emis.py](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/rt/emis.py) as the ArtEmisPure class, which computes emergent emission spectra given temperature-pressure profiles and opacity data. Opacity sources include OpaPremodit for line-by-line calculations (defined in [src/exojax/opacity/premodit.py](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/opacity/premodit.py)) and OpaCIA for collision-induced absorption ([src/exojax/opacity/cia.py](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/opacity/cia.py)).

Utility functions in [src/exojax/utils/grids.py](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/utils/grids.py) generate wavenumber grids via wavenumber_grid, while [src/exojax/utils/instfunc.py](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/utils/instfunc.py) provides resolution_to_gaussian_std for instrumental broadening. Because every operation is JAX-native, the entire pipeline from molecular opacities to detector response remains differentiable, enabling gradient-based optimization and sampling.

Hamiltonian Monte Carlo with No-U-Turn Sampling (HMC-NUTS)

ExoJAX supports HMC-NUTS through two distinct interfaces: the high-level NumPyro framework (recommended for most users) and the lower-level Blackjax library (for custom sampling logic).

High-Level Inference with NumPyro

The preferred approach uses NumPyro's NUTS kernel and MCMC interface. In [tests/endtoend/reverse/reverse_premodit.py](https://github.com/hajimekawahara/exojax/blob/master/tests/endtoend/reverse/reverse_premodit.py), the probabilistic model defines priors via numpyro.sample and connects them to the forward model:

import jax.numpy as jnp
import numpyro
import numpyro.distributions as dist
from numpyro.infer import MCMC, NUTS
from jax import random, config

config.update("jax_enable_x64", True)

def forward_spectrum(params, nu_grid):
    # Placeholder: Replace with actual ArtEmisPure pipeline

    line_center = 1300.0
    sigma = 0.5
    return jnp.exp(-0.5 * ((nu_grid - line_center) / sigma) ** 2)

def model(y_obs, nu_grid):
    Rp = numpyro.sample("Rp", dist.Uniform(0.4, 1.2))
    RV = numpyro.sample("RV", dist.Uniform(5.0, 15.0))
    MMR_CH4 = numpyro.sample("MMR_CH4", dist.Uniform(0.0, 0.015))
    T0 = numpyro.sample("T0", dist.Uniform(1000.0, 1500.0))
    alpha = numpyro.sample("alpha", dist.Uniform(0.05, 0.2))
    vsini = numpyro.sample("vsini", dist.Uniform(15.0, 25.0))

    params = (Rp, RV, MMR_CH4, T0, alpha, vsini)
    mu = forward_spectrum(params, nu_grid)
    sigma = numpyro.sample("sigma", dist.HalfCauchy(0.1))
    numpyro.sample("obs", dist.Normal(mu, sigma), obs=y_obs)

# Run sampler

rng_key = random.PRNGKey(0)
nu_grid = jnp.linspace(1200.0, 1400.0, 1500)
y_obs = forward_spectrum((1.0, 10.0, 0.005, 1200., 0.1, 20.), nu_grid) \
        + 0.05 * random.normal(rng_key, (nu_grid.size,))

nuts_kernel = NUTS(model, forward_mode_differentiation=False)
mcmc = MCMC(nuts_kernel, num_warmup=500, num_samples=1000)
mcmc.run(rng_key, y_obs=y_obs, nu_grid=nu_grid)
samples = mcmc.get_samples()

Key parameters include forward_mode_differentiation=False (toggle to True for potential speedups on specific hardware) and jax_enable_x64=True for double-precision floating point, which is essential for accurate radiative transfer calculations. The same forward_spectrum function used for the likelihood evaluation is automatically differentiated by JAX to compute gradients for the NUTS sampler.

Manual Control with Blackjax

For users requiring explicit control over the sampling loop, [tests/endtoend/reverse/reverse_premodit_blackjax.py](https://github.com/hajimekawahara/exojax/blob/master/tests/endtoend/reverse/reverse_premodit_blackjax.py) demonstrates direct Blackjax integration:

import jax.numpy as jnp
import blackjax
import numpyro.distributions as dist
from jax import random

def forward(params, nu_grid):
    return jnp.exp(-0.5 * ((nu_grid - 1300.) / 0.5) ** 2)

def logprob(params, nu_grid, y):
    mu = forward(params, nu_grid)
    sigma = 0.05
    return jnp.sum(dist.Normal(mu, sigma).log_prob(y))

# Setup

nu_grid = jnp.linspace(1200., 1400., 1500)
true_params = jnp.array([1.0, 10.0, 0.005, 1200., 0.1, 20.])
y = forward(true_params, nu_grid) + 0.05 * random.normal(random.PRNGKey(0), nu_grid.shape)

init = jnp.array([1.0, 10.0, 0.005, 1200., 0.1, 20.])
step_size = 1e-3
inverse_mass = jnp.ones_like(init)

nuts = blackjax.nuts(lambda p: logprob(p, nu_grid, y),
                    step_size=step_size,
                    inverse_mass_matrix=inverse_mass)

# Manual stepping

state = nuts.init(init)
rng_key = random.PRNGKey(42)
samples = []

for i in range(1000):
    rng_key, subkey = random.split(rng_key)
    state, _ = nuts.step(subkey, state)
    samples.append(state.position)

samples = jnp.stack(samples)

This approach requires manual tuning of the step_size and inverse_mass_matrix but offers flexibility for custom adaptation schemes or integration into larger JAX workflows.

Stochastic Variational Inference (SVI) for Fast Approximation

When full MCMC is computationally prohibitive, ExoJAX supports Stochastic Variational Inference via NumPyro's SVI class. The tutorial in documents/tutorials/get_started_svi.rst and the companion notebook get_started_svi.ipynb demonstrate variational approximations:

import jax.numpy as jnp
import numpyro
import numpyro.distributions as dist
from numpyro.infer import SVI, Trace_ELBO, AutoNormal
from jax import random, config

config.update("jax_enable_x64", True)

def model(y_obs=None, nu_grid=None):
    Rp = numpyro.sample("Rp", dist.Uniform(0.4, 1.2))
    RV = numpyro.sample("RV", dist.Uniform(5.0, 15.0))
    MMR_CH4 = numpyro.sample("MMR_CH4", dist.Uniform(0.0, 0.015))
    T0 = numpyro.sample("T0", dist.Uniform(1000.0, 1500.0))
    alpha = numpyro.sample("alpha", dist.Uniform(0.05, 0.2))
    vsini = numpyro.sample("vsini", dist.Uniform(15.0, 25.0))

    # Forward model (placeholder)

    mu = forward((Rp, RV, MMR_CH4, T0, alpha, vsini), nu_grid)
    sigma = numpyro.sample("sigma", dist.HalfCauchy(0.1))
    numpyro.sample("obs", dist.Normal(mu, sigma), obs=y_obs)

# Auto guide (mean-field Gaussian)

guide = AutoNormal(model)
optimizer = numpyro.optim.Adam(step_size=1e-3)
svi = SVI(model, guide, optimizer, loss=Trace_ELBO())

# Optimization loop

rng_key = random.PRNGKey(0)
state = svi.init(rng_key, y_obs=y_data, nu_grid=nu_grid)
for i in range(2000):
    state, loss = svi.update(state, y_obs=y_data, nu_grid=nu_grid)

params = svi.get_params(state)
posterior_samples = guide.sample_posterior(random.PRNGKey(1), params, sample_shape=(500,))

AutoNormal generates a mean-field Gaussian approximation automatically. For more expressive posteriors, users can substitute AutoMultivariateNormal or custom guide functions. SVI typically runs orders of magnitude faster than NUTS, making it suitable for rapid model exploration or large datasets.

Key Configuration and Utilities

Several configuration options ensure stable inference:

• Double precision: jax.config.update("jax_enable_x64", True) is required for accurate opacity calculations and radiative transfer.
• Forward mode differentiation: In NUTS(forward_mode_differentiation=...), setting this to True can accelerate gradient computation on certain accelerators, though False (reverse-mode) is generally preferred for high-dimensional parameter spaces.
• Grid utilities: wavenumber_grid and velocity_grid handle spectral sampling, while ipgauss_sampling applies instrumental broadening as a differentiable convolution.

The unified design means the same forward model function—whether implemented via ArtEmisPure or custom pipelines—powers all three inference modes, ensuring consistent physics across exact MCMC, manual HMC, and variational approximations.

Summary

• Unified differentiable pipeline: ExoJAX implements radiative transfer and opacity calculations in pure JAX, enabling automatic differentiation for gradient-based samplers.
• NumPyro integration: High-level HMC-NUTS is available through NUTS and MCMC classes, with automatic tuning and diagnostics.
• Blackjax support: Lower-level HMC is demonstrated in [reverse_premodit_blackjax.py](https://github.com/hajimekawahara/exojax/blob/master/tests/endtoend/reverse/reverse_premodit_blackjax.py), offering manual control over kernel stepping.
• SVI capability: Fast approximate inference uses SVI with AutoNormal guides, significantly reducing wall-clock time compared to MCMC.
• Critical configuration: Enable jax_enable_x64 for numerical stability and consider forward_mode_differentiation for hardware-specific optimization.

Frequently Asked Questions

What is the difference between using NumPyro and Blackjax for HMC-NUTS in ExoJAX?

NumPyro provides a high-level interface with automatic step-size adaptation, warm-up phases, and built-in diagnostics, making it the recommended choice for standard atmospheric retrievals. Blackjax offers a lower-level API where you manually initialize the kernel and iterate sampling steps, which is useful for custom adaptation schemes or integration into specialized JAX workflows. Both use the same ExoJAX forward model and JAX gradients.

Why is jax_enable_x64 required for ExoJAX inference?

Astrophysical radiative transfer calculations involve large dynamic ranges in opacity and flux values. Single-precision (float32) arithmetic can introduce numerical instabilities during the matrix exponentiation and convolution operations in ArtEmisPure and OpaPremodit. Enabling x64 double precision ensures accurate likelihood evaluations and stable gradients for HMC-NUTS and SVI.

How does Stochastic Variational Inference compare to HMC-NUTS for exoplanet retrieval?

SVI approximates the posterior using a parameterized distribution (typically a mean-field Gaussian) optimized via gradient descent on the ELBO. It runs significantly faster than HMC-NUTS, often by orders of magnitude, making it ideal for rapid model testing or large datasets. However, HMC-NUTS provides asymptotically exact samples and better captures complex posterior correlations, which is critical for precise uncertainty quantification in atmospheric parameter retrieval.

Can I use custom opacity models with ExoJAX inference methods?

Yes. Because the inference methods accept arbitrary JAX functions, you can substitute OpaPremodit with custom opacity classes or pre-computed grids as long as they return JAX arrays. The probabilistic model simply needs to call your custom forward function, and gradients will flow correctly through the entire pipeline for use with NumPyro, Blackjax, or SVI.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}