# How ExoJAX Supports HMC-NUTS and SVI for Bayesian Atmospheric Retrieval

> ExoJAX enables advanced Bayesian atmospheric retrieval with HMC-NUTS and SVI. Leverage its JAX-based pipeline with NumPyro and Blackjax for efficient MCMC sampling and variational inference on exoplanet spectra.

- Repository: [Hajime Kawahara/exojax](https://github.com/hajimekawahara/exojax)
- Tags: deep-dive
- Published: 2026-03-03

---

**ExoJAX supports HMC-NUTS and SVI by providing a fully differentiable JAX-based radiative transfer pipeline that integrates seamlessly with NumPyro and Blackjax, enabling both exact MCMC sampling and fast variational inference on exoplanet spectra.**

ExoJAX is an open-source Python library for exoplanet atmospheric retrieval built entirely on JAX. By implementing radiative transfer, opacity calculation, and instrumental convolution as differentiable functions, ExoJAX enables gradient-based Bayesian inference techniques including Hamiltonian Monte Carlo with No-U-Turn Sampling (HMC-NUTS) and Stochastic Variational Inference (SVI) through native integration with probabilistic programming frameworks.

## JAX-Based Architecture for Differentiable Radiative Transfer

The foundation of ExoJAX's inference capabilities lies in its **fully differentiable forward model**. All components—from the radiative transfer solver to opacity grids—are written using `jax.numpy`, allowing automatic differentiation via `jax.grad` and just-in-time compilation via `jax.jit`.

The core radiative transfer engine is implemented in [[`src/exojax/rt/emis.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/rt/emis.py)](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/rt/emis.py) as the [`ArtEmisPure`](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/rt/emis.py) class, which computes emergent emission spectra given temperature-pressure profiles and opacity data. Opacity sources include [`OpaPremodit`](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/opacity/premodit.py) for line-by-line calculations (defined in [[`src/exojax/opacity/premodit.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/opacity/premodit.py)](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/opacity/premodit.py)) and [`OpaCIA`](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/opacity/cia.py) for collision-induced absorption ([[`src/exojax/opacity/cia.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/opacity/cia.py)](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/opacity/cia.py)).

Utility functions in [[`src/exojax/utils/grids.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/utils/grids.py)](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/utils/grids.py) generate wavenumber grids via `wavenumber_grid`, while [[`src/exojax/utils/instfunc.py`](https://github.com/hajimekawahara/exojax/blob/main/src/exojax/utils/instfunc.py)](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/utils/instfunc.py) provides `resolution_to_gaussian_std` for instrumental broadening. Because every operation is JAX-native, the entire pipeline from molecular opacities to detector response remains differentiable, enabling gradient-based optimization and sampling.

## Hamiltonian Monte Carlo with No-U-Turn Sampling (HMC-NUTS)

ExoJAX supports HMC-NUTS through two distinct interfaces: the high-level **NumPyro** framework (recommended for most users) and the lower-level **Blackjax** library (for custom sampling logic).

### High-Level Inference with NumPyro

The preferred approach uses NumPyro's `NUTS` kernel and `MCMC` interface. In [[`tests/endtoend/reverse/reverse_premodit.py`](https://github.com/hajimekawahara/exojax/blob/main/tests/endtoend/reverse/reverse_premodit.py)](https://github.com/hajimekawahara/exojax/blob/master/tests/endtoend/reverse/reverse_premodit.py), the probabilistic model defines priors via `numpyro.sample` and connects them to the forward model:

```python
import jax.numpy as jnp
import numpyro
import numpyro.distributions as dist
from numpyro.infer import MCMC, NUTS
from jax import random, config

config.update("jax_enable_x64", True)

def forward_spectrum(params, nu_grid):
    # Placeholder: Replace with actual ArtEmisPure pipeline

    line_center = 1300.0
    sigma = 0.5
    return jnp.exp(-0.5 * ((nu_grid - line_center) / sigma) ** 2)

def model(y_obs, nu_grid):
    Rp = numpyro.sample("Rp", dist.Uniform(0.4, 1.2))
    RV = numpyro.sample("RV", dist.Uniform(5.0, 15.0))
    MMR_CH4 = numpyro.sample("MMR_CH4", dist.Uniform(0.0, 0.015))
    T0 = numpyro.sample("T0", dist.Uniform(1000.0, 1500.0))
    alpha = numpyro.sample("alpha", dist.Uniform(0.05, 0.2))
    vsini = numpyro.sample("vsini", dist.Uniform(15.0, 25.0))

    params = (Rp, RV, MMR_CH4, T0, alpha, vsini)
    mu = forward_spectrum(params, nu_grid)
    sigma = numpyro.sample("sigma", dist.HalfCauchy(0.1))
    numpyro.sample("obs", dist.Normal(mu, sigma), obs=y_obs)

# Run sampler

rng_key = random.PRNGKey(0)
nu_grid = jnp.linspace(1200.0, 1400.0, 1500)
y_obs = forward_spectrum((1.0, 10.0, 0.005, 1200., 0.1, 20.), nu_grid) \
        + 0.05 * random.normal(rng_key, (nu_grid.size,))

nuts_kernel = NUTS(model, forward_mode_differentiation=False)
mcmc = MCMC(nuts_kernel, num_warmup=500, num_samples=1000)
mcmc.run(rng_key, y_obs=y_obs, nu_grid=nu_grid)
samples = mcmc.get_samples()

```

Key parameters include `forward_mode_differentiation=False` (toggle to `True` for potential speedups on specific hardware) and `jax_enable_x64=True` for double-precision floating point, which is essential for accurate radiative transfer calculations. The same `forward_spectrum` function used for the likelihood evaluation is automatically differentiated by JAX to compute gradients for the NUTS sampler.

### Manual Control with Blackjax

For users requiring explicit control over the sampling loop, [[`tests/endtoend/reverse/reverse_premodit_blackjax.py`](https://github.com/hajimekawahara/exojax/blob/main/tests/endtoend/reverse/reverse_premodit_blackjax.py)](https://github.com/hajimekawahara/exojax/blob/master/tests/endtoend/reverse/reverse_premodit_blackjax.py) demonstrates direct Blackjax integration:

```python
import jax.numpy as jnp
import blackjax
import numpyro.distributions as dist
from jax import random

def forward(params, nu_grid):
    return jnp.exp(-0.5 * ((nu_grid - 1300.) / 0.5) ** 2)

def logprob(params, nu_grid, y):
    mu = forward(params, nu_grid)
    sigma = 0.05
    return jnp.sum(dist.Normal(mu, sigma).log_prob(y))

# Setup

nu_grid = jnp.linspace(1200., 1400., 1500)
true_params = jnp.array([1.0, 10.0, 0.005, 1200., 0.1, 20.])
y = forward(true_params, nu_grid) + 0.05 * random.normal(random.PRNGKey(0), nu_grid.shape)

init = jnp.array([1.0, 10.0, 0.005, 1200., 0.1, 20.])
step_size = 1e-3
inverse_mass = jnp.ones_like(init)

nuts = blackjax.nuts(lambda p: logprob(p, nu_grid, y),
                    step_size=step_size,
                    inverse_mass_matrix=inverse_mass)

# Manual stepping

state = nuts.init(init)
rng_key = random.PRNGKey(42)
samples = []

for i in range(1000):
    rng_key, subkey = random.split(rng_key)
    state, _ = nuts.step(subkey, state)
    samples.append(state.position)

samples = jnp.stack(samples)

```

This approach requires manual tuning of the `step_size` and `inverse_mass_matrix` but offers flexibility for custom adaptation schemes or integration into larger JAX workflows.

## Stochastic Variational Inference (SVI) for Fast Approximation

When full MCMC is computationally prohibitive, ExoJAX supports **Stochastic Variational Inference** via NumPyro's `SVI` class. The tutorial in [`documents/tutorials/get_started_svi.rst`](https://github.com/hajimekawahara/exojax/blob/master/documents/tutorials/get_started_svi.rst) and the companion notebook [`get_started_svi.ipynb`](https://github.com/hajimekawahara/exojax/blob/master/documents/tutorials/get_started_svi.ipynb) demonstrate variational approximations:

```python
import jax.numpy as jnp
import numpyro
import numpyro.distributions as dist
from numpyro.infer import SVI, Trace_ELBO, AutoNormal
from jax import random, config

config.update("jax_enable_x64", True)

def model(y_obs=None, nu_grid=None):
    Rp = numpyro.sample("Rp", dist.Uniform(0.4, 1.2))
    RV = numpyro.sample("RV", dist.Uniform(5.0, 15.0))
    MMR_CH4 = numpyro.sample("MMR_CH4", dist.Uniform(0.0, 0.015))
    T0 = numpyro.sample("T0", dist.Uniform(1000.0, 1500.0))
    alpha = numpyro.sample("alpha", dist.Uniform(0.05, 0.2))
    vsini = numpyro.sample("vsini", dist.Uniform(15.0, 25.0))

    # Forward model (placeholder)

    mu = forward((Rp, RV, MMR_CH4, T0, alpha, vsini), nu_grid)
    sigma = numpyro.sample("sigma", dist.HalfCauchy(0.1))
    numpyro.sample("obs", dist.Normal(mu, sigma), obs=y_obs)

# Auto guide (mean-field Gaussian)

guide = AutoNormal(model)
optimizer = numpyro.optim.Adam(step_size=1e-3)
svi = SVI(model, guide, optimizer, loss=Trace_ELBO())

# Optimization loop

rng_key = random.PRNGKey(0)
state = svi.init(rng_key, y_obs=y_data, nu_grid=nu_grid)
for i in range(2000):
    state, loss = svi.update(state, y_obs=y_data, nu_grid=nu_grid)

params = svi.get_params(state)
posterior_samples = guide.sample_posterior(random.PRNGKey(1), params, sample_shape=(500,))

```

**AutoNormal** generates a mean-field Gaussian approximation automatically. For more expressive posteriors, users can substitute `AutoMultivariateNormal` or custom guide functions. SVI typically runs orders of magnitude faster than NUTS, making it suitable for rapid model exploration or large datasets.

## Key Configuration and Utilities

Several configuration options ensure stable inference:

- **Double precision**: `jax.config.update("jax_enable_x64", True)` is required for accurate opacity calculations and radiative transfer.
- **Forward mode differentiation**: In `NUTS(forward_mode_differentiation=...)`, setting this to `True` can accelerate gradient computation on certain accelerators, though `False` (reverse-mode) is generally preferred for high-dimensional parameter spaces.
- **Grid utilities**: [`wavenumber_grid`](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/utils/grids.py) and [`velocity_grid`](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/utils/grids.py) handle spectral sampling, while [`ipgauss_sampling`](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/utils/instfunc.py) applies instrumental broadening as a differentiable convolution.

The unified design means the same forward model function—whether implemented via [`ArtEmisPure`](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/rt/emis.py) or custom pipelines—powers all three inference modes, ensuring consistent physics across exact MCMC, manual HMC, and variational approximations.

## Summary

- **Unified differentiable pipeline**: ExoJAX implements radiative transfer and opacity calculations in pure JAX, enabling automatic differentiation for gradient-based samplers.
- **NumPyro integration**: High-level HMC-NUTS is available through [`NUTS`](https://github.com/hajimekawahara/exojax/blob/master/tests/endtoend/reverse/reverse_premodit.py) and [`MCMC`](https://github.com/hajimekawahara/exojax/blob/master/tests/endtoend/reverse/reverse_premodit.py) classes, with automatic tuning and diagnostics.
- **Blackjax support**: Lower-level HMC is demonstrated in [[`reverse_premodit_blackjax.py`](https://github.com/hajimekawahara/exojax/blob/main/reverse_premodit_blackjax.py)](https://github.com/hajimekawahara/exojax/blob/master/tests/endtoend/reverse/reverse_premodit_blackjax.py), offering manual control over kernel stepping.
- **SVI capability**: Fast approximate inference uses [`SVI`](https://github.com/hajimekawahara/exojax/blob/master/documents/tutorials/get_started_svi.rst) with [`AutoNormal`](https://github.com/hajimekawahara/exojax/blob/master/documents/tutorials/get_started_svi.rst) guides, significantly reducing wall-clock time compared to MCMC.
- **Critical configuration**: Enable `jax_enable_x64` for numerical stability and consider `forward_mode_differentiation` for hardware-specific optimization.

## Frequently Asked Questions

### What is the difference between using NumPyro and Blackjax for HMC-NUTS in ExoJAX?

NumPyro provides a high-level interface with automatic step-size adaptation, warm-up phases, and built-in diagnostics, making it the recommended choice for standard atmospheric retrievals. Blackjax offers a lower-level API where you manually initialize the kernel and iterate sampling steps, which is useful for custom adaptation schemes or integration into specialized JAX workflows. Both use the same ExoJAX forward model and JAX gradients.

### Why is `jax_enable_x64` required for ExoJAX inference?

Astrophysical radiative transfer calculations involve large dynamic ranges in opacity and flux values. Single-precision (float32) arithmetic can introduce numerical instabilities during the matrix exponentiation and convolution operations in [`ArtEmisPure`](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/rt/emis.py) and [`OpaPremodit`](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/opacity/premodit.py). Enabling x64 double precision ensures accurate likelihood evaluations and stable gradients for HMC-NUTS and SVI.

### How does Stochastic Variational Inference compare to HMC-NUTS for exoplanet retrieval?

SVI approximates the posterior using a parameterized distribution (typically a mean-field Gaussian) optimized via gradient descent on the ELBO. It runs significantly faster than HMC-NUTS, often by orders of magnitude, making it ideal for rapid model testing or large datasets. However, HMC-NUTS provides asymptotically exact samples and better captures complex posterior correlations, which is critical for precise uncertainty quantification in atmospheric parameter retrieval.

### Can I use custom opacity models with ExoJAX inference methods?

Yes. Because the inference methods accept arbitrary JAX functions, you can substitute [`OpaPremodit`](https://github.com/hajimekawahara/exojax/blob/master/src/exojax/opacity/premodit.py) with custom opacity classes or pre-computed grids as long as they return JAX arrays. The probabilistic model simply needs to call your custom forward function, and gradients will flow correctly through the entire pipeline for use with NumPyro, Blackjax, or SVI.