How Scores Are Made Consistent and Cacheable in the Phoenix Model

The Phoenix model achieves deterministic, reproducible scores across training and serving by combining pure JAX linear algebra with consistent RMS normalization via the MuON optimizer, while leveraging JAX's persistent compilation cache to eliminate recompilation overhead.

The Phoenix recommendation engine, available in the xai-org/x-algorithm repository, ensures that embedding-based similarity scores remain bitwise-identical between training checkpoints and production inference. This consistency is achieved through three coordinated mechanisms: deterministic matrix operations without side effects, RMS scaling that stabilizes embedding magnitudes, and persistent binary caching of compiled scoring kernels.

Deterministic Scoring Logic in recsys_two_tower_model.py

At the core of the Phoenix model's consistency guarantee is the pure functional scoring implementation in phoenix/xrex/models/recsys_two_tower_model.py. The compute_scores function performs matrix multiplication between user embeddings and candidate post embeddings using deterministic JAX operations.

The implementation deliberately avoids randomness or stateful operations. Scores are computed as:

# From phoenix/xrex/models/recsys_two_tower_model.py

def compute_scores(user_emb: jnp.ndarray,
                   cand_emb: jnp.ndarray,
                   top_k: int) -> Tuple[jnp.ndarray, jnp.ndarray]:
    # Deterministic matrix multiply with bfloat16 for speed

    raw_scores = (user_emb.astype(jnp.bfloat16) @ cand_emb.T).astype(jnp.float32)
    # Deterministic top-k selection (no randomness)

    top_scores, top_idxs = jax.lax.top_k(raw_scores, k=top_k)
    return top_idxs, top_scores

The use of jax.lax.top_k ensures that the same input tensors always produce identical indices and score values, eliminating variance that could arise from non-deterministic sorting algorithms or approximate nearest neighbor libraries.

Consistent RMS Scaling via the MuON Optimizer

Score magnitudes remain stable across training restarts due to the consistent RMS mechanism implemented in phoenix/xrex/optimizers/recsys/muon.py. The MuON optimizer applies a muon_consistent_rms scaling factor to embedding weights during updates, normalizing the effective scale of embeddings regardless of initialization or training step.

This prevents "score drift" where reloading a checkpoint might produce different similarity values due to gradient scaling variations. The optimizer applies the scaling as:

# From phoenix/xrex/optimizers/recsys/muon.py

def muon_consistent_scale(param: jnp.ndarray,
                          cfg: MuonConfig) -> jnp.ndarray:
    if cfg.muon_consistent_rms is not None:
        # Scale by sqrt of max dimension times configured RMS factor

        scale = jnp.sqrt(jnp.maximum(param.shape[-2], param.shape[-1]))
        return param * (scale * cfg.muon_consistent_rms)
    return param

By baking this normalization into the weight updates, the model ensures that dot-product scores maintain consistent statistical distributions between training epochs and serving environments.

JAX Persistent Compilation Cache

To make scores cacheable across process restarts without recompilation overhead, the Phoenix system configures JAX's persistent compilation cache in phoenix/xrex/train/trainer.py. This stores compiled XLA binaries to disk, keyed by operation hashes.

# Configuration from phoenix/xrex/train/trainer.py

jax.config.update("jax_compilation_cache_dir", "/tmp/jax-cache")
jax.config.update("jax_persistent_cache_min_compile_time_secs", 2.0)

When the scoring function is first compiled, the resultant binary is written to the cache directory. Subsequent process launches load the cached executable directly, ensuring that:

• Inference latency remains consistent from the first request
• Score computation paths are identical across replicas
• No compilation nondeterminism affects score values

The phoenix/xrex/inference/metrics.py module tracks cache hit rates, confirming the cacheability of the scoring pipeline in production environments.

Summary

• The Phoenix model uses pure JAX operations (jax.lax.top_k) in recsys_two_tower_model.py to guarantee bitwise-deterministic scores across runs.
• Consistent RMS scaling in the MuON optimizer (muon.py) normalizes embedding magnitudes, preventing score drift when loading checkpoints.
• JAX's persistent compilation cache configuration in trainer.py makes scoring kernels immediately reusable across process restarts, eliminating warm-up variance.
• Together, these mechanisms ensure that recommendation scores remain identical from training through billion-scale serving.

Frequently Asked Questions

What is the purpose of consistent_rms in the Phoenix optimizer?

The consistent_rms parameter in the MuON optimizer normalizes the scale of embedding weights by multiplying parameters against a factor derived from the square root of the tensor dimensions. This ensures that the magnitude of dot-product scores remains stable across training restarts and checkpoint reloads, preventing the distribution drift that typically occurs with unnormalized gradient updates.

How does JAX persistent caching improve score consistency?

JAX persistent caching stores compiled XLA binaries to disk using jax_compilation_cache_dir, ensuring that the exact same machine code executes the scoring logic on every run. This eliminates compilation-time nondeterminism and guarantees that the compute_scores function in recsys_two_tower_model.py produces identical outputs from the very first inference request, rather than varying during initial warm-up phases.

Why does Phoenix use jax.lax.top_k for candidate selection?

The Phoenix model uses jax.lax.top_k instead of stochastic or approximate methods because it provides deterministic, reproducible rankings that are integral to the consistency guarantee. Unlike randomized approximation algorithms or library-specific sorting implementations that may vary across hardware, jax.lax.top_k produces the same indices and scores for identical input embeddings on every execution, which is critical for A/B testing and reproducible recommendations.

Where is the dataset abstraction that feeds deterministic batches?

The deterministic data pipeline is implemented in phoenix/xrex/data/parquet_recsys.py, which provides the PhoenixDataset class. This abstraction ensures that batches fed into the scoring functions maintain consistent ordering and tensor shapes, complementing the deterministic scoring logic by preventing data-loading variance from affecting score consistency.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}