How Phoenix Processes Recent Engagement History for the Transformer Model

Phoenix processes recent engagement history by loading raw interaction events into structured sequences, embedding post and author hashes alongside action type tokens, applying right-anchored positional encodings, and feeding the masked sequence through a standard multi-head Transformer to generate contextualized user representations for ranking.

The xai-org/x-algorithm repository implements Phoenix as a recommendation system that treats recent user interactions as first-class input tokens. Unlike systems that rely solely on aggregated user profiles, Phoenix processes recent engagement history for the transformer model as a temporally ordered sequence, enabling the architecture to learn patterns like "a recent like predicts the next click" or "view-to-like sequences boost relevance."

Loading and Encoding Historical Engagement Data

Phoenix begins by reading recent engagement events from Parquet files or Kafka streams through PhoenixDataset (or PhoenixKafkaDataset) defined in phoenix/xrex/data/parquet_recsys.py. Each user’s interaction history is stored in a history_seq dictionary containing three critical components:

• post_hashes: Identifiers for the posts the user engaged with
• auth_hashes: Identifiers for the authors of those posts
• actions: One-hot vectors representing the engagement type (click, like, retweet, etc.)

The system maps engagement types to integer token IDs using engagement_to_ids(metric_group) from phoenix/xrex/data/recsys/constants.py. This mapping converts raw behavioral signals—such as likes or replies—into discrete tokens that the model processes like any other input symbol.

from xrex.data.parquet_recsys import PhoenixDataset

# Load dataset with recent engagement history

dataset = PhoenixDataset(
    parquet_path="/data/training.parquet",
    history_seq_len=64,          # Retain up to 64 recent events

    metric_group="default",
)

batch = dataset.sample_batch(batch_size=32)

# batch["history_seq"] contains post_hashes, auth_hashes, and actions

Building Token Embeddings with Right-Anchored Positioning

Inside RecsysModel.__call__ (phoenix/xrex/models/recsys_model.py), the raw history tensors undergo padding and truncation to a fixed history_seq_len (e.g., 64 tokens). The system constructs embeddings through two parallel pathways:

Hash Embeddings: Post and author hashes pass through RecsysEmbedding, which implements hash table lookups to retrieve dense vector representations.

Action Type Embeddings: The integer action IDs (from engagement_to_ids) receive learned type embeddings through a dedicated embedding matrix, allowing the model to distinguish between different interaction semantics.

Positional Encoding: Phoenix applies right_anchored_rope_positions (lines 64-80 in the model implementation) to compute Rotary Position Embeddings (RoPE) that anchor the most recent events at the end of the history window. This ensures the Transformer always sees the newest interactions in consistent positional slots regardless of sequence length variation.

from xrex.models.recsys_model import right_anchored_rope_positions

# Compute right-aligned positions for history tokens

positions = right_anchored_rope_positions(
    padding_mask=batch["padding_mask"],
    history_seq_len=64,
    num_user_prefix_tokens=4,    # Account for user profile prefix tokens

)

# positions shape: (batch_size, total_seq_len, 3)

Integrating History into the Transformer Forward Pass

The RecsysModel stacks three token groups into a single input sequence consumed by the Transformer:

1. User-prefix tokens: Static user profile information
2. History tokens: The embedded recent engagement sequence
3. Candidate tokens: Posts under consideration for ranking

An attention mask marks valid history positions while ignoring padded slots. The system uses packed_per_user_history_event_count to determine exactly how many history slots contain actual data versus padding, ensuring the Transformer attends only to real interactions.

The standard Transformer implementation (phoenix/xrex/models/transformer.py) processes this combined sequence through multi-head self-attention and feed-forward layers. By attending across the full context—including recent engagements—the model generates contextualized representations where historical behavior directly influences the current ranking scores.

from xrex.models.recsys_model import RecsysModel

model_cfg = {
    "history_seq_len": 64,
    "num_user_prefix_tokens": 4,
    # ... additional config

}

model = RecsysModel(model_cfg)

# Forward pass automatically embeds history and runs Transformer

logits, hidden_states = model(batch)

# hidden_states contain contextualized representations influenced by history

Optional Feature Engineering for Engagement Patterns

Beyond raw token sequences, Phoenix supports additional feature preprocessing through build_feature_prep_inputs in phoenix/xrex/models/recsys_feature_prep.py (line 618). This module engineers structured features such as engagement-count buckets and temporal aggregation statistics, concatenating them to the token embeddings before they reach the Transformer layers. This hybrid approach combines the sequence modeling power of attention mechanisms with explicit feature engineering for specific behavioral patterns.

Summary

• PhoenixDataset loads recent engagements from Parquet/Kafka into structured history_seq dictionaries containing post hashes, author hashes, and action vectors.
• engagement_to_ids maps interaction types (likes, clicks, retweets) to integer tokens that receive dedicated type embeddings alongside hash-based content embeddings.
• right_anchored_rope_positions ensures temporal ordering by fixing the most recent events at the end of the history window using right-aligned positional encodings.
• The Transformer consumes a concatenated sequence of user prefix, history, and candidate tokens, with attention masks filtering padded history slots via packed_per_user_history_event_count.
• Optional feature prep layers add engineered engagement statistics to complement the raw sequence representation.

Frequently Asked Questions

How does Phoenix handle variable-length engagement histories?

Phoenix pads or truncates all history sequences to a fixed history_seq_len (typically 64 events) during batching in PhoenixDataset. The packed_per_user_history_event_count tensor tracks the actual number of valid events per user, allowing the attention mask to exclude padding tokens from the Transformer’s computation.

What types of engagement actions does Phoenix support?

According to phoenix/xrex/data/recsys/constants.py, Phoenix supports configurable engagement types mapped through engagement_to_ids(metric_group). Common actions include clicks, likes, retweets, replies, and follows, each converted to a unique integer ID that receives its own embedding vector in the RecsysEmbedding layer.

Why does Phoenix use right-anchored positional encoding for history?

The right_anchored_rope_positions function ensures the most recent engagement always occupies the final position of the history window regardless of how many events occurred. This stable positioning allows the Transformer to learn absolute temporal patterns—such as "the most recent interaction predicts the next action"—without positional ambiguity caused by variable sequence lengths.

Can Phoenix combine historical engagement with candidate post features?

Yes. The RecsysModel.__call__ method constructs a unified sequence combining user prefix tokens, historical engagement tokens, and candidate post tokens. The Transformer processes this entire concatenated sequence, enabling attention mechanisms to directly compare historical behavior against candidate characteristics for relevance scoring.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}