# How Phoenix Processes Recent Engagement History for the Transformer Model

> Discover how Phoenix processes recent engagement history for Transformer models. Learn about structured sequences, embeddings, positional encodings, and contextualized user representations for efficient ranking.

- Repository: [SpaceXAI Org/x-algorithm](https://github.com/xai-org/x-algorithm)
- Tags: deep-dive
- Published: 2026-09-11

---

**Phoenix processes recent engagement history by loading raw interaction events into structured sequences, embedding post and author hashes alongside action type tokens, applying right-anchored positional encodings, and feeding the masked sequence through a standard multi-head Transformer to generate contextualized user representations for ranking.**

The `xai-org/x-algorithm` repository implements Phoenix as a recommendation system that treats recent user interactions as first-class input tokens. Unlike systems that rely solely on aggregated user profiles, Phoenix processes recent engagement history for the transformer model as a temporally ordered sequence, enabling the architecture to learn patterns like "a recent like predicts the next click" or "view-to-like sequences boost relevance."

## Loading and Encoding Historical Engagement Data

Phoenix begins by reading recent engagement events from Parquet files or Kafka streams through `PhoenixDataset` (or `PhoenixKafkaDataset`) defined in [`phoenix/xrex/data/parquet_recsys.py`](https://github.com/xai-org/x-algorithm/blob/main/phoenix/xrex/data/parquet_recsys.py). Each user’s interaction history is stored in a `history_seq` dictionary containing three critical components:

- **`post_hashes`**: Identifiers for the posts the user engaged with
- **`auth_hashes`**: Identifiers for the authors of those posts  
- **`actions`**: One-hot vectors representing the engagement type (click, like, retweet, etc.)

The system maps engagement types to integer token IDs using `engagement_to_ids(metric_group)` from [`phoenix/xrex/data/recsys/constants.py`](https://github.com/xai-org/x-algorithm/blob/main/phoenix/xrex/data/recsys/constants.py). This mapping converts raw behavioral signals—such as likes or replies—into discrete tokens that the model processes like any other input symbol.

```python
from xrex.data.parquet_recsys import PhoenixDataset

# Load dataset with recent engagement history

dataset = PhoenixDataset(
    parquet_path="/data/training.parquet",
    history_seq_len=64,          # Retain up to 64 recent events

    metric_group="default",
)

batch = dataset.sample_batch(batch_size=32)

# batch["history_seq"] contains post_hashes, auth_hashes, and actions

```

## Building Token Embeddings with Right-Anchored Positioning

Inside `RecsysModel.__call__` ([`phoenix/xrex/models/recsys_model.py`](https://github.com/xai-org/x-algorithm/blob/main/phoenix/xrex/models/recsys_model.py)), the raw history tensors undergo padding and truncation to a fixed `history_seq_len` (e.g., 64 tokens). The system constructs embeddings through two parallel pathways:

**Hash Embeddings**: Post and author hashes pass through `RecsysEmbedding`, which implements hash table lookups to retrieve dense vector representations.

**Action Type Embeddings**: The integer action IDs (from `engagement_to_ids`) receive learned **type embeddings** through a dedicated embedding matrix, allowing the model to distinguish between different interaction semantics.

**Positional Encoding**: Phoenix applies `right_anchored_rope_positions` (lines 64-80 in the model implementation) to compute Rotary Position Embeddings (RoPE) that anchor the most recent events at the end of the history window. This ensures the Transformer always sees the newest interactions in consistent positional slots regardless of sequence length variation.

```python
from xrex.models.recsys_model import right_anchored_rope_positions

# Compute right-aligned positions for history tokens

positions = right_anchored_rope_positions(
    padding_mask=batch["padding_mask"],
    history_seq_len=64,
    num_user_prefix_tokens=4,    # Account for user profile prefix tokens

)

# positions shape: (batch_size, total_seq_len, 3)

```

## Integrating History into the Transformer Forward Pass

The `RecsysModel` stacks three token groups into a single input sequence consumed by the Transformer:

1. **User-prefix tokens**: Static user profile information
2. **History tokens**: The embedded recent engagement sequence
3. **Candidate tokens**: Posts under consideration for ranking

An attention mask marks valid history positions while ignoring padded slots. The system uses `packed_per_user_history_event_count` to determine exactly how many history slots contain actual data versus padding, ensuring the Transformer attends only to real interactions.

The standard `Transformer` implementation ([`phoenix/xrex/models/transformer.py`](https://github.com/xai-org/x-algorithm/blob/main/phoenix/xrex/models/transformer.py)) processes this combined sequence through multi-head self-attention and feed-forward layers. By attending across the full context—including recent engagements—the model generates contextualized representations where historical behavior directly influences the current ranking scores.

```python
from xrex.models.recsys_model import RecsysModel

model_cfg = {
    "history_seq_len": 64,
    "num_user_prefix_tokens": 4,
    # ... additional config

}

model = RecsysModel(model_cfg)

# Forward pass automatically embeds history and runs Transformer

logits, hidden_states = model(batch)

# hidden_states contain contextualized representations influenced by history

```

## Optional Feature Engineering for Engagement Patterns

Beyond raw token sequences, Phoenix supports additional feature preprocessing through `build_feature_prep_inputs` in [`phoenix/xrex/models/recsys_feature_prep.py`](https://github.com/xai-org/x-algorithm/blob/main/phoenix/xrex/models/recsys_feature_prep.py) (line 618). This module engineers structured features such as engagement-count buckets and temporal aggregation statistics, concatenating them to the token embeddings before they reach the Transformer layers. This hybrid approach combines the sequence modeling power of attention mechanisms with explicit feature engineering for specific behavioral patterns.

## Summary

- **PhoenixDataset** loads recent engagements from Parquet/Kafka into structured `history_seq` dictionaries containing post hashes, author hashes, and action vectors.
- **engagement_to_ids** maps interaction types (likes, clicks, retweets) to integer tokens that receive dedicated type embeddings alongside hash-based content embeddings.
- **right_anchored_rope_positions** ensures temporal ordering by fixing the most recent events at the end of the history window using right-aligned positional encodings.
- The **Transformer** consumes a concatenated sequence of user prefix, history, and candidate tokens, with attention masks filtering padded history slots via `packed_per_user_history_event_count`.
- Optional **feature prep** layers add engineered engagement statistics to complement the raw sequence representation.

## Frequently Asked Questions

### How does Phoenix handle variable-length engagement histories?

Phoenix pads or truncates all history sequences to a fixed `history_seq_len` (typically 64 events) during batching in `PhoenixDataset`. The `packed_per_user_history_event_count` tensor tracks the actual number of valid events per user, allowing the attention mask to exclude padding tokens from the Transformer’s computation.

### What types of engagement actions does Phoenix support?

According to [`phoenix/xrex/data/recsys/constants.py`](https://github.com/xai-org/x-algorithm/blob/main/phoenix/xrex/data/recsys/constants.py), Phoenix supports configurable engagement types mapped through `engagement_to_ids(metric_group)`. Common actions include clicks, likes, retweets, replies, and follows, each converted to a unique integer ID that receives its own embedding vector in the `RecsysEmbedding` layer.

### Why does Phoenix use right-anchored positional encoding for history?

The `right_anchored_rope_positions` function ensures the most recent engagement always occupies the final position of the history window regardless of how many events occurred. This stable positioning allows the Transformer to learn absolute temporal patterns—such as "the most recent interaction predicts the next action"—without positional ambiguity caused by variable sequence lengths.

### Can Phoenix combine historical engagement with candidate post features?

Yes. The `RecsysModel.__call__` method constructs a unified sequence combining user prefix tokens, historical engagement tokens, and candidate post tokens. The Transformer processes this entire concatenated sequence, enabling attention mechanisms to directly compare historical behavior against candidate characteristics for relevance scoring.