# The Seven Stages of the Candidate Pipeline in x-algorithm: A Deep Dive into the For You Feed

> Explore the seven stages of the x-algorithm candidate pipeline. Understand how your For You feed is generated from Query Hydration to Post-Selection Filters. Optimize your recommendations today.

- Repository: [SpaceXAI Org/x-algorithm](https://github.com/xai-org/x-algorithm)
- Tags: deep-dive
- Published: 2026-09-12

---

**The x-algorithm candidate pipeline processes feed requests through seven sequential stages—Query Hydration, Candidate Sources, Candidate Hydration, Pre-Scoring Filters, Scoring, Selection, and Post-Selection Filters—to generate the For You feed.**

The x-algorithm repository powers the recommendation logic behind X's For You feed. Understanding the seven stages of the candidate pipeline in x-algorithm reveals how billions of posts are filtered, scored, and ranked in milliseconds to deliver personalized content. Each stage operates as a composable unit that can be toggled via configuration, creating a flexible framework for feed generation.

## Overview of the Seven Pipeline Stages

The pipeline follows a fixed sequence defined in the `PipelineStage` enum located in [[`candidate-pipeline/candidate_pipeline.rs`](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/candidate_pipeline.rs)](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/candidate_pipeline.rs) (lines 24-35). This architecture orchestrates each request from initial context gathering through final filtering before returning results to the client.

According to the repository's README diagram and source code, the seven stages execute in the following order:

1. **Query Hydration** – Context enrichment
2. **Candidate Sources** – Data retrieval
3. **Candidate Hydration** – Payload completion
4. **Pre-Scoring Filters** – Quality pruning
5. **Scoring** – Relevance prediction
6. **Selection** – Top-K ranking
7. **Post-Selection Filters** – Final visibility filtering

## Stage 1: Query Hydration

**Query Hydration** enriches the incoming viewer request with additional context required by downstream stages. This stage pulls recent engagements, follows, blocks, and muted keywords to build a comprehensive viewer profile.

The implementation resides in [[`candidate-pipeline/candidate_pipeline.rs`](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/candidate_pipeline.rs)](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/candidate_pipeline.rs#L22-L30), where query hydrators attach metadata to the initial request before any candidate fetching occurs.

## Stage 2: Candidate Sources

The **Candidate Sources** stage pulls raw candidate posts in parallel from two primary retrieval systems:

- **Thunder** – In-network sources (accounts the user follows)
- **Phoenix / SimClusters** – Out-of-network sources (algorithmic recommendations)

This stage is implemented in [[`home-mixer/candidate_pipeline/phoenix_candidate_pipeline.rs`](https://github.com/xai-org/x-algorithm/blob/main/home-mixer/candidate_pipeline/phoenix_candidate_pipeline.rs)](https://github.com/xai-org/x-algorithm/blob/main/home-mixer/candidate_pipeline/phoenix_candidate_pipeline.rs), which orchestrates parallel fetching from multiple source adapters.

## Stage 3: Candidate Hydration

**Candidate Hydration** adds missing payload data to each candidate post. This includes attaching text content, media URLs, author details, quoted posts, language detection, engagement counts, and subscription status.

The hydrator logic lives in [[`candidate-pipeline/hydrator.rs`](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/hydrator.rs)](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/hydrator.rs), ensuring all candidates contain complete metadata before entering the filtering and scoring phases.

## Stage 4: Pre-Scoring Filters

Before any scoring calculations occur, **Pre-Scoring Filters** apply a cascade of filters to prune low-quality or disallowed candidates. This stage removes duplicates, applies age filters, excludes self-tweets, and filters out blocked accounts or muted keywords.

The filter implementations are located in [`home-mixer/filters/`](https://github.com/xai-org/x-algorithm/tree/main/home-mixer/filters), providing early termination for candidates that violate quality or safety constraints.

## Stage 5: Scoring

The **Scoring** stage runs one or more scorers that predict probabilities for each possible viewer action. These predictions are combined into a final relevance score using implementations such as:

- `PhoenixScorer`
- `RankingScorer` 
- `VMRanker`

Source code for these scoring algorithms is found in [`home-mixer/scorers/`](https://github.com/xai-org/x-algorithm/tree/main/home-mixer/scorers), where machine learning models evaluate candidate quality and predicted engagement.

## Stage 6: Selection

**Selection** orders candidates by their final composite score and applies a selection strategy—typically keeping the top-K highest-scoring posts. This stage determines the actual content that will appear in the user's feed and establishes the final ranking order.

The primary implementation uses [[`home-mixer/selectors/top_k_score_selector.rs`](https://github.com/xai-org/x-algorithm/blob/main/home-mixer/selectors/top_k_score_selector.rs)](https://github.com/xai-org/x-algorithm/blob/main/home-mixer/selectors/top_k_score_selector.rs) to perform the cutoff-based selection.

## Stage 7: Post-Selection Filters

After the ranking order is fixed, **Post-Selection Filters** run additional visibility checks that depend on the final selection context. These include:

- `VFFilter` (Visibility Filtering)
- `AncillaryVFFilter`
- `DedupConversationFilter`

These filters reside in [`home-mixer/filters/`](https://github.com/xai-org/x-algorithm/tree/main/home-mixer/filters) and handle edge cases like conversation deduplication that require knowledge of the final candidate set.

## Implementation Architecture

Concrete pipelines implement the `CandidatePipeline` trait to wire these seven stages together. The `components()` method (lines 64-112 in [[`candidate_pipeline.rs`](https://github.com/xai-org/x-algorithm/blob/main/candidate_pipeline.rs)](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/candidate_pipeline.rs)) exposes the stage ordering for debugging and monitoring purposes.

The following Rust example demonstrates how a `PhoenixCandidatePipeline` configures the seven stages:

```rust
use candidate_pipeline::{CandidatePipeline, PipelineStage};

/// A simplified pipeline that uses the default Phoenix components.
pub struct SimplePhoenixPipeline;

impl<Q, C> CandidatePipeline<Q, C> for SimplePhoenixPipeline
where
    Q: PipelineQuery,
    C: PipelineCandidate,
{
    fn query_hydrators(&self) -> &[Box<dyn QueryHydrator<Q>>] { self.phoenix_query_hydrators() }
    fn sources(&self) -> &[Box<dyn Source<Q, C>>] { self.phoenix_sources() }
    fn hydrators(&self) -> &[Box<dyn Hydrator<Q, C>>] { self.phoenix_hydrators() }
    fn filters(&self) -> &[Box<dyn Filter<Q, C>>] { self.phoenix_pre_filters() }
    fn scorers(&self) -> &[Box<dyn Scorer<Q, C>>] { self.phoenix_scorers() }
    fn selector(&self) -> &dyn Selector<Q, C> { &self.top_k_selector }
    fn post_selection_hydrators(&self) -> &[Box<dyn Hydrator<Q, C>>] { &[] }
    fn post_selection_filters(&self) -> &[Box<dyn Filter<Q, C>>] {
        self.phoenix_post_filters()
    }
    fn side_effects(&self) -> std::sync::Arc<Vec<Box<dyn SideEffect<Q, C>>>> {
        std::sync::Arc::new(vec![])
    }
    fn result_size(&self) -> usize { 50 }
}

```

## Execution Flow Example

To execute the complete seven-stage pipeline:

```rust
let query = MyQuery::default();
let pipeline = SimplePhoenixPipeline;
let result = pipeline.execute(query).await;
println!("{} selected candidates", result.selected_candidates.len());

```

The pipeline orchestrates each stage sequentially, records metrics at each step, and returns the final selected candidates to the client after completing all seven stages.

## Summary

- **Query Hydration** enriches viewer context before candidate retrieval
- **Candidate Sources** fetches posts in parallel from Thunder and Phoenix/SimClusters
- **Candidate Hydration** attaches complete metadata to each post
- **Pre-Scoring Filters** removes low-quality candidates before scoring
- **Scoring** calculates relevance using machine learning models
- **Selection** ranks candidates and selects the top-K for display
- **Post-Selection Filters** applies final visibility and deduplication logic

## Frequently Asked Questions

### What is the purpose of the PipelineStage enum in x-algorithm?

The `PipelineStage` enum in [[`candidate-pipeline/candidate_pipeline.rs`](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/candidate_pipeline.rs)](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/candidate_pipeline.rs) (lines 24-35) defines the seven fixed stages of the candidate pipeline as a type-safe abstraction. It allows the system to track which stage is currently executing for metrics, debugging, and configuration toggling, ensuring each request flows through the identical sequence of processing steps.

### How does the candidate pipeline handle in-network vs out-of-network sources?

During **Stage 2: Candidate Sources**, the pipeline queries both sources in parallel. **Thunder** provides in-network candidates from accounts the user follows, while **Phoenix** and **SimClusters** generate out-of-network recommendations algorithmically. The [[`phoenix_candidate_pipeline.rs`](https://github.com/xai-org/x-algorithm/blob/main/phoenix_candidate_pipeline.rs)](https://github.com/xai-org/x-algorithm/blob/main/home-mixer/candidate_pipeline/phoenix_candidate_pipeline.rs) implementation merges these streams before passing unified candidates to the hydration stage.

### What is the difference between pre-scoring and post-selection filters?

**Pre-scoring filters** (Stage 4) operate on individual candidates before scoring occurs, removing obvious violations like blocked accounts or muted keywords to save computation. **Post-selection filters** (Stage 7) operate on the final ranked list, handling visibility filtering decisions and conversation deduplication that require context about the complete selected set. The former optimizes performance; the latter ensures final feed quality.

### Where is the final candidate ranking determined in the pipeline?

The final ranking is determined in **Stage 6: Selection**, specifically within [[`home-mixer/selectors/top_k_score_selector.rs`](https://github.com/xai-org/x-algorithm/blob/main/home-mixer/selectors/top_k_score_selector.rs)](https://github.com/xai-org/x-algorithm/blob/main/home-mixer/selectors/top_k_score_selector.rs). This stage orders candidates by their composite scores from Stage 5 and truncates the list to the configured `result_size` (typically 50 candidates), establishing the fixed order that post-selection filters may prune but cannot reorder.