# How the Candidate Pool Is Merged and Ranked in X-Algorithm

> Discover how X-Algorithm merges candidate pools by concatenating vectors and ranks them using a configurable Selector trait. Learn about score sorting, truncation, and diversity reranking.

- Repository: [SpaceXAI Org/x-algorithm](https://github.com/xai-org/x-algorithm)
- Tags: internals
- Published: 2026-09-12

---

**X-Algorithm merges candidate pools by concatenating vectors from heterogeneous sources in the fetch stage, then ranks them using a configurable Selector trait that sorts by descending scores and applies optional truncation before final DPP-based diversity reranking.**

The xai-org/x-algorithm repository implements a modular recommendation pipeline in Rust that demonstrates exactly how the candidate pool is merged and ranked across distinct retrieval and scoring phases. The architecture separates data ingestion from ranking logic through strict trait boundaries, enabling pluggable sources and selection strategies.

## The Four-Stage Pipeline Architecture

The candidate pipeline processes items through four logical stages defined in [`candidate-pipeline/candidate_pipeline.rs`](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/candidate_pipeline.rs):

1. **Fetch** - Raw candidates are pulled from external services via implementations of the `Source` trait defined in [`candidate-pipeline/source.rs`](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/source.rs).
2. **Hydrate** - The `Hydrator` enriches raw items with embeddings, safety labels, and metadata in [`candidate-pipeline/hydrator.rs`](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/hydrator.rs).
3. **Filter** - Business rule validation drops invalid candidates in [`candidate-pipeline/filter.rs`](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/filter.rs).
4. **Score & Select** - The `Selector` trait handles final ranking and truncation in [`candidate-pipeline/selector.rs`](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/selector.rs).

## Merging Candidates from Multiple Sources

The merge operation occurs during the fetch phase when `fetch_candidates` aggregates results from all registered sources into a single vector. In [`candidate-pipeline/candidate_pipeline.rs`](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/candidate_pipeline.rs), the pipeline controller iterates over source implementations and extends a master vector with each returned batch:

```rust
// candidate-pipeline/candidate_pipeline.rs (lines 119-124)
let candidates = self.fetch_candidates(&hydrated_query).await; // Vec<C> from every source
let hydrated_candidates = self.hydrate(&hydrated_query, candidates).await;
let (kept_candidates, mut filtered_candidates) = self.filter(&hydrated_query, hydrated_candidates.clone());

```

Each source produces a `Vec<C>` of candidates, and the pipeline concatenates these vectors through the `extend` operation within `fetch_candidates`. This merging strategy preserves ordering from individual sources while creating a unified pool for downstream processing.

## Ranking and Selection Strategy

After merging and filtering, the pipeline delegates ranking to the `Selector` trait implemented in [`candidate-pipeline/selector.rs`](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/selector.rs). The selector defines two critical methods: `score` for computing candidate relevance and `sort` for ordering the final list.

### Scoring and Sorting

The `sort` method implements a descending-order sort based on the `score` function. Concrete selectors implement `score` to return an `f64` relevance metric, while the generic `sort` routine handles the ordering logic:

```rust
// candidate-pipeline/selector.rs (lines 66-73)
fn sort(&self, candidates: Vec<C>) -> Vec<C> {
    let mut sorted = candidates;
    sorted.sort_by(|a, b| {
        self.score(b)
            .partial_cmp(&self.score(a))
            .unwrap_or(std::cmp::Ordering::Equal)
    });
    sorted
}

```

The sort uses `partial_cmp` to handle floating-point scores safely, falling back to `Equal` when comparisons fail. This ensures deterministic ordering even with NaN values or equal scores.

### Truncation and Result Splitting

The `select` method applies the final ranking cutoff based on the `size()` configuration. When a limit is specified, the method splits the sorted vector at the threshold, returning both selected and non-selected groups for metrics tracking:

```rust
// candidate-pipeline/selector.rs (lines 48-56)
if let Some(limit) = self.size() {
    let non_selected = sorted.split_off(limit.min(sorted.len()));
    SelectResult { selected: sorted, non_selected }
} else {
    SelectResult { selected: sorted, non_selected: vec![] }
}

```

The `split_off` operation truncates the merged candidate pool to the configured limit while preserving the remainder for fallback logic or analytics.

## Post-Selection Diversity Ranking

Following the initial selection phase, the pipeline optionally invokes the VM-Ranker service for diversity optimization. Located in [`vm-ranker/ranker_service.rs`](https://github.com/xai-org/x-algorithm/blob/main/vm-ranker/ranker_service.rs), this component employs a Determinantal Point Process (DPP) model to rerank the selected candidates and enforce content diversity.

The `rank_post_selection` method takes the output from the selector and applies DPP-based scoring before final serving. This occurs after truncation, meaning diversity ranking operates on the already-scored subset rather than the full merged pool.

## Summary

- **Vector concatenation** in `fetch_candidates` merges heterogeneous source outputs into a unified `Vec<C>` before hydration.
- **Trait-based scoring** allows custom selectors to implement domain-specific relevance metrics while reusing generic sorting logic.
- **Descending sort** with floating-point safety ensures deterministic ranking across partial comparisons.
- **Configurable truncation** via `split_off` separates top-ranked candidates from residual items for analytics.
- **DPP reranking** in the VM-Ranker layer provides optional diversity optimization after initial selection.

## Frequently Asked Questions

### How does X-Algorithm combine candidates from different sources?

The pipeline merges candidates by extending a master vector with results from each registered `Source` implementation. In [`candidate-pipeline/candidate_pipeline.rs`](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/candidate_pipeline.rs), the `fetch_candidates` method aggregates all source outputs into a single `Vec<C>` before passing the unified pool to the hydration and filtering stages.

### What determines the final order of candidates in the ranked list?

Final ordering derives from the `Selector` trait's `score` method implementation, which returns floating-point relevance values. The generic `sort` method in [`candidate-pipeline/selector.rs`](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/selector.rs) arranges candidates in descending score order using `partial_cmp`, with concrete selectors specializing the scoring logic for specific content types.

### Where is the candidate limit enforced in the pipeline?

The limit enforces at the selection stage via the `size()` method in [`candidate-pipeline/selector.rs`](https://github.com/xai-org/x-algorithm/blob/main/candidate-pipeline/selector.rs). When `size()` returns `Some(limit)`, the `select` method calls `split_off` on the sorted vector to separate the top N candidates from the remainder before returning the `SelectResult` struct.

### How does the DPP ranker affect the final output?

The Determinantal Point Process (DPP) ranker operates post-selection in [`vm-ranker/ranker_service.rs`](https://github.com/xai-org/x-algorithm/blob/main/vm-ranker/ranker_service.rs) through the `rank_post_selection` method. It receives the already-truncated candidate list and applies diversity-based reranking, ensuring the final served content balances relevance with topical variety.