How the Inventory Holdout Filter Works in X's Home-Mixer Algorithm

The inventory holdout filter is a deterministic, per-viewer random-sampling mechanism that removes a configurable percentage of posts from candidate lists before ranking by comparing a 64-bit hash bucket against per-kind percentage thresholds.

The inventory holdout filter in the xai-org/x-algorithm repository controls content exposure in X's Home-Mixer recommendation pipeline. This Rust-based component ensures that specific viewer-post pairs consistently yield the same holdout decision while allowing different users to see different subsets of content.

When the Filter Is Enabled

The filter only activates when the master feature switch rust_home_mixer_enable_inventory_holdout evaluates to true and at least one per-kind holdout percentage exceeds zero. According to the source code in home-mixer/filters/inventory_holdout_filter.rs (lines 53-57), the system validates these feature-switch parameters before processing any candidates. If disabled, the filter bypasses all candidates without modification.

Post Kind Classification

Before applying holdout logic, the filter classifies each PostCandidate into a PostKind variant. As implemented in lines 10-24 of inventory_holdout_filter.rs, the classification inspects two optional fields:

• Original: Posts with neither retweeted_tweet_id nor in_reply_to_tweet_id
• Reply: Posts containing in_reply_to_tweet_id
• Retweet: Posts containing retweeted_tweet_id

This classification determines which percentage threshold applies to each candidate during the holdout decision.

Deterministic Bucket Calculation

The core mechanism relies on a deterministic 64-bit hash function that maps (post_id, viewer_id) pairs to buckets 0-99. In home-mixer/filters/inventory_holdout_filter.rs (lines 31-44), the holdout_bucket function performs the following operations:

1. Mixes the IDs with two large cryptographic constants
2. Scrambles the result through two multiplications and a final XOR operation
3. Takes modulo 100 to produce a consistent bucket value

Because this calculation uses pure arithmetic without random seeds or external state, identical ID pairs always generate identical buckets across repeated requests and server instances.

Holdout Decision and Filtering

The is_held_out method (lines 46-48) compares the calculated bucket against the configured percentage for the candidate's kind. A candidate is removed only if its bucket is strictly lower than the threshold (e.g., bucket 19 passes at 20%, bucket 20 does not).

During the main filter execution (lines 64-78), the system:

• Retrieves per-kind percentages from the ScoredPostsQuery parameters, capping values at 100
• Partitions the candidate vector into removed and kept collections based on the holdout decision
• Returns a FilterResult containing both sets for pipeline logging and analysis

Verified Behavioral Properties

The test suite in inventory_holdout_filter.rs validates several critical guarantees:

• Determinism: Identical (post_id, viewer_id) pairs always map to the same bucket (test lines 55-59)
• Viewer specificity: Different viewer_id values produce different holdout sets for the same content (test lines 62-71)
• Statistical accuracy: A 1% configuration produces observed holdout rates between 0.7-1.3% over 200,000 samples (test lines 75-85)
• Boundary conditions: 0% holds out nothing; 100% holds out everything (test lines 89-99)
• Retweet consistency: Retweets share holdout decisions with their original tweets to prevent partial visibility (test lines 101-113)

Implementation Example

The following example demonstrates configuring feature switches and applying the filter to a candidate list:

use xai_candidate_pipeline::filter::Filter;
use home_mixer::filters::inventory_holdout_filter::InventoryHoldoutFilter;
use home_mixer::models::query::ScoredPostsQuery;
use xai_feature_switches::{FeatureSwitches, Params, RecipientBuilder};

// Build feature‑switch parameters
let mut fs = FeatureSwitches::new(vec![]).unwrap()
    .match_recipient(&RecipientBuilder::new().build());
fs.override_fs("rust_home_mixer_enable_inventory_holdout".to_string(), "true");
fs.override_fs("rust_home_mixer_inventory_holdout_originals_percent".to_string(), "20");
fs.override_fs("rust_home_mixer_inventory_holdout_replies_percent".to_string(), "0");
fs.override_fs("rust_home_mixer_inventory_holdout_retweets_percent".to_string(), "0");
let params: Params = fs.into();

// Assemble a query for viewer 12345
let query = ScoredPostsQuery {
    user_id: 12345,
    params,
    ..Default::default()
};

// Example candidates
let candidates = vec![
    // original tweet
    PostCandidate { tweet_id: 1, ..Default::default() },
    // reply
    PostCandidate { tweet_id: 2, in_reply_to_tweet_id: Some(1), ..Default::default() },
    // retweet
    PostCandidate { tweet_id: 3, retweeted_tweet_id: Some(1), ..Default::default() },
];

// Apply the filter
let result = InventoryHoldoutFilter.filter(&query, candidates);

// `result.kept` holds the candidates that passed the holdout test,
// `result.removed` holds those that were dropped.
println!("kept: {}, removed: {}", result.kept.len(), result.removed.len());

To check a single holdout decision programmatically:

let post_id = 42u64;
let viewer_id = 12345u64;
let percent = 20u32; // 20 % holdout for this kind
let held_out = InventoryHoldoutFilter::is_held_out(post_id, viewer_id, percent);
println!("Post {} is held out for viewer {}: {}", post_id, viewer_id, held_out);

Integration with the Candidate Pipeline

The InventoryHoldoutFilter implements the Filter trait defined in xai_candidate_pipeline/filter.rs. It receives a ScoredPostsQuery (from home-mixer/models/query.rs) containing the viewer's user_id and feature-switch parameters, along with PostCandidate objects (from home-mixer/models/candidate.rs). The feature-switch definitions reside in home-mixer/params/param.rs, exposing configuration options for enabling holdouts and setting per-kind percentages.

Summary

• The inventory holdout filter requires explicit activation via rust_home_mixer_enable_inventory_holdout and at least one non-zero percentage parameter.
• Deterministic hashing ensures consistent treatment of (post, viewer) pairs using 64-bit arithmetic in the holdout_bucket function.
• PostKind classification separates originals, replies, and retweets for independent percentage configuration.
• Strictly lower bucket comparison determines removal, supporting 0-100% holdout ranges with precise boundary behavior.
• Retweet consistency ensures retweets inherit their original tweet's holdout status to maintain content integrity.

Frequently Asked Questions

What makes the inventory holdout filter deterministic?

The filter computes a 64-bit hash from the post_id and viewer_id using pure arithmetic operations (multiplications, XOR, and modulo) without random number generators. As implemented in home-mixer/filters/inventory_holdout_filter.rs lines 31-44, this ensures the same ID pair always yields identical hash buckets across repeated executions and server restarts.

How does the filter handle different post types?

The filter classifies candidates into Original, Reply, or Retweet kinds by inspecting retweeted_tweet_id and in_reply_to_tweet_id fields (lines 10-24). Each kind maintains an independent percentage threshold, allowing configurations like 20% holdout for originals while keeping replies at 0%.

What happens at 0% and 100% holdout thresholds?

At 0%, no candidates are held out because no bucket (0-99) can be strictly lower than zero. At 100%, all candidates are held out because every bucket (0-99) is strictly lower than 100. The test suite verifies these boundary conditions in lines 89-99 of the implementation file.

Why use per-viewer hashing instead of global random sampling?

Per-viewer hashing ensures that while different users see different subsets of held-out content (verified in test lines 62-71), each specific user-post relationship remains consistent. This prevents a user from seeing a post, refreshing the timeline, and having it disappear due to random chance, improving user experience while maintaining the statistical benefits of inventory reduction.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}