# the-algorithm | X (fka Twitter) | Knowledge Base | Instagit

Source code for the X Recommendation Algorithm

GitHub Stars: 72.8k

Repository: https://github.com/twitter/the-algorithm

---

## Articles

### [Scaling GraphJet-Based Services to Handle Millions of Users and Interactions](/twitter/the-algorithm/scaling-graphjet-services-millions-users-interactions)

Learn how Twitter scales GraphJet services for millions of users. Discover strategies for handling massive interactions with a multi-segment power-law bipartite graph and Kafka pipelines.

- Tags: performance
- Published: 2026-03-03

### [How the Representation Scorer Handles Updates to SimClusters Embeddings and Their Impact on Downstream Scoring](/twitter/the-algorithm/how-representation-scorer-handles-embedding-updates-downstream-impact)

Discover how the Representation Scorer updates SimClusters embeddings and impacts downstream scoring. Learn about its stateless architecture and 2-hour cache window for seamless integration.

- Tags: deep-dive
- Published: 2026-03-03

### [How to Extend and Customize the EdgeCollector in Twitter's Recommendation Algorithm](/twitter/the-algorithm/how-extend-customize-edgecollector-specific-needs)

Learn how to extend the EdgeCollector in Twitters recommendation algorithm. Inject custom logic into writer pipelines by implementing the addEdge method. Customize edge processing with ease.

- Tags: how-to-guide
- Published: 2026-03-03

### [Common Pitfalls When Integrating Custom Graph Building Logic Using Graph Common Utilities](/twitter/the-algorithm/common-pitfalls-edge-cases-integrating-custom-graph-building-logic)

Avoid common pitfalls when integrating custom graph building logic with Graph Common utilities. Learn about configurations, error handling, and data integrity for successful implementation.

- Tags: best-practices
- Published: 2026-03-03

### [How Power-Law Degree Distribution Optimization Improves Graph Storage and Retrieval in Twitter's Algorithm](/twitter/the-algorithm/power-law-degree-distribution-impact-graph-storage-retrieval)

Discover how Power-law degree distribution optimization in Twitter's algorithm boosts graph storage and retrieval for millions of requests per second with bounded heap usage.

- Tags: performance
- Published: 2026-03-03

### [SimClusters Embeddings in the Twitter Recommendation Algorithm: Architecture and Integration Patterns](/twitter/the-algorithm/architectural-role-integration-simclusters-embeddings-recommendation-framework)

Explore SimClusters embeddings in Twitters recommendation algorithm. Learn about their architecture and integration patterns for ANN lookups in candidate generation and ranking.

- Tags: architecture
- Published: 2026-03-03

### [How Twitter's Algorithm Manages Lifecycle and Stale Data Cleanup in In-Memory Graphs](/twitter/the-algorithm/how-system-manages-stale-data-cleanup-in-memory-graphs)

Explore how Twitter's algorithm manages in-memory graph lifecycle and stale data cleanup. Discover TTL caches, query-time filtering, and live validation, ensuring data accuracy.

- Tags: internals
- Published: 2026-03-03

### [FinagleStatsReceiverWrapper Metrics: Monitoring Graph Operations in Twitter's Algorithm](/twitter/the-algorithm/metrics-finagle-stats-receiver-wrapper-monitoring-graph-operations)

Discover FinagleStatsReceiverWrapper metrics like poll, pollLatency, failure, and queryTweetDegree used in Twitter's algorithm for monitoring graph operations and performance. Optimize your GraphJet services today.

- Tags: performance
- Published: 2026-03-03

### [How UnifiedGraphWriter Handles Concurrent Writes and Data Conflicts in Twitter's Algorithm](/twitter/the-algorithm/how-unifiedgraphwriter-handles-concurrent-writes-data-conflicts)

Discover how UnifiedGraphWriter prevents data conflicts with thread isolation and lock-free queues, ensuring safe, high-volume Kafka stream ingestion for Twitter's algorithm. Learn its concurrency strategy.

- Tags: internals
- Published: 2026-03-03

### [LeftIndexedPowerLawMultiSegmentBipartiteGraphBuilder: Data Structures and Algorithms Explained](/twitter/the-algorithm/data-structures-algorithms-leftindexedpowerlawmultisegmentbipartitegraphbuilder)

Explore the data structures and algorithms powering LeftIndexedPowerLawMultiSegmentBipartiteGraphBuilder. Discover efficient O(1) insertions and O(k) retrievals for massive bipartite graphs with bounded memory.

- Tags: deep-dive
- Published: 2026-03-03

### [How to Debug Load Shedding Issues in the Recos Decider](/twitter/the-algorithm/how-debug-load-shedding-issues-recos-decider)

Effectively debug load shedding issues in the Recos Decider by verifying flag keys, inspecting fractional values, checking overlay precedence, and adding targeted logs. Learn essential techniques for developers.

- Tags: debugging-guide
- Published: 2026-03-03

### [Edge Type Masks for Different Engagement Types in Graphs: Best Practices from Twitter's Algorithm](/twitter/the-algorithm/best-practices-edge-type-masks-engagement-types-graphs)

Learn best practices for edge type masks at Twitter. Encode engagement types like follows and retweets into integer masks for efficient graph storage and retrieval. Optimize your algorithm today.

- Tags: best-practices
- Published: 2026-03-03

### [How Kafka Streams Are Integrated and Utilized for State Persistence in Twitter's Recommendation Algorithm Services](/twitter/the-algorithm/how-kafka-streams-integrated-state-persistence-recommendation-algorithm)

Discover how Twitter's recommendation algorithm integrates Kafka Streams for state persistence. Learn about stateful processing, RocksDB, and changelog topics in this technical deep dive.

- Tags: internals
- Published: 2026-03-03

### [Performance Implications of Using GraphJet for In-Memory Graph Storage at Twitter Scale](/twitter/the-algorithm/performance-implications-graphjet-in-memory-graph-storage)

Discover GraphJet's performance for in-memory graph storage at Twitter scale. Achieve sub-10ms latency and 10k requests per second per instance with efficient design.

- Tags: performance
- Published: 2026-03-03

### [How the Representation Scorer (RSX) Manages and Serves SimClusters Embeddings for Scoring](/twitter/the-algorithm/how-representation-scorer-manages-serves-simclusters-embeddings)

Learn how the Representation Scorer RSX system manages SimClusters embeddings and serves similarity scores through on-the-fly pair-embedding computations in Twitters algorithm.

- Tags: how-to-guide
- Published: 2026-03-03

### [How the Interaction Graph Tracks User Interactions for Recommendation Feedback Loops](/twitter/the-algorithm/role-interaction-graph-tracking-user-interactions-recommendation-feedback)

Discover how Twitter's Interaction Graph tracks user interactions to build powerful recommendation feedback loops using billions of real-time signals. Learn its core role in the algorithm.

- Tags: deep-dive
- Published: 2026-03-03

### [How the Hose Common Module Handles Efficient Edge Processing and Writing in Twitter's Algorithm](/twitter/the-algorithm/hose-common-edge-processing-writing-unifiedgraphwriter)

Discover how Twitter's Hose Common module efficiently processes millions of Kafka events for edge writing using UnifiedGraphWriter. Learn about its decoupled pipeline, batching, and back-pressure handling.

- Tags: internals
- Published: 2026-03-03

### [Core Utilities in Graph Common for Building GraphJet-Based Graphs with Power-Law Degree Distributions](/twitter/the-algorithm/graph-common-utilities-building-graphjet-graphs-power-law)

Discover Graph Common utilities like MultiSegmentPowerLawBipartiteGraphBuilder for building power-law distributed graphs with GraphJet. Minimize boilerplate and gain observability.

- Tags: deep-dive
- Published: 2026-03-03

### [How the Recos Decider Component Is Configured for Dynamic Feature Flagging and Effective Load Shedding](/twitter/the-algorithm/how-recos-decider-configured-feature-flagging-load-shedding)

Learn how to configure the Recos Decider for dynamic feature flagging and load shedding using its dual-layer YAML system. Enable runtime changes without deployments.

- Tags: internals
- Published: 2026-03-03

### [Architectural Considerations for the User Video Graph (UVG) in Delivering Video Recommendations](/twitter/the-algorithm/architectural-considerations-user-video-graph-for-video-recommendations)

Learn about the architectural considerations for the User Video Graph UVG, a high-performance graph service powering sub-millisecond video recommendations with GraphJet and Finagle APIs.

- Tags: architecture
- Published: 2026-03-03

### [How the User User Graph (UUG) Leverages the Follow Graph for Recommendations](/twitter/the-algorithm/how-user-user-graph-leverages-follow-graph-for-user-recommendations)

Discover how the User User Graph (UUG) uses your recent follows and Twitter's GraphJet engine to deliver personalized and effective recommendations. Enhance your user experience today.

- Tags: deep-dive
- Published: 2026-03-03

### [How the User Tweet Entity Graph (UTEG) Generates "XXX Liked" Out-of-Network Recommendations](/twitter/the-algorithm/uteg-implementation-strategy-liked-out-of-network-tweets)

Discover how UTEG generates 'XXX Liked' out-of-network recommendations by querying a real-time user-tweet graph and applying unique filters for tailored content.

- Tags: architecture
- Published: 2026-03-03

### [How the User Tweet Graph (UTG) Uses Collaborative Filtering and Random Walks for Advanced Tweet Recommendations](/twitter/the-algorithm/how-does-user-tweet-graph-utilize-collaborative-filtering-random-walks-for-advanced-tweet-recommendations)

Discover how Twitter's User Tweet Graph (UTG) uses collaborative filtering and random walks to deliver advanced tweet recommendations by analyzing user engagement.

- Tags: deep-dive
- Published: 2026-03-03