the-algorithm
Source code for the X Recommendation Algorithm
Learn how Twitter scales GraphJet services for millions of users. Discover strategies for handling massive interactions with a multi-segment power-law bipartite graph and Kafka pipelines.
How the Representation Scorer Handles Updates to SimClusters Embeddings and Their Impact on Downstream ScoringDiscover how the Representation Scorer updates SimClusters embeddings and impacts downstream scoring. Learn about its stateless architecture and 2-hour cache window for seamless integration.
How to Extend and Customize the EdgeCollector in Twitter's Recommendation AlgorithmLearn how to extend the EdgeCollector in Twitters recommendation algorithm. Inject custom logic into writer pipelines by implementing the addEdge method. Customize edge processing with ease.
Common Pitfalls When Integrating Custom Graph Building Logic Using Graph Common UtilitiesAvoid common pitfalls when integrating custom graph building logic with Graph Common utilities. Learn about configurations, error handling, and data integrity for successful implementation.
How Power-Law Degree Distribution Optimization Improves Graph Storage and Retrieval in Twitter's AlgorithmDiscover how Power-law degree distribution optimization in Twitter's algorithm boosts graph storage and retrieval for millions of requests per second with bounded heap usage.
SimClusters Embeddings in the Twitter Recommendation Algorithm: Architecture and Integration PatternsExplore SimClusters embeddings in Twitters recommendation algorithm. Learn about their architecture and integration patterns for ANN lookups in candidate generation and ranking.
How Twitter's Algorithm Manages Lifecycle and Stale Data Cleanup in In-Memory GraphsExplore how Twitter's algorithm manages in-memory graph lifecycle and stale data cleanup. Discover TTL caches, query-time filtering, and live validation, ensuring data accuracy.
FinagleStatsReceiverWrapper Metrics: Monitoring Graph Operations in Twitter's AlgorithmDiscover FinagleStatsReceiverWrapper metrics like poll, pollLatency, failure, and queryTweetDegree used in Twitter's algorithm for monitoring graph operations and performance. Optimize your GraphJet services today.
How UnifiedGraphWriter Handles Concurrent Writes and Data Conflicts in Twitter's AlgorithmDiscover how UnifiedGraphWriter prevents data conflicts with thread isolation and lock-free queues, ensuring safe, high-volume Kafka stream ingestion for Twitter's algorithm. Learn its concurrency strategy.
LeftIndexedPowerLawMultiSegmentBipartiteGraphBuilder: Data Structures and Algorithms ExplainedExplore the data structures and algorithms powering LeftIndexedPowerLawMultiSegmentBipartiteGraphBuilder. Discover efficient O(1) insertions and O(k) retrievals for massive bipartite graphs with bounded memory.
How to Debug Load Shedding Issues in the Recos DeciderEffectively debug load shedding issues in the Recos Decider by verifying flag keys, inspecting fractional values, checking overlay precedence, and adding targeted logs. Learn essential techniques for developers.
Edge Type Masks for Different Engagement Types in Graphs: Best Practices from Twitter's AlgorithmLearn best practices for edge type masks at Twitter. Encode engagement types like follows and retweets into integer masks for efficient graph storage and retrieval. Optimize your algorithm today.
Have a question about this repo?
These articles cover the highlights, but your codebase questions are specific. Give your agent direct access to the source. Share this with your agent to get started:
curl -s "https://instagit.com/install.md" 
Maintain an open-source project? Get it listed too →