Performance Implications of Using GraphJet for In-Memory Graph Storage at Twitter Scale

GraphJet delivers sub-10ms latency and sustains over 10,000 requests per second per instance by combining power-law-aware bipartite graphs, pre-warmed runner pools, and primitive collections that minimize GC pressure.

GraphJet serves as the core in-memory graph engine powering Twitter’s real-time recommendation services, including the User-User Graph (UUG), User-Video Graph, and User-Tweet Entity Graph. Understanding the performance implications of using GraphJet for in-memory graph storage is critical for architects building high-throughput recommendation systems, as its design decisions directly impact latency, throughput, and resource predictability when serving billions of daily queries.

In-Memory Bipartite Architecture for O(1) Edge Lookup

GraphJet’s performance foundation rests on the NodeMetadataLeftIndexedPowerLawMultiSegmentBipartiteGraph class, which maintains the entire graph structure in RAM using a left-indexed, power-law-aware multi-segment design.

This architecture enables O(1) edge lookup and fast random access to high-degree nodes. In src/scala/com/twitter/recos/user_user_graph/RecommendUsersHandler.scala, the handler interacts with this structure to traverse millions of edges per second while keeping per-request latency in the low-millisecond range. The power-law segmentation specifically reduces memory pressure on celebrity nodes with millions of connections, preventing them from dominating heap allocation and causing tail latency spikes.

Throughput Optimization via Async Runner Pools

To eliminate per-request allocation overhead, RecommendUsersHandler maintains an AsyncQueue of pre-instantiated TopSecondDegreeByCountForUser objects (see lines 50-58 of the handler implementation). This pool eliminates the cost of constructing new runners for each incoming request.

By reusing these heavyweight traversal objects across thousands of concurrent requests, the service reduces GC churn and maintains CPU stability even when processing >10,000 RPS per instance on modest 2-4 core hardware. The queue acts as a buffer against traffic bursts, ensuring that graph mutations never block recommendation computation.

Memory Efficiency and GC-Friendly Data Structures

GraphJet aggressively minimizes garbage collection pressure through two primary mechanisms: primitive collections and bounded retention windows.

Primitive Collections: The engine relies on fastutil libraries, specifically Long2DoubleOpenHashMap and LongOpenHashSet, to store edge metadata and visited node sets. These structures avoid autoboxing of primitive long and double types, significantly reducing heap allocation rates during graph traversals—a critical optimization when processing millions of requests per minute.

Retention Windows: Services like UUG retain only the last seven days of user engagements, as documented in src/scala/com/twitter/recos/user_user_graph/README.md. This fixed window triggers periodic garbage collection of stale edges, keeping the memory footprint bounded and predictable regardless of sustained high write rates.

Operational Safeguards and Real-Time Observability

Production reliability relies on tight integration with Twitter’s Finagle ecosystem through FinagleStatsReceiverWrapper (src/scala/com/twitter/recos/graph_common/FinagleStatsReceiverWrapper.scala).

Time-Bounded Execution: The handler measures pollLatencyStat and enforces salsaRunnerConfig.timeoutSalsaRunner (lines 92-104), establishing a hard upper bound on request latency. If queue poll times exceed thresholds, the handler aborts immediately, preventing tail-latency cascades into downstream services.

Inline Filtering: Rather than materializing full result sets for post-processing, the handler constructs a ResultFilterChain containing SocialProofTypesFilter and RequestedSetFilter (lines 82-86). These filters execute during graph traversal, minimizing CPU consumption and network transfer by pruning invalid candidates before they exit the graph layer.

Implementation Examples

Instantiating the Core Graph Structure

import com.twitter.graphjet.bipartite.NodeMetadataLeftIndexedPowerLawMultiSegmentBipartiteGraph
import com.twitter.graphjet.stats.StatsReceiver

// Configure for 500M users with power-law degree distribution
val maxLeftNodes = 500_000_000L
val avgDegree = 100
val segmentSize = 1_000_000  // Tuned for power-law node density
val stats: StatsReceiver = ...

val graph = new NodeMetadataLeftIndexedPowerLawMultiSegmentBipartiteGraph(
  maxLeftNodes,
  avgDegree,
  segmentSize,
  stats.scope("graphJet")
)

Submitting Requests via the Handler

import com.twitter.recos.user_user_graph._
import com.twitter.recos.user_user_graph.thriftscala._

val request = RecommendUserRequest(
  requesterId = 123456L,
  seedsWithWeights = Map(111L -> 1.0, 222L -> 0.5),
  excludedUserIds = Some(Seq(999L)),
  maxNumResults = Some(50)
)

val handler = RecommendUsersHandlerImpl(
  bipartiteGraph = graph,
  salsaRunnerConfig = SalsaRunnerConfig(
    numSalsaRunners = 8, 
    timeoutSalsaRunner = 200  // milliseconds
  ),
  decider = new UserUserGraphDecider(),
  statsReceiverWrapper = FinagleStatsReceiverWrapper(stats)
)

val responseFut = handler.apply(request)

Monitoring Production Health

// Key metrics exposed via FinagleStatsReceiverWrapper
statsReceiver.counter("pollTimeout").get()
statsReceiver.stat("pollLatency").get()
statsReceiver.counter("failure").get()

Summary

• Sub-10ms latency is achieved through in-memory storage and O(1) edge lookups in the power-law segmented bipartite graph.
• >10k RPS throughput per instance relies on pre-instantiated runner pools stored in AsyncQueue to eliminate per-request allocation costs.
• Predictable memory usage results from fixed retention windows (typically 7 days) and primitive collections from fastutil that minimize GC pressure.
• Operational resilience comes from time-bounded polling, inline result filtering, and real-time Finagle metrics that expose queue health and traversal bottlenecks.

Frequently Asked Questions

How does GraphJet handle memory pressure from high-degree nodes?

GraphJet uses a power-law-aware segmentation strategy in NodeMetadataLeftIndexedPowerLawMultiSegmentBipartiteGraph that isolates heavy nodes into dedicated segments. This prevents high-degree celebrities from causing heap fragmentation and ensures that traversal algorithms maintain consistent performance regardless of node degree distribution.

What limits the scalability of a single GraphJet instance?

The primary bottleneck is heap size for the bipartite graph structure. Since GraphJet stores all edges in RAM, capacity is bounded by the retention window (typically one week of data) and the JVM heap allocated to the service. Horizontal scaling through sharding is required when the working set exceeds available memory, as documented in the UUG and UserVideoGraph service configurations.

Can GraphJet tolerate JVM garbage collection pauses?

Yes, through aggressive reduction of object allocation. By using Long2DoubleOpenHashMap and LongOpenHashSet from the fastutil library instead of standard Java collections, GraphJet minimizes temporary object creation during traversals. Combined with the async runner pool that reuses traversal objects, the system maintains steady throughput even during concurrent GC cycles.

How do operators tune GraphJet for specific latency requirements?

Operators adjust salsaRunnerConfig.timeoutSalsaRunner to establish hard latency ceilings and modify numSalsaRunners in the handler configuration to match CPU core counts. The FinagleStatsReceiverWrapper exposes pollLatency statistics that guide capacity planning—if queue wait times trend upward, operators scale instances or reduce the retention window to decrease graph size.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}