# Core Utilities in Graph Common for Building GraphJet-Based Graphs with Power-Law Degree Distributions

> Discover Graph Common utilities like MultiSegmentPowerLawBipartiteGraphBuilder for building power-law distributed graphs with GraphJet. Minimize boilerplate and gain observability.

- Repository: [X (fka Twitter)/the-algorithm](https://github.com/twitter/the-algorithm)
- Tags: deep-dive
- Published: 2026-03-03

---

**The Graph Common package provides specialized builders and wrappers—such as `MultiSegmentPowerLawBipartiteGraphBuilder` and `FinagleStatsReceiverWrapper`—that enable the construction of in-memory, power-law distributed bipartite graphs with minimal boilerplate and full observability integration.**

The `graph_common` package located at `src/scala/com/twitter/recos/graph_common` within the Twitter recommendation algorithm repository supplies the foundational components for creating robust GraphJet-based graphs. These core utilities encapsulate graph-initialization parameters and GraphJet-specific details, allowing Recos services to efficiently model user-item relationships—such as user-video, user-tweet, and user-user interactions—as multi-segment structures exhibiting power-law degree distributions.

## Multi-Segment Power-Law Graph Builders

### MultiSegmentPowerLawBipartiteGraphBuilder

Located in [`MultiSegmentPowerLawBipartiteGraphBuilder.scala`](https://github.com/twitter/the-algorithm/blob/main/MultiSegmentPowerLawBipartiteGraphBuilder.scala), this utility creates a `MultiSegmentPowerLawBipartiteGraph` that automatically shards the graph into time-based segments while preserving power-law characteristics. The builder uses a `GraphBuilderConfig` case class to bundle initialization parameters including `maxNumSegments`, `expectedNumLeftNodes`, `leftPowerLawExponent`, and degree caps. The `apply(config, statsReceiver)` method instantiates the graph with an `ActionEdgeTypeMask` and a wrapped stats receiver, handling internal segment management transparently.

### NodeMetadataLeftIndexedPowerLawMultiSegmentBipartiteGraphBuilder

For scenarios requiring rich node features, [`NodeMetadataLeftIndexedPowerLawMultiSegmentBipartiteGraphBuilder.scala`](https://github.com/twitter/the-algorithm/blob/main/NodeMetadataLeftIndexedPowerLawMultiSegmentBipartiteGraphBuilder.scala) provides a builder that attaches metadata to right nodes. This is essential when downstream recommendation models require embeddings or feature vectors per item. The configuration extends the base builder with `numRightNodeMetadataTypes` and a custom `EdgeTypeMask`, returning a `NodeMetadataLeftIndexedPowerLawMultiSegmentBipartiteGraph` optimized for left-indexed queries with right-side attributes.

### RightNodeMetadataLeftIndexedPowerLawMultiSegmentBipartiteGraphBuilder

The builder defined in [`RightNodeMetadataLeftIndexedPowerLawMultiSegmentBipartiteGraphBuilder.scala`](https://github.com/twitter/the-algorithm/blob/main/RightNodeMetadataLeftIndexedPowerLawMultiSegmentBipartiteGraphBuilder.scala) mirrors the metadata functionality but structures the graph to store metadata on the right side while maintaining left indexing. This variant supports Recos pipelines where right-side entities carry per-item features that must remain accessible during graph traversal operations.

## Observability and Metrics Integration

### FinagleStatsReceiverWrapper and FinagleCounterWrapper

GraphJet requires a `StatsReceiver` interface for monitoring, but Twitter's infrastructure relies on Finagle. The `FinagleStatsReceiverWrapper` class in [`FinagleStatsReceiverWrapper.scala`](https://github.com/twitter/the-algorithm/blob/main/FinagleStatsReceiverWrapper.scala) adapts Finagle's `StatsReceiver` to GraphJet's interface, implementing `scope` and `counter` methods that delegate to Finagle's telemetry pipeline. Complementing this, `FinagleCounterWrapper` in [`FinagleCounterWrapper.scala`](https://github.com/twitter/the-algorithm/blob/main/FinagleCounterWrapper.scala) wraps Finagle counters to satisfy GraphJet's `Counter` interface, ensuring that metrics like edge insertions and segment drops emit to standard Twitter monitoring systems without direct Finagle dependencies in the graph logic.

## Edge Classification and Graph Helpers

### ActionEdgeTypeMask

The `ActionEdgeTypeMask` class in [`ActionEdgeTypeMask.scala`](https://github.com/twitter/the-algorithm/blob/main/ActionEdgeTypeMask.scala) implements GraphJet's `EdgeTypeMask` interface to classify edge actions such as follows, retweets, or likes. This mask provides the `mask` logic used by all graph builders to distinguish between interaction types within the same bipartite structure, enabling weighted recommendation signals without requiring separate graph instances.

### BipartiteGraphHelper and NodeInfoHandler

[`BipartiteGraphHelper.scala`](https://github.com/twitter/the-algorithm/blob/main/BipartiteGraphHelper.scala) contains static utility functions for common graph operations including `getDegree` queries and power-law validation checks. For persistence operations, [`NodeInfoHandler.scala`](https://github.com/twitter/the-algorithm/blob/main/NodeInfoHandler.scala) supplies `writeNodeInfo` and `readNodeInfo` methods that handle serialization and deserialization of node metadata, ensuring that graph state can be saved and restored without losing power-law configuration or node attributes.

## Configuration and Implementation Examples

Constructing a power-law graph requires defining the degree distribution parameters through the builder configuration case classes. The following examples demonstrate standard patterns used in Twitter's Recos services.

Basic multi-segment power-law graph:

```scala
import com.twitter.recos.graph_common._
import com.twitter.graphjet.stats.StatsReceiver
import com.twitter.finagle.stats.{StatsReceiver => FinagleStats}

val cfg = MultiSegmentPowerLawBipartiteGraphBuilder.GraphBuilderConfig(
  maxNumSegments = 10,
  maxNumEdgesPerSegment = 10000000,
  expectedNumLeftNodes = 5000000,
  expectedMaxLeftDegree = 500,
  leftPowerLawExponent = 2.5,
  expectedNumRightNodes = 10000000,
  expectedMaxRightDegree = 200,
  rightPowerLawExponent = 2.0
)

val finagleStats: FinagleStats = ???
val graph = MultiSegmentPowerLawBipartiteGraphBuilder(
  cfg,
  FinagleStatsReceiverWrapper(finagleStats)
)

```

Graph with right-node metadata:

```scala
val metaCfg = NodeMetadataLeftIndexedPowerLawMultiSegmentBipartiteGraphBuilder.GraphBuilderConfig(
  maxNumSegments = 8,
  maxNumEdgesPerSegment = 5000000,
  expectedNumLeftNodes = 3000000,
  expectedMaxLeftDegree = 400,
  leftPowerLawExponent = 2.2,
  expectedNumRightNodes = 8000000,
  numRightNodeMetadataTypes = 2,  // e.g., embedding + category
  edgeTypeMask = new ActionEdgeTypeMask()
)

val metaGraph = NodeMetadataLeftIndexedPowerLawMultiSegmentBipartiteGraphBuilder(
  metaCfg,
  FinagleStatsReceiverWrapper(finagleStats)
)

```

Accessing metrics via the wrapper:

```scala
val segmentCounter = FinagleStatsReceiverWrapper(finagleStats).counter("segments_created")
segmentCounter.incr()

```

## Summary

- **Multi-segment builders** like `MultiSegmentPowerLawBipartiteGraphBuilder` automate the creation of time-sharded, power-law distributed bipartite graphs through configuration-driven initialization in `src/scala/com/twitter/recos/graph_common/`.
- **Metadata-aware variants** extend base graphs with right-node or left-node feature storage, enabling rich recommendation contexts without altering core graph topology.
- **Finagle integration wrappers** bridge GraphJet's monitoring interfaces with Twitter's telemetry infrastructure, exposing counters and stats for operational visibility.
- **Edge classification and helper utilities** provide action-type masking via `ActionEdgeTypeMask` and operational methods for degree queries and serialization via `BipartiteGraphHelper` and `NodeInfoHandler`.

## Frequently Asked Questions

### What is the purpose of the GraphBuilderConfig case class in Graph Common?

The `GraphBuilderConfig` case class encapsulates all initialization parameters required to construct a power-law bipartite graph, including the number of segments, expected node counts, maximum degrees, and power-law exponents for both left and right nodes. This configuration object ensures type-safe, immutable parameter passing to the builder's `apply` method.

### How does Graph Common handle monitoring and statistics for GraphJet graphs?

Graph Common adapts Twitter's Finagle metrics to GraphJet's interface through `FinagleStatsReceiverWrapper` and `FinagleCounterWrapper`. These classes delegate GraphJet's statistical operations—such as segment creation counters and edge insertion rates—to Finagle's `StatsReceiver`, allowing seamless integration with Twitter's existing monitoring dashboards.

### When should I use NodeMetadataLeftIndexedPowerLawMultiSegmentBipartiteGraphBuilder instead of the standard builder?

Use `NodeMetadataLeftIndexedPowerLawMultiSegmentBipartiteGraphBuilder` when your recommendation pipeline requires storing auxiliary data—such as embeddings, categories, or feature vectors—directly on right nodes. This builder extends the standard power-law graph with `numRightNodeMetadataTypes` support, eliminating the need for external lookup tables during graph traversal.

### What role does ActionEdgeTypeMask play in graph construction?

`ActionEdgeTypeMask` implements GraphJet's `EdgeTypeMask` interface to distinguish between different interaction types within the same graph structure. It assigns bit-masked identifiers to actions like follows or retweets, enabling the graph to store heterogeneous relationships while maintaining a single, memory-efficient bipartite representation.