# How Doc2Graph Handles Layout Analysis and Table Detection: A Technical Deep Dive

> Discover how Doc2Graph leverages geometric edge features and edge classification for effective layout analysis and table detection in scanned documents. Learn the technical details.

- Repository: [Andrea Gemelli/doc2graph](https://github.com/andreagemelli/doc2graph)
- Tags: deep-dive
- Published: 2026-02-24

---

**Doc2Graph converts scanned documents into graph structures where geometric edge features capture spatial layout relationships, and a binary edge classifier detects tables by identifying connected components of "table" edges between OCR tokens.**

Doc2Graph is an open-source framework that transforms document images and PDFs into graph representations suitable for Graph Neural Networks (GNNs). Understanding how Doc2Graph performs layout analysis and table detection requires examining its graph construction pipeline in `GraphBuilder` and feature enrichment in `FeatureBuilder`. The system combines OCR tokenization with geometric feature engineering to jointly learn document structure and tabular region identification according to the `andreagemelli/doc2graph` source code.

## Layout Analysis Through Graph Topology and Geometric Features

### Token Extraction and Graph Construction

Layout analysis begins when the OCR engine (`easyocr`) returns a list of bounding boxes (`boxs`) and associated texts (`texts`) for each input image. The `GraphBuilder` class constructs the initial topology using either a **fully-connected** strategy via `fully_connected` or a **k-NN** neighborhood approach via `knn_connection`. Both functions return source (`u`) and destination (`v`) node indices that define the graph edges.

In [`doc2graph/data/graph_builder.py`](https://github.com/andreagemelli/doc2graph/blob/main/doc2graph/data/graph_builder.py), the `__fromIMG` method handles this edge creation in lines **[48‑54](https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/data/graph_builder.py#L48-L54)**.

### Spatial Feature Engineering

Once the graph topology exists, `FeatureBuilder.add_features` enriches nodes and edges with geometric information. Each node receives normalized geometry features (`geom`) representing the four coordinates of its bounding box divided by image width and height (`self.sg`).

For edges, the system computes **Euclidean distance** and **angle** between connected nodes using the `polar` function. These values are normalized and quantized into polar bins via `to_bin`, then stored as edge features in `g.edata["feat"]`. This allows the GNN to reason about directional relationships such as "this token is to the right of that token."

These operations appear in [`doc2graph/data/feature_builder.py`](https://github.com/andreagemelli/doc2graph/blob/main/doc2graph/data/feature_builder.py) at lines **[71‑75](https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/data/feature_builder.py#L71-L75)** for node geometry and lines **[33‑55](https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/data/feature_builder.py#L33-L55)** for edge weights.

## Table Detection via Edge Classification

### Ground Truth Labeling from XML Annotations

Table detection relies on edge-level supervision derived from dataset annotations. In the PAU and FUNSD datasets, XML entries contain `<TextRegion>` tags with labels *"positions"* or *"total"* that identify table cells. During graph construction in `__fromPAU`, each OCR token is assigned a node label (`nl`) based on which region it falls into geometrically (`if r[0] < c[0] < r[2] …`).

This mapping logic resides in [`doc2graph/data/graph_builder.py`](https://github.com/andreagemelli/doc2graph/blob/main/doc2graph/data/graph_builder.py) at lines **[80‑88](https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/data/graph_builder.py#L80-L88)**.

### Edge Label Generation

After node labeling, the system generates binary edge targets. If **both endpoint nodes belong to a table region** (`nl[e[0]] == "positions" or nl[e[0]] == "total"`), the edge label is set to **"table"**; otherwise, it is **"none"**. These labels become the edge-class target during model training, where `edge_num_classes = 2` in the pretraining configuration.

This labeling mechanism appears in the same file at lines **[96‑114](https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/data/graph_builder.py#L96-L114)**.

### Inference and Bounding Box Extraction

During inference, the model predicts edge classes, and table regions are extracted by analyzing edges classified as **class 1** (table). The inference pipeline:

1. Collects table edges: `links = (epreds == 1).nonzero(as_tuple=True)[0]`
2. Builds an induced subgraph: `dgl.edge_subgraph`
3. Computes bounding boxes from the min/max of node geometries within the subgraph
4. Scales coordinates back to pixel space

This extraction logic is implemented in [`doc2graph/training/pau.py`](https://github.com/andreagemelli/doc2graph/blob/main/doc2graph/training/pau.py) at lines **[66‑84](https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/training/pau.py#L66-L84)**.

### Evaluation Metrics

Predicted table bounding boxes are compared against ground-truth regions using the `match_pred_w_gt` helper function, which implements IoU-based matching. The system accumulates precision, recall, and F1 scores per-sample and aggregates them across the dataset (`all_precisions`, `all_recalls`, `all_f1`).

The metric computation appears in [`doc2graph/training/pau.py`](https://github.com/andreagemelli/doc2graph/blob/main/doc2graph/training/pau.py) at lines **[49‑63](https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/training/pau.py#L49-L63)**.

## End-to-End Implementation Example

The following example demonstrates the complete pipeline from image input to graph-based inference:

```python
from doc2graph.data.graph_builder import GraphBuilder
from doc2graph.data.feature_builder import FeatureBuilder
from doc2graph.inference import inference

# 1️⃣ Build graphs from a list of image/PDF paths

gb = GraphBuilder()
graphs, _, _, feats = gb.get_graph(["sample1.png", "sample2.pdf"], "CUSTOM")

# 2️⃣ Enrich graphs with layout and visual features

fb = FeatureBuilder(d="cpu")
chunks, _ = fb.add_features(graphs, feats)

# 3️⃣ Run inference (weights must be a checkpoint list)

#    e.g. ["funsd-e2e-20230101-1234.pt"]

preds = inference(weights=["funsd-e2e-20230101-1234.pt"],
                  paths=["sample1.png", "sample2.pdf"],
                  device=0)   # GPU 0

```

The `inference` function visualizes predicted key-value pairs, while the **edge class "table"** remains stored in the graph metadata for downstream extraction.

## Summary

- **Layout analysis** in Doc2Graph relies on geometric node features (normalized bounding boxes) and polar edge features (distance and angle) computed in `FeatureBuilder`.
- **Graph construction** supports both fully-connected and k-NN topologies via `GraphBuilder.__fromIMG`.
- **Table detection** is formulated as binary edge classification, where edges connecting table cells are labeled "table" and others "none."
- **Inference** extracts tables by finding connected components of predicted table edges and computing their bounding boxes from node geometries.
- **Evaluation** uses IoU-based matching in [`training/pau.py`](https://github.com/andreagemelli/doc2graph/blob/main/training/pau.py) to compute precision, recall, and F1 for table detection tasks.

## Frequently Asked Questions

### How does Doc2Graph represent spatial relationships between text tokens?

Doc2Graph represents spatial relationships through **polar edge features** computed in `FeatureBuilder.add_features`. The system calculates the Euclidean distance and angle between connected nodes, normalizes these values, and quantizes them into bins. These features allow the GNN to learn directional and proximity-based relationships, such as whether one token appears above or to the left of another.

### What distinguishes table edges from non-table edges in the graph?

Table edges are distinguished by **node membership in annotated table regions**. During training data preparation in `GraphBuilder.__fromPAU`, nodes receive labels based on whether they fall within XML-annotated table regions (labeled "positions" or "total"). An edge is labeled "table" only if both endpoints reside within such regions; otherwise, it receives the "none" label. The model learns to predict this binary classification during training.

### How does the model convert edge predictions into table bounding boxes?

After inference, the model identifies all edges predicted as class 1 (table) and constructs an induced subgraph using `dgl.edge_subgraph`. The bounding box for each predicted table is derived by computing the minimum and maximum coordinates (`min` and `max`) of the node geometries within this subgraph. These normalized coordinates are then scaled back to the original image dimensions to produce the final pixel-level bounding box.

### Which datasets support the table detection training pipeline?

The table detection training pipeline primarily supports the **PAU** and **FUNSD** datasets. These datasets provide XML annotations containing `<TextRegion>` entries specifically labeled for table content. The `GraphBuilder` class parses these annotations in `__fromPAU` to generate ground-truth node and edge labels necessary for supervised learning of table boundaries.