# How code-review-graph Handles Parsing for Jupyter and Databricks Notebooks

> code-review-graph parses Jupyter and Databricks notebooks by extracting code cells, filtering magic commands, and feeding normalized source into Tree-sitter parsers for unified dependency graphs. Discover efficient notebook ana...

- Repository: [Tirth Kanani/code-review-graph](https://github.com/tirth8205/code-review-graph)
- Tags: how-to-guide
- Published: 2026-07-26

---

**code-review-graph treats notebook files as first-class source types by detecting `.ipynb` extensions and Databricks Python exports, extracting executable code cells while filtering magic commands, and feeding normalized source into language-specific Tree-sitter parsers to generate unified dependency graphs.**

The `code-review-graph` repository provides a unified parsing pipeline that handles both standard Jupyter notebooks and Databricks notebook exports as native source files. By treating `.ipynb` JSON files and Databricks `.py` scripts as parseable code artifacts, the system enables dependency analysis and code review across polyglot notebook environments. This article examines the implementation details of how the parser detects, processes, and converts notebook cells into graph structures.

## File Type Detection and Routing

The parsing workflow begins with extension-based language detection defined in [`code_review_graph/parser.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/parser.py). At line 689, the extension mapping registers `.ipynb` as the language `"notebook"`, triggering specialized handling for JSON-based Jupyter files.

When the `parse()` method encounters a file whose detected language is `"notebook"`, it routes the input to the `_parse_notebook` method (lines 2470–2471). For Databricks exports saved as `.py` files, the parser checks the first line for the header `# Databricks notebook source` and diverts to `_parse_databricks_py_notebook` instead (lines 2471–2473).

## Jupyter Notebook Parsing Pipeline

### Kernel Detection and Cell Extraction

The `_parse_notebook` method loads the JSON payload and extracts the kernel language from the notebook metadata, defaulting to **Python** when unspecified (lines 2915–2920). Currently, the parser only supports **Python** and **R** kernels (lines 2423–2426).

The method iterates over the `cells` array, keeping only code cells while discarding markdown and raw cells. For each code cell, it examines magic directives to determine the effective programming language.

### Magic Command Processing

Cell-level language detection handles Jupyter magic commands through pattern matching. The parser looks for **magic** directives including `%python`, `%sql`, and `%r` to override the kernel language for that specific cell (lines 2828–2835). Conversely, it drops cells containing skip magics such as `%scala`, `%md`, or `%sh`.

For Python and R cells, the parser strips lines beginning with `%` or `!` (lines 3262–3268), removing shell-style commands that should not be parsed as executable code. The cleaned source of each valid cell is encapsulated in a `CellInfo` object (lines 2744–2749), preserving the original cell index and language metadata.

## Databricks Python Export Parsing

### Header Detection and Cell Separation

Databricks notebooks exported as `.py` files use a distinct format recognized by the header comment `# Databricks notebook source`. When detected, the parser invokes `_parse_databricks_py_notebook` (lines 3125–3130), strips the header, and splits the file on `# COMMAND ----------` delimiters to reconstruct logical cells (lines 3137–3143).

### MAGIC Directive Handling

Each extracted cell may contain `# MAGIC` prefixes indicating the cell's language, such as `# MAGIC %sql` or `# MAGIC %r`. The parser ignores cells marked with `# MAGIC %md` or `# MAGIC %sh`. For Python cells, it removes the `# MAGIC` prefix and stores the remaining source as a `CellInfo` with language `"python"` (lines 4445–4470).

After processing, the collected cells feed into the shared parsing pipeline, and the resulting `File` node receives the extra metadata field `notebook_format="databricks_py"` (lines 3215–3219).

## Unified Cell Processing and Graph Generation

### Language Grouping and AST Parsing

Both Jupyter and Databricks paths converge on `_parse_notebook_cells`, which groups cells by language (`python`, `r`, `sql`). For **Python** and **R** groups, the concatenated source strings are submitted to the Tree-sitter parser for that language. Line numbers are remapped so that each language group starts at line 1, with each cell's offset recorded to maintain accurate source references (lines 2810–2830).

**SQL** cells undergo lightweight analysis using regex patterns that extract table names and generate `IMPORTS_FROM` edges (lines 3220–3227), bypassing the full AST parsing used for general-purpose languages.

### Node Construction and Notebook Metadata

The routine returns `NodeInfo` and `EdgeInfo` objects representing classes, functions, imports, and calls, augmented with notebook-specific metadata such as `cell_index` (lines 3010–3031). The final `File` node records the original notebook path, marks test files when appropriate, and attaches format-specific extras, enabling the dependency analysis pipeline to treat notebooks identically to standard source files.

## Practical Usage Example

```python
from code_review_graph.parser import CodeParser
from pathlib import Path

# Parse a Jupyter notebook

parser = CodeParser()
nodes, edges = parser.parse(Path("example.ipynb"))

# Each node includes language, line numbers, and cell metadata

for n in nodes:
    print(n.kind, n.name, n.language, n.extra.get("cell_index"))

# Parse a Databricks .py notebook export

db_nodes, db_edges = parser.parse(Path("export.py"))

# The File node carries notebook_format="databricks_py"

print(db_nodes[0].extra["notebook_format"])

```

The `parse` method internally chains `_detect_language` → language-specific dispatch → `_parse_notebook` or `_parse_databricks_py_notebook` → `_parse_notebook_cells`, producing a unified graph representation regardless of the original format.

## Summary

- **code-review-graph** registers `.ipynb` files as the `"notebook"` language type in [`code_review_graph/parser.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/parser.py) (line 689) and detects Databricks exports via header inspection (lines 2471–2473).
- Jupyter parsing extracts Python and R code cells while filtering magic commands (`%scala`, `%md`, `%sh`) and shell prefixes to produce clean `CellInfo` objects (lines 2744–2749).
- Databricks `.py` exports split on `# COMMAND ----------` delimiters and process `# MAGIC` directives to normalize cells before shared processing (lines 3137–3143).

- The unified `_parse_notebook_cells` routine groups code by language, remaps line numbers, and generates `NodeInfo`/`EdgeInfo` objects compatible with the standard graph pipeline (lines 3010–3031).

## Frequently Asked Questions

### What notebook formats does code-review-graph support?

The parser supports standard Jupyter notebooks (`.ipynb` files) and Databricks notebook exports saved as `.py` files. For Jupyter files, it handles JSON structures with Python and R kernels (lines 2423–2426). For Databricks, it recognizes the `# Databricks notebook source` header and `COMMAND` delimiters (lines 2471–2473).

### How does the parser handle magic commands like %sql or %md?

Magic commands are processed differently based on type. Active magics like `%python`, `%sql`, or `%r` override the cell language, while skip magics including `%md`, `%scala`, and `%sh` cause the cell to be excluded from parsing (lines 2828–2835). In Databricks exports, the `# MAGIC` prefix is stripped before language detection (lines 4445–4470).

### Can code-review-graph parse notebooks with mixed languages (Python and R)?

Yes. The parser supports polyglot notebooks by grouping cells by detected language after processing magic directives. Python and R cells are fed to their respective Tree-sitter parsers, while SQL cells receive specialized regex handling (lines 2810–2830). Each cell retains its `cell_index` metadata to preserve notebook structure.

### How are line numbers preserved when parsing notebook cells?

The parser remaps line numbers so that each language group starts at line 1, while recording each cell's offset within the original notebook (lines 2810–2830). This allows accurate cross-referencing between the generated graph nodes and the source notebook locations, even when cells are concatenated for language-specific parsing.