# Understanding the Computational Graph Structure in micrograd Value Objects

> Explore the computational graph structure in micrograd Value objects. Learn how parent references, operations, and backward closures enable reverse-mode automatic differentiation for efficient deep learning.

- Repository: [Andrej/nn-zero-to-hero](https://github.com/karpathy/nn-zero-to-hero)
- Tags: internals
- Published: 2026-05-23

---

**The micrograd `Value` class implements a directed acyclic graph (DAG) where each node stores parent references in `_prev`, records the operation in `_op`, and encapsulates local gradient logic in a `_backward` closure to enable reverse-mode automatic differentiation.**

The educational autograd engine in the `karpathy/nn-zero-to-hero` repository demonstrates how modern deep learning frameworks track computations. The `Value` object serves as the fundamental unit of a computational graph structure in micrograd `Value` objects, recording scalar values alongside their lineage to calculate gradients via backpropagation.

## Core Attributes of the Value Node

As implemented in `lectures/micrograd/micrograd_lecture_first_half_roughly.ipynb` (lines 71–81), each `Value` instance maintains six critical attributes that define its role in the graph:

- **`data`** — The scalar numeric value (float).
- **`grad`** — Accumulated gradient initialized to `0.0`, representing the derivative of the final output with respect to this node.
- **`_prev`** — A Python `set` containing parent `Value` objects used as operands to compute the current node.
- **`_op`** — A string identifier (e.g., `'+'`, `'*'`, `'tanh'`) describing the operation that created this node.
- **`label`** — An optional string for human-readable identification during visualization.
- **`_backward`** — A closure attached during the forward pass that knows how to propagate gradients backward to parent nodes.

## Graph Construction During Forward Pass

Operations like addition and multiplication construct the graph dynamically. When you execute `c = a + b`, the `__add__` method creates a new `Value` containing the result data while establishing edges back to its parents.

In `lectures/micrograd/micrograd_lecture_first_half_roughly.ipynb`, the binary operation implementation follows this pattern:

```python
out = Value(self.data + other.data, (self, other), '+')

```

The second argument `(self, other)` populates `_prev`, and the third argument `'+'` populates `_op`. Immediately after construction, micrograd attaches a `_backward` function that implements the local gradient for that specific operation:

```python
def _backward():
    self.grad += out.grad          # dL/da

    other.grad += out.grad         # dL/db

out._backward = _backward

```

Similarly, multiplication defines its own `_backward` closure that scales the upstream gradient by the opposing operand’s value, implementing the product rule.

## Topological Ordering and the Backward Pass

Calling `some_value.backward()` triggers reverse-mode differentiation through a three-phase process defined in the notebook (line 16):

1. **Topological sort** — A depth-first traversal builds a list `topo` containing all reachable nodes, ensuring that children appear before their parents in the ordering.
2. **Seed gradient** — The target node’s gradient is set to `1.0` (representing ∂L/∂L).
3. **Reverse traversal** — Nodes are visited in reverse topological order, invoking each `_backward` closure to accumulate gradients into `_prev` parents via the chain rule.

This traversal guarantees that when a node’s `_backward` runs, its own `grad` already contains the complete upstream derivative, allowing it to correctly distribute gradients to its parents.

## Visualizing the Computational Graph

The notebook provides `trace` and `draw_dot` utilities that walk the `_prev` links to collect nodes and edges. These functions generate Graphviz diagrams showing each node’s label, data value, and current gradient, making the abstract DAG concrete for debugging and education.

To render a graph visualization:

```python
from graphviz import Digraph

dot = draw_dot(L)          # L is the final Value node

dot.render('micrograd_graph', view=True)

```

## Complete Example: Building and Traversing the Graph

The following example constructs the expression **L = (a × b + c) × f** and computes gradients:

```python

# Build the computational graph

a = Value(2.0, label='a')
b = Value(-3.0, label='b')
c = Value(10.0, label='c')
f = Value(-2.0, label='f')

e = a * b; e.label = 'e'      # Intermediate node

d = e + c; d.label = 'd'      # Addition node

L = d * f; L.label = 'L'      # Final output

# Forward pass

print(f"L.data = {L.data}")   # Output: 40.0

# Backward pass

L.backward()

# Inspected gradients

print(f"∂L/∂a = {a.grad}")    # -20.0

print(f"∂L/∂b = {b.grad}")    # 40.0

print(f"∂L/∂c = {c.grad}")    # -2.0

print(f"∂L/∂f = {f.grad}")    # 8.0

```

This demonstrates how the computational graph structure maintains the necessary information to analytically compute ∂L/∂a and other partial derivatives through recursive application of the chain rule.

## Summary

- The **computational graph structure in micrograd `Value` objects** forms a directed acyclic graph using the `_prev` attribute to store parent references.
- Each node encapsulates its local gradient calculation in a `_backward` closure, enabling modular backpropagation.
- The `backward()` method performs a reverse topological sort to ensure gradients propagate from children to parents in the correct order.
- Visualization utilities in `lectures/micrograd/micrograd_lecture_first_half_roughly.ipynb` leverage the `_prev` links to render the complete graph structure.

## Frequently Asked Questions

### What does the `_prev` attribute store in micrograd Value objects?

The `_prev` attribute stores a Python `set` containing the parent `Value` nodes that served as operands in the operation that created the current node. For example, when computing `c = a + b`, the new `Value` `c` holds its `_prev` set containing both `a` and `b`, establishing the directed edge in the computational graph.

### How does micrograd perform the backward pass?

The `backward()` method first performs a topological sort to establish a linear ordering where children precede parents. It then sets the output gradient to `1.0` and iterates through the sorted nodes in reverse, calling each stored `_backward` closure to accumulate gradients into parent nodes according to the chain rule.

### Is the computational graph in micrograd a DAG?

Yes, the computational graph structure is strictly a directed acyclic graph (DAG). The `_prev` references always point from result nodes to their operand nodes, and since values flow forward while gradients flow backward, no cycles can exist in the computation history.

### How can I visualize the micrograd computational graph?

Use the `draw_dot` function defined in `lectures/micrograd/micrograd_lecture_first_half_roughly.ipynb`. Pass the final output `Value` node to this function, which traverses `_prev` references to collect all nodes and edges, then returns a Graphviz `Digraph` object that can be rendered to SVG or PNG.