# How Archify Visualizes Data Pipelines with the Dataflow Renderer

> Learn how Archify visualizes data pipelines using the Dataflow Renderer. Explore the six-stage process that transforms JSON into interactive SVG diagrams. See your data flow clearly.

- Repository: [tt-a1i/archify](https://github.com/tt-a1i/archify)
- Tags: how-to-guide
- Published: 2026-09-05

---

**Archify converts declarative JSON pipeline descriptions into polished, interactive SVG diagrams through a six-stage layout, routing, and rendering pipeline driven by `render-dataflow.mjs`.**

The dataflow renderer is the core visualization engine in [tt-a1i/archify](https://github.com/tt-a1i/archify), transforming abstract data pipeline definitions into publication-ready diagrams. It treats every pipeline as a **data-flow diagram** composed of stages (columns), nodes (processing units), and flows (data movement arrows). This article explains how the renderer works, from JSON input to SVG output, with specific references to the source implementation.

## Loading the Pipeline Definition

The renderer begins by ingesting a JSON **intermediate representation (IR)** that declares the complete pipeline structure. The entry point `loadDiagramWithBrandMarks` in `archify/renderers/dataflow/render-dataflow.mjs` parses this IR, extracting `stages`, `nodes`, `flows`, and optional metadata including view-box dimensions and animation profiles【1†L45-L49】.

The default example [`product-analytics.dataflow.json`](https://github.com/tt-a1i/archify/blob/main/product-analytics.dataflow.json) demonstrates a complete IR with business domains, security boundaries, and narrative cards【2†L1-L46】:

```json
{
  "diagram_type": "dataflow",
  "stages": [
    { "label": "Source" },
    { "label": "Process" },
    { "label": "Sink" }
  ],
  "nodes": [
    { "id": "api", "type": "frontend", "label": "API Gateway", "stage": 0, "row": 0 },
    { "id": "worker", "type": "backend", "label": "Worker Pool", "stage": 1, "row": 0 },
    { "id": "db", "type": "database", "label": "Analytics DB", "stage": 2, "row": 0 }
  ],
  "flows": [
    { "from": "api", "to": "worker", "label": "events" },
    { "from": "worker", "to": "db", "label": "aggregations" }
  ]
}

```

## Calculating the Layout

The layout engine computes precise geometry for every visual element before generating SVG.

### Stage Positioning

Stages receive horizontal placement through `stageX`, `stageW`, and `colGap` parameters. Each stage renders as a rectangular `stageFrame` that visually groups its contained nodes【1†L73-L88】.

### Node Measurement

The `measureNode` function determines each node's width, height, center-point (`cx`), and vertical position (`y`) based on its assigned `stage` and `row` indices. This ensures consistent spacing regardless of label length or node type【1†L92-L105】.

### Flow Routing

Flows undergo validation for unique IDs, valid node references, minimum path length, and non-diagonal routing constraints. The `routeVia` function applies explicit routing directives when provided via `via`, `channelX`, `channelY`, or `route` properties; otherwise, automatic orthogonal routing selects the shortest valid path【1†L96-L119】【1†L124-L142】.

## Generating Connection Paths

For each validated flow, the renderer:

1. Obtains start and end attachment points via the `anchor` function
2. Applies routing hints through `routeVia`
3. Removes degenerate or redundant path points
4. Constructs an SVG polyline stored in `pathCache` for reuse【1†L37-L63】

This caching strategy prevents redundant path calculations when the same connection geometry appears multiple times or requires label positioning.

## Rendering to SVG

The rendering phase executes four specialized functions that emit SVG elements:

- **`renderStage`** — draws stage frames as `<rect>` elements with labels【1†L65-L70】
- **`renderNode`** — draws nodes as styled rectangles with type-based fills, optional sub-labels, tags, and brand marks【1†L72-L95】
- **`renderFlowPath`** — draws flows as `<path>` elements with computed points and arrowhead markers【1†L98-L104】
- **`renderFlowLabel`** — positions descriptive labels at the midpoint of each flow path【1†L105-L115】

The renderer also generates an automatic legend via `renderLegend`, mapping flow variants (e.g., "emphasis", "dashed", "secure") to visual styles【1†L118-L132】.

## Validation and Diagnostics

Before final output, `validateDataflow()` traverses all nodes and flows to detect:

- Overlapping node geometries
- Out-of-bounds coordinates
- Flow label collisions
- Disconnected or invalid references

Detected problems accumulate in a diagnostic structure; if any exist, `throwDiagnosticProblems` halts execution with a detailed report【1†L18-L93】. This strict validation ensures output quality and catches subtle layout errors early.

## Writing Final Output

The completed SVG includes:

- Accessibility metadata and `<title>` elements
- `<defs>` for gradients, patterns, and reusable arrowheads
- Optional background grid
- Optional narrative cards for annotations
- Responsive viewBox sizing

`writeDiagram` serializes this structure to the target file path【1†L74-L84】.

## Command-Line and Programmatic Usage

Generate diagrams directly from the CLI:

```bash
npx archify --type dataflow --source examples/product-analytics.dataflow.json

```

Output writes to [`examples/dataflow-product-analytics.html`](https://github.com/tt-a1i/archify/blob/main/examples/dataflow-product-analytics.html) with embedded SVG and interactive controls.

Embed finished diagrams in documentation:

```html
<iframe src="gallery/artifacts/product-analytics.dataflow.html?embed=1&theme=dark"
        title="Product Analytics data flow"
        width="100%"
        height="600"
        loading="lazy">
</iframe>

```

## Key Implementation Files

| File | Purpose |
|------|---------|
| `archify/renderers/dataflow/render-dataflow.mjs` | Core renderer implementing layout, validation, and SVG generation【1†L45-L84】【1†L92-L130】 |
| [`examples/product-analytics.dataflow.json`](https://github.com/tt-a1i/archify/blob/main/examples/product-analytics.dataflow.json) | Reference IR with stages, flows, security zones, and cards【2†L1-L46】 |
| [`docs/gallery.html`](https://github.com/tt-a1i/archify/blob/main/docs/gallery.html) | Gallery loader for rendered artifacts【1†L428-L452】 |
| [`scripts/start-template.html`](https://github.com/tt-a1i/archify/blob/main/scripts/start-template.html) | CLI bootstrap template for new `dataflow` diagrams【1†L221-L284】 |

## Summary

- **Input format**: Declarative JSON IR with `stages`, `nodes`, and `flows`
- **Layout engine**: Calculates geometry for stages, measures nodes, routes flows orthogonally
- **Path generation**: Caches polylines with anchor points and routing hints
- **Rendering**: Specialized functions emit stage frames, node boxes, flow paths, labels, and legends
- **Quality assurance**: `validateDataflow()` enforces geometric and topological constraints
- **Output**: Optimized SVG with accessibility features, responsive sizing, and optional interactivity

## Frequently Asked Questions

### What file format does the dataflow renderer accept?

The renderer accepts a JSON intermediate representation with top-level keys for `diagram_type` (must be `"dataflow"`), `stages` (array of column definitions), `nodes` (array of processing units with `stage`/`row` positioning), and `flows` (array of connections with `from`/`to` node references). Optional keys include `viewBox`, `theme`, `cards` for annotations, and `animation` for temporal sequence rendering.

### How does Archify prevent overlapping nodes or crossed flows?

The `validateDataflow()` function executes geometric checks before SVG generation. It computes axis-aligned bounding boxes for all nodes and flags intersections. For flows, it validates that automatic routing produces non-degenerate paths and that explicit `via` coordinates don't create self-intersections. Validation failures throw detailed diagnostics with node/flow IDs and specific problem descriptions【1†L18-L93】.

### Can I customize the visual style of dataflow diagrams?

Yes. The IR accepts `theme` overrides for colors, `variant` properties on flows to select from predefined styles (solid, dashed, emphasis, secure), and `brandMarks` for logo overlays. The `renderNode` function selects fill colors based on node `type` values like `"frontend"`, `"backend"`, `"database"`, or `"queue"`【1†L72-L95】. For deeper customization, modify the template in [`scripts/start-template.html`](https://github.com/tt-a1i/archify/blob/main/scripts/start-template.html) or post-process the generated SVG.

### Is the dataflow renderer suitable for real-time or animated pipelines?

The renderer supports static SVG output with optional animation sequences. The `animation` property in the IR can specify frame-by-frame visibility changes or staged reveals. For fully interactive or real-time visualizations, export the SVG and manipulate DOM elements via JavaScript, or regenerate the diagram when pipeline topology changes.