# How the Data Flow Renderer Visualizes Pipelines and Data Lineage in Archify

> Explore the Archify Data Flow renderer and discover how it visualizes data pipelines and lineage through interactive SVG diagrams. Understand data movement and privacy boundaries effortlessly.

- Repository: [tt-a1i/archify](https://github.com/tt-a1i/archify)
- Tags: deep-dive
- Published: 2026-09-01

---

**The Archify Data Flow renderer transforms JSON specifications into interactive SVG diagrams that map data movement through transformation stages, highlighting privacy-sensitive boundaries and processing paths with visual variants.**

The **Data Flow renderer** is one of five built-in diagram types in the Archify open-source repository (tt-a1i/archify). It converts structured JSON descriptions into layered visualizations that make complex data pipelines, lineage relationships, and compliance boundaries immediately comprehensible.

## Core Concepts of Data Flow Visualization

The renderer operates on a schema-validated JSON structure that defines five key elements:

| Element | JSON Expression | Visual Treatment |
|---------|---------------|------------------|
| **Stages** | Ordered list in `stages` array | Horizontal rows grouping nodes by processing layer |
| **Nodes** | Objects with `id`, `label`, `type`, `stage` | Boxes or circles positioned in stage rows; type determines icon and color |
| **Flows** | Edge objects with `source`, `target`, `variant`, optional `via` and `labelAt` | Arrows with style driven by **variant** |
| **Legend** | Boolean `legend` flag and `cards` array | Side panel explaining variants and summarizing paths |
| **Annotations** | `label` on flows, `note` on nodes | Inline text or hover tooltips |

### Flow Variants and Their Visual Meanings

The **`variant`** field is the primary mechanism for communicating data lineage characteristics:

- **`emphasis`** — Solid bold arrow for primary analytic paths
- **`security`** — Dashed rose line with lock icon for PII or privacy boundaries
- **`dashed`** — Simple dashed line for batch or derived processing

## JSON Schema and Validation

All Data Flow specifications validate against [`archify/schemas/dataflow.schema.json`](https://github.com/tt-a1i/archify/blob/main/archify/schemas/dataflow.schema.json). This schema enforces required fields and constrains variant values to ensure consistent rendering.

A valid Data Flow JSON requires three top-level arrays:

1. **`stages`** — Defines the horizontal layers (e.g., Sources → Ingest → Process → Store → Consume)
2. **`nodes`** — Places components into stages with type classification
3. **`flows`** — Connects nodes with directed edges carrying variant metadata

## Rendering Pipeline: From JSON to SVG

The Data Flow renderer executes a four-stage transformation process:

1. **Stage layout** assigns fixed vertical slots; nodes distribute evenly across width
2. **Edge routing** draws paths from source to target, respecting optional `via` waypoints
3. **Style application** maps `variant` values to CSS classes controlling line style, color, and icons
4. **Interactivity binding** attaches hover states for labels/notes and click handlers for cards

## Complete Example: Product Analytics Data Flow

The repository includes a fully-featured example at [`archify/examples/product-analytics.dataflow.json`](https://github.com/tt-a1i/archify/blob/main/archify/examples/product-analytics.dataflow.json). Below is a simplified variant demonstrating all core features:

```json
{
  "title": "Simple Order Processing Data Flow",
  "stages": [
    { "id": "sources", "label": "Sources" },
    { "id": "ingest", "label": "Ingest" },
    { "id": "process", "label": "Process" },
    { "id": "store", "label": "Store" },
    { "id": "consume", "label": "Consume" }
  ],
  "nodes": [
    { "id": "web-sdk", "label": "Web SDK", "stage": "sources" },
    { "id": "edge-api", "label": "Edge API", "stage": "ingest" },
    { "id": "consent", "label": "Consent Gate", "stage": "process", "type": "security" },
    { "id": "event-bus", "label": "Event Bus", "stage": "process" },
    { "id": "warehouse", "label": "Analytics Warehouse", "stage": "store" },
    { "id": "dashboard", "label": "Dashboard", "stage": "consume" }
  ],
  "flows": [
    { "source": "web-sdk", "target": "edge-api", "variant": "emphasis" },
    { "source": "edge-api", "target": "consent", "variant": "security" },
    { "source": "consent", "target": "event-bus", "variant": "security" },
    { "source": "event-bus", "target": "warehouse", "variant": "emphasis" },
    { "source": "warehouse", "target": "dashboard", "variant": "emphasis" }
  ],
  "legend": true,
  "cards": [
    {
      "title": "Primary Path",
      "variant": "emphasis",
      "summary": "Web → Edge → Event Bus → Warehouse → Dashboard"
    },
    {
      "title": "Privacy Boundary",
      "variant": "security",
      "summary": "Consent Gate protects PII"
    }
  ]
}

```

Save as [`simple-order.dataflow.json`](https://github.com/tt-a1i/archify/blob/main/simple-order.dataflow.json) and render with:

```bash
archify view simple-order.dataflow.json

```

## Viewing the Live Demo

The repository provides a pre-rendered HTML demonstration at [`examples/dataflow-product-analytics.html`](https://github.com/tt-a1i/archify/blob/main/examples/dataflow-product-analytics.html). To launch it locally:

```bash

# After cloning tt-a1i/archify

archify view https://github.com/tt-a1i/archify/blob/main/archify/examples/product-analytics.dataflow.json

```

This renders five stages, ten nodes, and ten variant-coded flows with interactive legend and summary cards.

## Key Source Files

| Path | Purpose |
|------|---------|
| [`archify/schemas/dataflow.schema.json`](https://github.com/tt-a1i/archify/blob/main/archify/schemas/dataflow.schema.json) | Validation schema for Data Flow JSON |
| [`archify/renderers/dataflow/README.md`](https://github.com/tt-a1i/archify/blob/main/archify/renderers/dataflow/README.md) | Renderer configuration and embedding guide |
| [`archify/examples/product-analytics.dataflow.json`](https://github.com/tt-a1i/archify/blob/main/archify/examples/product-analytics.dataflow.json) | Full-featured example specification |
| [`examples/dataflow-product-analytics.html`](https://github.com/tt-a1i/archify/blob/main/examples/dataflow-product-analytics.html) | Browser-rendered output demonstration |
| `docs/assets/archify-dataflow.png` | Visual summary for documentation |

## What Data Lineage Visualization Reveals

The Archify Data Flow renderer makes three critical aspects of data architecture explicit:

- **End-to-end pipelines** — Complete traceability from ingestion through consumption
- **Privacy boundaries** — Immediate visual identification of PII touchpoints via `security` variant styling
- **Processing modes** — Distinction between real-time and batch paths through `emphasis` vs `dashed` variants

## Summary

- The **Data Flow renderer** converts JSON to interactive SVG using a validated schema-based pipeline
- **Flow variants** (`emphasis`, `security`, `dashed`) encode semantic meaning about data lineage and sensitivity
- **Stages** create horizontal layers that correspond to logical processing phases
- **Cards and legends** provide contextual summaries alongside the diagram
- All implementations reference source files in `archify/schemas/`, `archify/renderers/dataflow/`, and `archify/examples/`

## Frequently Asked Questions

### What JSON schema does the Data Flow renderer use?

The renderer validates input against [`archify/schemas/dataflow.schema.json`](https://github.com/tt-a1i/archify/blob/main/archify/schemas/dataflow.schema.json), which defines required `stages`, `nodes`, and `flows` arrays plus constrained `variant` values. Validation ensures consistent rendering across all Data Flow diagrams.

### How do I mark sensitive data flows in a Data Flow diagram?

Apply the **`security`** variant to flows handling PII or crossing privacy boundaries. This renders as a dashed rose line with a lock icon, making sensitive paths immediately visible in the visualization.

### Can I customize node placement within stages?

Yes. Use the optional `via` array in flow definitions to specify waypoint coordinates. The renderer routes edges through these points, enabling precise control over path geometry and overlap avoidance.

### Where can I see a complete rendered example?

The file [`examples/dataflow-product-analytics.html`](https://github.com/tt-a1i/archify/blob/main/examples/dataflow-product-analytics.html) in the tt-a1i/archify repository contains a live demonstration. Open it in any browser after cloning, or use `archify view` with the remote URL to render the bundled [`product-analytics.dataflow.json`](https://github.com/tt-a1i/archify/blob/main/product-analytics.dataflow.json) example.