What Data Can Be Represented in Archify Data Flow Diagrams

Archify data flow diagrams model the end-to-end movement and transformation of data assets across systems using a strict JSON-IR schema that captures metadata, processing stages, component nodes, directional flows, and contextual annotations.

The open-source Archify repository (tt-a1i/archify) provides a structured approach to visualizing complex data pipelines. Understanding what data can be represented in Archify data flow diagrams requires examining the specific JSON schema defined in archify/schemas/dataflow.schema.json that drives the visualization engine.

Core Data Elements in Archify Data Flow Diagrams

Diagram Metadata

The meta section defines high-level diagram properties including the title, optional subtitle, animation settings, and canvas dimensions. The viewBox array establishes the coordinate system for the entire visualization. For example, "viewBox": [1080, 760] sets the width and height boundaries for the diagram canvas.

Processing Stages

The stages array represents logical phases of the data pipeline. Each stage object contains a label property that groups nodes into vertical columns, such as Sources, Ingest, Process, Store, and Consume. This structure enforces a left-to-right reading pattern that mirrors data movement through the system.

Data Nodes and Components

Nodes represent individual components that hold or process data. According to archify/schemas/common.schema.json, each node requires:

• id: Unique identifier referenced by flows
• type: Semantic component type (frontend, cloud, security, messagebus, database, backend, etc.)
• label: Human-readable name
• stage and row: Placement coordinates within the diagram grid
• Optional: sublabel and tag for additional context

Data Flows and Connections

The flows array defines directed edges between nodes using these properties:

• from and to: Node IDs defining the data direction
• label: Description of the data being transported (e.g., "clickstream", "raw events")
• classification: Semantic grouping (e.g., "user events", "PII")
• variant: Visual styling (default, emphasis, security, dashed)
• Routing hints: route, fromSide, toSide, via, labelAt

Narrative Annotations

Optional cards provide narrative context alongside the diagram. Each card includes a dot color indicator, title, and items array describing specific paths or security boundaries. These annotations appear in archify/examples/product-analytics.dataflow.json to highlight primary data paths and sensitive processing stages.

Supported Data Workflow Patterns

Archify data flow diagrams can represent complete data-centric workflows:

• Sources: Web applications, mobile SDKs, IoT devices, external API feeds
• Ingestion and Transport: Edge APIs, message buses (Kafka, RabbitMQ), streaming services
• Processing and Security: Consent gates, transformation services, PII vaults, security filters
• Storage: Data warehouses, feature stores, relational databases, NoSQL stores
• Consumers: Analytics dashboards, ML models, downstream microservices, reporting tools

JSON Structure and Validation

The formal schema definition resides in archify/schemas/dataflow.schema.json, which enforces strict typing for componentType, id, and variant via references to archify/schemas/common.schema.json. The validation ensures that every flow references existing node IDs in the nodes array, maintaining referential integrity across the data pipeline representation.

Practical Implementation Examples

Basic Three-Stage Pipeline

{
  "schema_version": 1,
  "diagram_type": "dataflow",
  "meta": {
    "title": "Simple ETL Flow",
    "viewBox": [800, 600]
  },
  "stages": [
    { "label": "Extract" },
    { "label": "Transform" },
    { "label": "Load" }
  ],
  "nodes": [
    { "id": "src", "type": "frontend", "label": "Source API", "stage": 0, "row": 0 },
    { "id": "proc", "type": "backend", "label": "Processor", "stage": 1, "row": 0 },
    { "id": "db", "type": "database", "label": "Data Lake", "stage": 2, "row": 0 }
  ],
  "flows": [
    {
      "from": "src",
      "to": "proc",
      "label": "raw data",
      "classification": "extraction",
      "variant": "default",
      "fromSide": "right",
      "toSide": "left"
    },
    {
      "from": "proc",
      "to": "db",
      "label": "cleaned data",
      "classification": "load",
      "variant": "emphasis",
      "fromSide": "right",
      "toSide": "left"
    }
  ]
}

Adding Narrative Highlights

{
  "cards": [
    {
      "dot": "emerald",
      "title": "Hot Path",
      "items": [
        "Raw events flow from Source to Processor to Data Lake",
        "These flows are marked with the `emphasis` variant"
      ]
    },
    {
      "dot": "rose",
      "title": "Sensitive Data",
      "items": [
        "Any PII would be routed through a `security` node before storage"
      ]
    }
  ]
}

Summary

• Archify data flow diagrams use a JSON-IR schema defined in archify/schemas/dataflow.schema.json to model data asset movement across systems.
• The structure comprises five main sections: meta for diagram properties, stages for logical phases, nodes for system components, flows for data connections, and cards for annotations.
• Nodes support semantic types including frontend, backend, database, security, messagebus, and cloud as defined in archify/schemas/common.schema.json.
• Flows capture directional data movement with properties for labeling, classification, and visual styling via variant options (default, emphasis, security, dashed).
• The schema enforces strict validation, ensuring all flow references point to existing node IDs and component types conform to allowed enumerations.

Frequently Asked Questions

What file format does Archify use for data flow diagrams?

Archify uses a JSON-IR format defined by the Data Flow schema in archify/schemas/dataflow.schema.json. This structured JSON captures metadata, stages, nodes, flows, and optional cards, validated against strict typing rules in archify/schemas/common.schema.json to ensure diagram integrity.

How do I define what type of data moves between components?

Data movement is defined in the flows array using the label property to describe the payload and classification for semantic grouping. For example, "label": "clickstream" and "classification": "user events" clearly identify the data type traversing from a frontend node to a processing node.

Can I highlight security boundaries or critical paths in the diagram?

Yes. Use the optional cards array to add narrative annotations that highlight primary paths, security boundaries, or derived consumers. Each card supports a dot color indicator (e.g., "emerald" or "rose"), a title, and descriptive items to explain specific data flows or security considerations.

What component types are available for representing data nodes?

The schema supports multiple semantic component types defined in archify/schemas/common.schema.json, including frontend, cloud, security, messagebus, database, and backend. These types determine the visual representation and semantic meaning of each node within the data pipeline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}