# How Import Filters Transform Source Content During Diagram Redrawing

> Discover how cathrynlavery/diagram-design import filters transform source content through sanitisation, normalization, and tracking. Convert Mermaid, Excalidraw, and Draw.io to SVG for redrawing.

- Repository: [Cathryn Lavery/diagram-design](https://github.com/cathrynlavery/diagram-design)
- Tags: internals
- Published: 2026-09-11

---

**Import filters in the cathrynlavery/diagram-design repository execute a three-phase pipeline—sanitisation, structural normalisation, and fidelity tracking—to convert raw Mermaid, Excalidraw, and Draw.io sources into canonical, render-ready SVG during diagram redrawing.**

The cathrynlavery/diagram-design project imports diagrams from diverse authoring tools and must normalize them into a unified internal representation before redrawing for various output formats. When import filters transform source content during diagram redrawing, they perform critical security cleaning, geometric standardization, and audit logging to ensure consistent rendering across PowerPoint, Figma, HTML, and PDF exports. This transformation pipeline is implemented across three specialized extract scripts that share a common filter-framework.

## The Three-Phase Transformation Pipeline

Import filters operate through a strict sequence of transformations that guarantee deterministic output regardless of the source authoring tool.

### Phase 1: Sanitisation and Security Filtering

The first phase strips disallowed CSS directives such as `@import` and `url()` functions that could expose external resources. Filters remove any HTML elements that fall outside the allowed SVG vocabulary, permitting only tags like `<path>`, `<image>`, and specific `style` attributes. This guarantees that generated SVG can be safely embedded in downstream consumers without breaking the strict SVG 1.1 specification or introducing security vulnerabilities.

### Phase 2: Structural Normalisation

During normalisation, filters convert relative transforms to absolute coordinates, collapse groups containing only decorative metadata, and rewrite connector way-points into orthogonal elbows. According to the Draw.io implementation in [`drawio_extract.py`](https://github.com/cathrynlavery/diagram-design/blob/main/drawio_extract.py), this phase enforces routing rules defined in SKILL.md § 6 to ensure the Diagram Design engine can animate and style layouts consistently. This structural standardisation eliminates routing heuristics that vary between source tools, providing a deterministic geometry that the animation system can manipulate reliably.

### Phase 3: Fidelity Ledger and Post-Processing

The final phase emits a **fidelity ledger** summarising what content was kept, altered, or dropped—entries such as "silently dropping content" or "filter-bleed removed" provide visibility into transformation decisions. After ledger generation, optional post-processors like the **Sketchy filter** (described in [`primitive-sketchy.md`](https://github.com/cathrynlavery/diagram-design/blob/main/primitive-sketchy.md)) apply SVG turbulence and displacement maps to create hand-drawn aesthetics, or normalize `rgba(...)` colour values for SVG-only viewers. These visual tweaks modify presentation without affecting the underlying data model.

## Implementation Across Extract Scripts

The concrete implementation lives in three extract scripts within `skills/diagram-design/scripts/`, each sharing a common filter-framework that parses source content, runs attribute whitelists, and applies visual filters.

### Mermaid Extraction Pipeline

In [`mermaid_extract.py`](https://github.com/cathrynlavery/diagram-design/blob/main/mermaid_extract.py), the filter parses `.mmd` files or fenced Mermaid blocks, drops unsupported CSS, and creates the fidelity ledger. The script maintains an explicit `ALLOWED_TAGS` whitelist and implements a `sanitise_svg()` function that iterates through the DOM, removing unauthorized elements and stripping disallowed attributes like `class` or inline `style` unless they match specific safe patterns.

### Excalidraw Extraction Pipeline

The [`excalidraw_extract.py`](https://github.com/cathrynlavery/diagram-design/blob/main/excalidraw_extract.py) script reads Excalidraw JSON, filters out non-SVG elements that cannot be rendered in the target environment, and rewrites layer structures to match the canonical representation. This ensures that hand-drawn elements from Excalidraw translate cleanly into the geometric primitives expected by the Diagram Design renderer.

### Draw.io Extraction Pipeline

For Draw.io sources, [`drawio_extract.py`](https://github.com/cathrynlavery/diagram-design/blob/main/drawio_extract.py) consumes `.drawio` XML files, removes connector way-points that conflict with orthogonal routing rules, and enforces the structural constraints required for animation. This script specifically handles the complexity of Draw.io's compressed XML format while applying the same sanitisation and normalisation rules as the other extractors.

## Code Examples: Import Filters in Action

### Command-Line Invocation

You can trigger the import filter pipeline directly from the command line to observe the transformation and fidelity ledger output:

```bash

# Import a Mermaid file – the import filter will sanitise CSS, drop unsupported nodes,

# and output a fidelity ledger at the end of the run.

diagram-design:import-mermaid scripts/fixtures/sample-flowchart.mmd \
    --format html --size doc-inline --detail balanced --audience mixed

```

### Python Filter Implementation

The core sanitisation logic inside [`mermaid_extract.py`](https://github.com/cathrynlavery/diagram-design/blob/main/mermaid_extract.py) demonstrates how the whitelist operates:

```python

# Inside mermaid_extract.py (simplified)

ALLOWED_TAGS = {"svg", "g", "line", "circle", "rect", "text", "title", "desc"}

def sanitise_svg(svg_root):
    for el in list(svg_root.iter()):
        if el.tag.split('}')[-1] not in ALLOWED_TAGS:
            ledger['dropped'].append(el.tag)
            el.getparent().remove(el)
        else:
            # Strip disallowed attributes like `style` or `class`

            for attr in list(el.attrib):
                if attr not in {"id", "data-*", "fill", "stroke", "x", "y", "width", "height"}:
                    ledger['removed_attrs'].append((el.tag, attr))
                    del el.attrib[attr]

```

### Optional Sketchy Visual Filter

As documented in [`primitive-sketchy.md`](https://github.com/cathrynlavery/diagram-design/blob/main/primitive-sketchy.md), you can apply a post-processing filter to give diagrams a hand-drawn appearance:

```html
<!-- Example of the optional Sketchy filter (primitive-sketchy.md) -->
<filter id="sketchy" x="-2%" y="-2%" width="104%" height="104%">
  <feTurbulence type="fractalNoise" baseFrequency="0.02" numOctaves="2"/>
  <feDisplacementMap in="SourceGraphic" scale="2"/>
</filter>

<g filter="url(#sketchy)">
  <!-- filtered diagram content -->
</g>

```

## Key Source Files Controlling Import Behavior

Understanding the import filter architecture requires familiarity with these specific files in the repository:

- **[`skills/diagram-design/scripts/mermaid_extract.py`](https://github.com/cathrynlavery/diagram-design/blob/main/skills/diagram-design/scripts/mermaid_extract.py)** – Parses Mermaid source, applies sanitisation and structural filters, and generates fidelity ledgers.

- **[`skills/diagram-design/scripts/excalidraw_extract.py`](https://github.com/cathrynlavery/diagram-design/blob/main/skills/diagram-design/scripts/excalidraw_extract.py)** – Handles Excalidraw JSON imports, enforces SVG element whitelists, and normalises layer structures.

- **[`skills/diagram-design/scripts/drawio_extract.py`](https://github.com/cathrynlavery/diagram-design/blob/main/skills/diagram-design/scripts/drawio_extract.py)** – Processes Draw.io XML, removes connector way-points, and applies SKILL.md § 6 routing rules.

- **[`skills/diagram-design/references/primitive-sketchy.md`](https://github.com/cathrynlavery/diagram-design/blob/main/skills/diagram-design/references/primitive-sketchy.md)** – Defines the optional Sketchy visual filter using SVG turbulence primitives.

- **[`skills/diagram-design/references/import-mermaid.md`](https://github.com/cathrynlavery/diagram-design/blob/main/skills/diagram-design/references/import-mermaid.md)** – Documents the Mermaid import flow, including "silently dropping content" behaviors and fidelity ledger formats.

- **[`skills/diagram-design/references/import-excalidraw.md`](https://github.com/cathrynlavery/diagram-design/blob/main/skills/diagram-design/references/import-excalidraw.md)** – Documents Excalidraw-specific import transformations and limitations.

- **[`skills/diagram-design/references/import-drawio.md`](https://github.com/cathrynlavery/diagram-design/blob/main/skills/diagram-design/references/import-drawio.md)** – Documents Draw.io XML processing and connector normalisation rules.

## Summary

Import filters transform source content during diagram redrawing through a rigorous, repeatable process:

- **Sanitisation** removes unsafe CSS and non-SVG HTML elements to ensure cross-platform security.
- **Structural normalisation** converts relative transforms to absolute coordinates and standardises connector routing.
- **Fidelity ledgers** provide transparent reporting of what content was modified or dropped during import.
- **Three extract scripts** ([`mermaid_extract.py`](https://github.com/cathrynlavery/diagram-design/blob/main/mermaid_extract.py), [`excalidraw_extract.py`](https://github.com/cathrynlavery/diagram-design/blob/main/excalidraw_extract.py), [`drawio_extract.py`](https://github.com/cathrynlavery/diagram-design/blob/main/drawio_extract.py)) implement format-specific parsing while sharing a common whitelist framework.
- **Optional post-processors** like the Sketchy filter apply visual effects without altering the underlying data model.

## Frequently Asked Questions

### What happens to unsupported HTML elements during import?

The import filter silently removes any HTML elements not present in the `ALLOWED_TAGS` whitelist (such as `<script>` or arbitrary `<div>` tags) and records their removal in the fidelity ledger. This ensures only safe, renderable SVG primitives remain in the final output.

### How does the fidelity ledger help debug import issues?

The fidelity ledger is a structured report emitted after filter processing that lists every element dropped, every attribute removed, and every geometric transformation applied. By reviewing entries such as "silently dropping content" or "filter-bleed removed," users can understand why specific visual elements disappeared or changed during the import process.

### What is the Sketchy filter and when should it be used?

The Sketchy filter is an optional post-processor defined in [`primitive-sketchy.md`](https://github.com/cathrynlavery/diagram-design/blob/main/primitive-sketchy.md) that applies SVG turbulence and displacement map effects to create a hand-drawn aesthetic. It should be used when presentation requirements call for informal, sketch-like diagrams rather than precise geometric rendering, as it modifies visual appearance without affecting the underlying coordinate data.

### Why are connector way-points rewritten during Draw.io import?

Draw.io files often contain complex routing heuristics and way-point data that conflict with the orthogonal routing requirements of the Diagram Design animation engine. The [`drawio_extract.py`](https://github.com/cathrynlavery/diagram-design/blob/main/drawio_extract.py) script rewrites these way-points to enforce deterministic, orthogonal elbows as specified in SKILL.md § 6, ensuring consistent animation paths across all output formats.