How Instatic's Three-Layer Publishing Pipeline Works: Architecture Explained

Instatic renders pages through a three-layer pipeline—disk bake, in-memory LRU cache, and dynamic-island runtime—that separates static and dynamic content for maximum cacheability and performance.

Instatic's three-layer publishing pipeline is the core architecture that enables static-first publishing with selective dynamic hydration. This article explains how Layer A (disk bake), Layer B (in-memory LRU), and Layer C (dynamic-island runtime) interact to serve both cached static pages and personalized dynamic fragments.

Layer A: Disk Bake (Static Shell Generation)

The disk bake layer walks the page tree and writes complete HTML files—or static shells with placeholders—to the filesystem.

In server/publish/staticArtefact.ts, the pipeline performs an atomic two-slot symlink swap so readers never encounter half-written files. Pages write to uploads/published/current/<route>.html through a versioning system that keeps the previous build intact until the new one is fully ready.

Key characteristics of Layer A:

• Full tree traversal: publishPage in src/core/publisher/render.ts recursively renders every node
• Dynamic detection: findDynamicNodeIds in src/core/publisher/dynamicDetection.ts classifies request-dependent nodes
• Hole insertion: Request-dependent nodes become <instatic-hole> placeholders
• Asset co-location: CSS bundles (assembled in siteCssBundle.ts) and JS modules are written alongside HTML

The disk layer serves as the fast path for requests without loop pagination parameters—Nginx or the application server returns the file directly without invoking Node.js.

Layer B: In-Memory LRU (Dynamic Page Cache)

The in-memory LRU layer caches HTML for dynamic pages—those containing request-dependent nodes that vary by query parameters or visitor context.

Located in server/publish/renderCache.ts, this layer uses a versioned key structure:

// Cache key: (urlPath, canonicalQuery)
// Normalized by canonicalRenderQuery() which strips non-loop pagination params

Cache entries automatically expire when bumpPublishVersion() increments the global publishVersion after a new publish. This lazy eviction ensures stale content is never served without requiring a full cache flush.

The Layer B flow:

1. publicRouter.ts normalizes the incoming query with canonicalRenderQuery()
2. For requests with loop pagination (e.g., ?loop_posts_page=2), check the LRU
3. Cache hit: Return HTML instantly
4. Cache miss: Trigger single-flight render via publishPage, store result, return response

Single-flight rendering prevents thundering herd—concurrent requests for the same uncached page coordinate through publishState.ts so only one render executes.

Layer C: Dynamic-Island Runtime (Client Hydration)

The dynamic-island runtime hydrates placeholders with personalized content after the initial page load.

During Layer A, renderNode.ts replaces dynamic nodes with:

<instatic-hole data-instatic-hole-id="abc123" data-instatic-version="42">
  <!-- ~1KB IntersectionObserver script -->
</instatic-hole>

The client-side runtime from server/publish/holeRuntime.ts observes these elements:

new IntersectionObserver((entries) => {
  entries.forEach(async (e) => {
    if (e.isIntersecting) {
      const nodeId = e.target.dataset.instaticHoleId;
      const version = e.target.dataset.instaticVersion;
      const url = `/_instatic/hole/${nodeId}?v=${version}&u=${location.pathname}`;
      const res = await fetch(url);
      e.target.replaceWith(await res.text());
    }
  });
}).observe(document.querySelectorAll('instatic-hole'));

Server-side, the /_instatic/hole/<nodeId> endpoint re-renders just that subtree using the same publishPage logic, optionally bypassing Layer B for truly per-visitor fragments.

Dynamic Detection: What Becomes a Hole?

dynamicDetection.ts applies four classification rules (plus one promotion rule):

Rule	Condition
1	Module flagged dynamic: true
2	Node has dynamicBindings reading request params
3	Loop source declared requestDependent or perVisitor
4	Visual Component containing any dynamic node
3.5	Promotion: Static loop with dynamic child → entire loop becomes one hole

Rule 3.5 prevents duplicate placeholders and reduces client-side fetch overhead.

End-to-End Publish Flow

A complete site publish orchestrates all three layers:

// Triggered via POST /admin/api/cms/publish/site
import { publishDraftSite } from '@/server/publish/publishSite';

// 1. Write SiteDocument snapshot
// 2. For each page: publishPage() walks tree, detects dynamic nodes, emits HTML
// 3. Post-process: publishedHtmlPipeline.ts (DOMPurify, CSP, asset injection)
// 4. Atomic write to disk via staticArtefact.ts (Layer A)
// 5. bumpPublishVersion() expires Layer B entries

Request-time routing chains the layers:

• No pagination params → Serve disk file (Layer A fast-path)
• Pagination present → Layer B LRU → miss triggers single-flight render
• Hole hydration → Layer C runtime fetches /_instatic/hole/<nodeId>

Asset Pipeline Integration

Published assets are version-hashed for immutable caching:

• CSS: Four bundles (reset, framework, style, userStyles) deduped via CssCollector
• JS: Per-module scripts collected in renderAccumulators.jsMap

All assets receive Cache-Control: immutable headers—browsers cache forever, invalidated only by filename change on new publishes.

Code Examples

Triggering a Full Site Publish

// POST /admin/api/cms/publish/site
await fetch('/admin/api/cms/publish/site', { method: 'POST' });

The handler in server/publish/publishSite.ts performs the complete Layer A + Layer B sequence.

Accessing the Layer B Cache Directly

import { renderCache } from '@/server/publish/renderCache';

const cached = await renderCache.getOrRender(
  { urlPath: '/blog', queryString: '?loop_posts_page=2' },
  async () => {
    const { html } = await publishPage(blogPage, site, registry);
    return { body: html, headers: {}, status: 200 };
  },
);

Programmatic Page Render (Layer A Entry Point)

import { publishPage } from '@/core/publisher/render';
import { site, registry } from '@/server/context';

const { html, filename } = await publishPage(page, site, registry);
// html contains full document or static shell with <instatic-hole>

Summary

• Layer A (Disk Bake): Atomic filesystem writes via two-slot symlink swap; serves static files directly
• Layer B (In-Memory LRU): Versioned cache for dynamic pages with lazy eviction on publish; uses canonicalRenderQuery for cache key normalization
• Layer C (Dynamic-Island Runtime): Client-side IntersectionObserver hydrates <instatic-hole> placeholders from /_instatic/hole/<nodeId> endpoints
• Dynamic Detection: Four-rule classification in dynamicDetection.ts with loop promotion prevents duplicate holes
• Single-Flight Rendering: Prevents stampede through coordination in publishState.ts

Frequently Asked Questions

What triggers invalidation of the Layer B cache?

The bumpPublishVersion() function in server/publish/publishState.ts increments a global version counter. Cache keys incorporate this version, so existing entries become unreachable—new requests miss and trigger fresh renders with the updated version.

Why use a two-slot symlink swap instead of direct file writes?

Direct writes risk serving half-written HTML if a request arrives mid-write. The two-slot pattern writes to a staging path, then atomically swaps the symlink target—readers always see complete files, and rollbacks are instant by reverting the symlink.

Can dynamic islands be server-rendered without client JavaScript?

No—the <instatic-hole> architecture requires client-side JavaScript for hydration. For fully static output, ensure no nodes trigger the dynamic detection rules. The runtime script (~1KB) is injected only on pages containing holes.

How does canonical query normalization affect caching?

canonicalRenderQuery() in the router strips parameters unrelated to loop pagination. This collapses equivalent URLs (e.g., tracking parameters) into single cache entries, improving hit rates while preserving intentional variations like ?loop_posts_page=2.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}