Instatic's Three-Layer Publishing Pipeline for Static and Dynamic Content

Instatic renders a page tree into clean HTML through a three-layer pipeline that bakes static content to disk, caches dynamic renders in memory, and lazily hydrates request-dependent fragments via client-side islands.

The CoreBunch/Instatic repository implements a sophisticated publishing system that separates concerns between static generation, in-memory caching, and runtime hydration. This three-layer publishing pipeline maximizes cacheability while ensuring dynamic content remains fresh and personalized.

The Three-Layer Architecture

Instatic divides its rendering responsibilities across distinct layers, each optimized for specific performance characteristics and cache semantics.

Layer	Purpose	Key Implementation
Layer A – Disk Fast-Path	Bakes every page to the file system under uploads/published/current/<route>.html. Fully static pages are written as complete documents; pages with dynamic islands contain <instatic-hole> placeholders. Uses atomic two-slot symlink swaps for zero-downtime updates.	src/core/publisher/render.ts (entry point publishPage), server/publish/staticArtefact.ts (writes artefacts and performs symlink swap)
Layer B – In-Memory LRU	Caches rendered HTML of dynamic pages keyed by (urlPath, canonicalQuery). Versioned by publishVersion; publishes bump the version and lazily evict stale entries.	server/publish/renderCache.ts (LRU implementation), server/publish/publishState.ts (monotonic publishVersion)
Layer C – Dynamic Islands	Emits <instatic-hole> placeholders for request-dependent nodes. Client-side IntersectionObserver lazily fetches fragments from /_instatic/hole/<nodeId> and injects them.	src/core/publisher/dynamicDetection.ts (detection rules), server/publish/holeRuntime.ts (client runtime)

Layer A – Disk Fast-Path

The disk layer serves as the immutable foundation of the pipeline. When publishPage in src/core/publisher/render.ts processes a route, it determines whether the page is fully static or requires dynamic hydration. Static pages are written as complete HTML documents to uploads/published/current/<route>.html. For pages containing dynamic content, the system writes a static shell containing <instatic-hole> placeholders where dynamic content will later inject.

The server/publish/staticArtefact.ts module handles these writes using a two-slot symlink swap pattern. This ensures atomic updates—readers always see either the previous version or the new version, never a partially written file.

Layer B – In-Memory LRU

Dynamic pages—those with query parameters or dynamic islands—bypass the disk cache and hit the in-memory LRU cache. The cache key combines urlPath and canonicalQuery (generated by server/publish/loopPrefetch.ts) to ensure cache hits only occur for semantically equivalent requests.

The server/publish/renderCache.ts implementation versions each entry using the monotonic publishVersion from server/publish/publishState.ts. When a new publish occurs, the version increments, and stale entries are lazily evicted on access. This prevents mid-publish renders from poisoning the cache.

Layer C – Dynamic Islands

For content that cannot be cached—such as user-specific data or real-time information—Instatic uses dynamic islands. During the render phase, src/core/publisher/dynamicDetection.ts runs four detection rules (plus Rule 3.5 promotion) via findDynamicNodeIds to classify request-dependent nodes. These nodes render as <instatic-hole> tags containing a unique nodeId.

The server/publish/holeRuntime.ts module serves a tiny (~1 KB) client-side runtime. This runtime uses IntersectionObserver to detect when holes enter the viewport, then fetches fragments from /_instatic/hole/<nodeId>?v=<publishVersion>&u=<page-url>. The server re-runs the render pipeline for that specific subtree and returns the HTML fragment.

How the Pipeline Works

Understanding the data flow reveals how these layers interact during a publish cycle:

1. Publish Trigger: The admin API receives POST /admin/api/cms/publish/site, triggering publishDraftSite. This loads the draft site, builds runtime assets, and iterates over every page.

2. Node Tree Rendering: For each page, publishPage walks the node tree via renderNode.ts. Hidden nodes are pruned early. Specialized renderers handle base.visual-component-ref and base.loop nodes. Standard nodes invoke their module's pure render() function, with CSS deduplicated via CssCollector.

3. Dynamic Detection: dynamicDetection.ts classifies nodes requiring runtime rendering. If any exist, the page emits as a static shell with placeholders; otherwise, a full static HTML file is written.

4. Atomic Deployment: staticArtefact.ts writes the resulting HTML and CSS bundles to disk, performing the atomic symlink swap to activate the new version.

5. Cache Warming: On the first request for a dynamic page, publicRouter.ts falls back to the Layer B cache. Misses trigger a render, with results stored in the LRU cache stamped with the current publishVersion.

6. Client Hydration: The browser loads the static shell, then the hole runtime fetches and injects dynamic fragments as the user scrolls.

Key Implementation Files

The pipeline spans both core publisher logic and server-side infrastructure:

• src/core/publisher/render.ts: Main entry point publishPage; orchestrates node walking, CSS collection, and final HTML assembly.
• src/core/publisher/renderNode.ts: Recursive walker invoking specialized renderers and collecting hole IDs.
• src/core/publisher/dynamicDetection.ts: Implements detection rules (including Rule 3.5 promotion) driving Layer A and Layer C decisions.
• src/core/publisher/cssCollector.ts: Dedupes module CSS and sanitizes style bundles.
• server/publish/staticArtefact.ts: Writes baked HTML/CSS and performs atomic two-slot symlink swaps.
• server/publish/renderCache.ts: In-memory LRU cache keyed by URL and canonical query.
• server/publish/publishState.ts: Manages monotonic publishVersion and single-flight rendering.
• server/publish/holeRuntime.ts: Serves the tiny client runtime for fetching hole fragments.
• server/publish/publicRouter.ts: Gateway selecting Layer A fast-path, Layer B cache, or fallback rendering.
• server/publish/loopPrefetch.ts: Generates canonical query strings for cache keys.

Practical Code Examples

Publishing a Page Programmatically

Use the publishPage function to render pages outside the standard admin flow:

import { publishPage } from '@core/publisher';
import { siteRegistry } from '@/site/registry';

// Assume `page`, `site` and `registry` are loaded from the draft DB.
const { filename, html, jsModuleIds } = await publishPage(page, site, siteRegistry);
console.log(`Published ${filename}: ${html.length} bytes`);

Source: [src/core/publisher/render.ts](https://github.com/CoreBunch/Instatic/blob/main/src/core/publisher/render.ts)

Router Logic for Layer Selection

The publicRouter.ts implements the logic choosing between disk, cache, and fresh renders:

import { readArtefact } from 'server/publish/staticArtefact';
import { getOrRender } from 'server/publish/renderCache';
import { canonicalRenderQuery } from 'server/publish/loopPrefetch';

export async function renderPublicResolution(req) {
  const url = new URL(req.url);
  const canonical = canonicalRenderQuery(url.searchParams);
  
  // Layer A fast-path
  const staticHtml = await readArtefact(url.pathname);
  if (staticHtml && canonical === '') return staticHtml;

  // Layer B cache or fallback to render
  const { body } = await getOrRender(
    { urlPath: url.pathname, queryString: canonical }, 
    async () => {
      const { html } = await publishPage(page, site, registry);
      return { body: html, headers: { 'Content-Type': 'text/html' } };
    }
  );
  return body;
}

Source: [server/publish/publicRouter.ts](https://github.com/CoreBunch/Instatic/blob/main/server/publish/publicRouter.ts)

Client-Side Hole Runtime

The server automatically injects this runtime for pages containing dynamic islands:

<script>
  // Provided by server/publish/holeRuntime.ts as HOLE_RUNTIME_JS
  // It observes <instatic-hole> elements and fetches their content.
  runInstaticHoleRuntime();
</script>

Source: [server/publish/holeRuntime.ts](https://github.com/CoreBunch/Instatic/blob/main/server/publish/holeRuntime.ts)

Creating a Pure Module Renderer

New modules implement pure render functions compatible with the pipeline:

// src/modules/base/example/example.ts
export const ExampleModule = {
  render: (props, children) => ({
    html: `<div class="example">${children.join('')}</div>`,
    css: `.example{color:var(--example-color)}`,
  })
};
registry.registerOrReplace('base.example', ExampleModule);

Source: [src/modules/base/index.ts](https://github.com/CoreBunch/Instatic/blob/main/src/modules/base/index.ts)

Summary

• Layer A provides atomic, filesystem-based static hosting with zero-downtime deployments via symlink swaps.
• Layer B offers high-performance in-memory caching for dynamic pages, using versioned keys to ensure publish consistency.
• Layer C enables personalization through lazy-loaded dynamic islands that fetch server-rendered fragments on demand.
• The pipeline automatically classifies content using findDynamicNodeIds and the four detection rules, ensuring only necessary components trigger runtime rendering.
• All layers respect the monotonic publishVersion from publishState.ts, preventing stale content from serving during active publishes.

Frequently Asked Questions

How does Instatic decide whether to use the disk fast-path or in-memory cache?

The publicRouter.ts checks if a static artefact exists on disk and whether the request has a canonical query string. Requests with empty queries hit Layer A (disk), while requests with query parameters or cache misses route to Layer B (in-memory LRU). Dynamic pages containing <instatic-hole> tags are always served from memory or freshly rendered, bypassing the disk cache.

What triggers a cache invalidation in the Layer B LRU?

Cache invalidation occurs implicitly through versioning. Each publish increments the publishVersion stored in server/publish/publishState.ts. When getOrRender in server/publish/renderCache.ts retrieves entries, it validates them against the current version. Mismatches trigger lazy eviction and re-rendering, ensuring users never see content from previous publishes.

Why does Instatic use a two-slot symlink swap for static files?

The two-slot pattern in server/publish/staticArtefact.ts ensures atomic updates. Instead of overwriting files in place—which risks serving partial content during writes—the system writes to a staging directory and swaps symlinks. This guarantees readers always see complete, consistent files, eliminating race conditions during high-traffic publishes.

How do dynamic islands maintain security for user-specific content?

Dynamic islands render placeholders in the initial static shell, containing no sensitive data. The actual user-specific fragments are fetched via /_instatic/hole/<nodeId> endpoints that execute within the user's authenticated session. Because these fragments are rendered server-side on demand, they bypass the shared caches (Layer A and B) entirely, ensuring private data never leaks into public caches.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}