Zero-Copy Asset Management During Development in Astro Big Doc

Zero-copy asset management streams files directly from the source content/ directory via API endpoints during development, eliminating disk duplication by serving assets straight from disk rather than copying them to the build output.

In the microwebstacks/astro-big-doc framework, zero-copy asset management optimizes development server performance by avoiding redundant file copies. When running in development mode, the framework serves images and static files through dynamic API routes that read directly from the source tree. This architecture ensures that large asset collections do not bloat the build directory while maintaining fast hot-reload times.

Development Mode vs. Production: The Copy Assets Configuration

The framework toggles between zero-copy streaming and physical file copying via the config.copy_assets setting. In config.js, this value defaults to false for development, enabling the streaming behavior.

• Development (copy_assets: false): Assets remain in src/content/ and are served via API routes at /assets/....
• Production (copy_assets: true): The relAssetToUrlCopy() function copies files into the output directory with hashed filenames for cache-busting.

This conditional logic lives in src/libs/assets.js, where assetToUrl() acts as the router between streaming and copying strategies.

How Asset URLs Route to API Endpoints in Development

When copy_assets is disabled, the assetToUrl(path, dir) helper constructs URLs pointing to the internal API endpoint rather than static file paths.

// src/libs/assets.js – assetToUrl logic
if (config.copy_assets) {
    return await config.base + relAssetToUrlCopy(relativepath, dirpath);
}
// Zero-copy path: handled by /assets/[...path].js
const newurl = join("assets", dirpath, relativepath);
return config.base + "/" + newurl.replaceAll('\\','/');

The resulting URL pattern /assets/<directory>/<filename> maps directly to the dynamic route handler. This design ensures that browser requests for images trigger the streaming endpoint instead of looking for files in the dist/ folder.

Streaming Assets with Zero-Copy API Routes

The dynamic route at src/pages/assets/[...path].js handles all asset requests during development. Its GET handler resolves the physical file location and streams content using Node.js read streams.

// src/pages/assets/[...path].js – GET handler
export async function GET({params, props}) {
    // Safety check: this route is inactive when copy_assets is enabled
    if (config.copy_assets) {
        return new Response('Not supported …', {status: 404});
    }

    // Resolve source file path from content directory
    let imagePath = resolve(join(config.content_path, params.path));
    imagePath = remove_base(imagePath);
    if (props.asset?.path?.startsWith("/") && props.asset?.exists) {
        imagePath = props.asset.abs_path;   // Handle linked assets
    }

    // Zero-copy: stream directly from disk
    const stream = createReadStream(imagePath);
    const contentType = file_mime(imagePath);
    return new Response(stream, {
        status: 200, 
        headers: {'Content-Type': contentType}
    });
}

Key implementation details:

• createReadStream pipes the file directly to the response without loading the entire buffer into memory.
• file_mime derives the Content-Type header from the file extension.
• config.content_path anchors the resolution to the source directory, preventing directory traversal outside the project.

Static Path Generation for Asset Discovery

For Astro to recognize valid routes during development, the getStaticPaths() function exports all available assets as static paths. This reads from a pre-generated manifest rather than scanning the filesystem on every request.

// src/pages/assets/[...path].js – getStaticPaths
export async function getStaticPaths() {
    if (config.copy_assets) return [];

    const asset_list = await load_json_abs(join(
        config.collect_content.outdir, 'asset_list.json'));

    // Filter valid assets (exclude external links)
    const assets = asset_list.filter(asset => (
        (asset.type !== "link" && Object.hasOwn(asset, "path")) ||
        (asset.type === "link" && !asset.external && asset.filter_ext)
    ));

    return assets.map(asset => ({
        params: {path: asset.path},
        props: {asset}
    }));
}

The asset_list.json file is generated during the content collection build step, providing a lightweight index that the development server consults once at startup. This approach avoids expensive filesystem operations while serving requests.

Zero-Copy Pattern for Code Diagrams

The same zero-copy architecture applies to dynamically generated diagrams through src/pages/codes/[...path].js. This endpoint streams SVG and code text files from config.code_path using identical createReadStream logic, ensuring that generated visualizations also avoid unnecessary disk duplication during development.

Summary

• Zero-copy asset management serves files directly from src/content/ via API routes when config.copy_assets is false (development default).
• assetToUrl() in src/libs/assets.js generates /assets/... URLs that route to the streaming endpoint.
• createReadStream in the API handler enables true zero-copy streaming without loading files into memory or duplicating them to the build directory.
• getStaticPaths() populates valid routes by reading asset_list.json, ensuring the development server knows which assets are available without filesystem scanning.
• Production builds switch to relAssetToUrlCopy(), physically copying and hashing files for static deployment and CDN caching.

Frequently Asked Questions

What happens if copy_assets is set to true during development?

The API endpoint immediately returns a 404 response with the message "Not supported," as the streaming routes are designed exclusively for zero-copy development mode. When copy_assets is enabled, assetToUrl() bypasses the API routes entirely and instead copies files to the output directory, generating hashed filenames suitable for production caching strategies.

How does the framework determine MIME types for streamed assets?

The file_mime() utility function in src/libs/assets.js inspects the file extension of the requested path and maps it to the appropriate Content-Type header. This ensures browsers correctly interpret images, fonts, JSON files, and other static assets even though they are being streamed dynamically rather than served as static files.

Can external linked assets be served via the zero-copy API?

Yes, the GET handler includes specific logic for linked assets through the props.asset object. When an asset has path.startsWith("/") and exists is true, the handler uses props.asset.abs_path as the file location instead of resolving against config.content_path. This supports symlinks or assets referenced outside the standard content tree while maintaining the zero-copy streaming behavior.

Why use zero-copy streaming instead of copying files during development?

Zero-copy streaming eliminates redundant disk I/O and conserves storage space by keeping a single source of truth in the content/ directory. This approach significantly speeds up startup times for sites with large asset libraries and ensures that file changes are immediately reflected without waiting for copy operations to complete, supporting rapid iteration during development.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}