How to Use GPU Surfaces with Metal for Custom Rendering in the Native SDK

The Native SDK's ViewKind.gpu_surface gives you direct Metal access through an MTKView on macOS, enabling custom GPU rendering that composites seamlessly with native controls and the embedded WebView.

The vercel-labs/native repository provides a first-class GPU surface API so you can use GPU surfaces with Metal for custom rendering in Native SDK applications without sacrificing the declarative shell layout or WebView integrations. By declaring a gpu_surface view in your shell manifest, the runtime allocates a Metal-backed MTKView, forwards platform events, and drives the frame lifecycle from a single event loop.

Declare a Metal GPU Surface in the Shell Manifest

Add a ShellView entry with kind = .gpu_surface and gpu_backend = .metal to tell the runtime to create a Metal view. Additional parameters such as gpu_pixel_format, gpu_present_mode, and gpu_vsync configure the surface format and presentation behavior.

In examples/gpu-surface/src/main.zig, the manifest defines an animated Metal surface side-by-side with a WebView:

const shell_views = [_]native_sdk.ShellView{
    .{ .label = "toolbar", .kind = .toolbar, .edge = .top, .height = 52, .layer = 20 },
    .{ .label = "body", .kind = .split, .fill = true, .axis = .row },
    .{
        .label = "canvas",
        .kind = .gpu_surface,
        .parent = "body",
        .width = 680,
        .min_width = 480,
        .layer = 10,
        .role = "Animated Metal surface",
        .accessibility_label = "Animated GPU surface",
        .gpu_backend = .metal,
        .gpu_pixel_format = .bgra8_unorm,
        .gpu_present_mode = .timer,
        .gpu_alpha_mode = .@"opaque",
        .gpu_color_space = .srgb,
        .gpu_vsync = true,
    },
    .{ .label = "inspector", .kind = .webview, .parent = "body", .url = "zero://inline", .fill = true },
};

This declaration is parsed by src/tooling/manifest.zig and code-generated through src/tooling/templates.zig before the runtime instantiates the view in src/runtime/window_view_runtime.zig.

Handle GPU Surface Events at Runtime

When the platform fires a frame, resize, or input event, the runtime dispatches it through RuntimeGpuSurfaceEvents. Your application implements an event handler that matches on gpu_surface_frame, gpu_surface_resized, and gpu_surface_input.

The example application in examples/gpu-surface/src/main.zig demonstrates handling these events:

fn event(context: *anyopaque, runtime: *native_sdk.Runtime, ev: native_sdk.Event) anyerror!void {
    const self = @ptrCast(@alignCast(context));
    switch (ev) {
        .gpu_surface_frame => |frame| {
            if (std.mem.eql(u8, frame.label, "canvas") and self.gpu_frame_count == 0) {
                self.gpu_frame_count = frame.frame_index + 1;
                try runtime.updateView(frame.window_id, "status-label",
                    .{ .text = "First GPU frame received." });
            }
        },
        .gpu_surface_resized => |resize| {
            if (std.mem.eql(u8, resize.label, "canvas")) {
                // resize.gpu_size contains the new Metal texture size
                // allocate or update your custom Metal resources here
            }
        },
        .gpu_surface_input => |input| {
            // Optional: forward pointer or keyboard input to your Metal renderer
        },
        else => {},
    }
}

Each gpu_surface_frame event carries the current size, scale factor, timestamps, and backend identifier, letting you update animations or UI state exactly when the platform is ready to present.

Inject Custom Metal Commands Into the Frame Pipeline

Behind the scenes, src/runtime/gpu_surface_events.zig implements dispatchGpuSurfaceFrame, where the runtime updates view state and prepares a CanvasFrame. The SDK converts high-level canvas commands into a Metal command buffer via CanvasFrameMethods().planCanvasFrameForView.

If you need to insert your own Metal work, the architecture of dispatchGpuSurfaceFrame allows you to augment the command buffer after the SDK has recorded its canvas commands but before the event is re-dispatched to the platform:

pub fn dispatchGpuSurfaceFrame(self: *Runtime, app: runtime_api.App(Runtime), evt: platform.GpuSurfaceFrameEvent) anyerror!void {
    // … existing SDK logic …
    // After the SDK has prepared the CanvasFrame, insert custom Metal commands:
    const cmd_buf = try self.gpuCommandBufferForView(evt.label);
    // e.g. draw a custom triangle
    try cmd_buf.encodeDrawTriangle(...);
    // Finally hand the enriched frame to the platform:
    try self.dispatchEvent(app, .{ .gpu_surface_frame = evt });
}

This integration point lives alongside enrichGpuSurfaceFrameDiagnostics in src/runtime/gpu_surface_events.zig, ensuring that custom rendering and the built-in canvas system share the same Metal presentation loop.

Key Source Files for GPU Surface Rendering

Understanding the following files clarifies how a shell declaration becomes a presented Metal frame:

• examples/gpu-surface/src/main.zig — Complete sample app that defines a gpu_surface shell view and handles gpu_surface_* events.
• src/runtime/gpu_surface_events.zig — Core runtime logic that processes platform events through dispatchGpuSurfaceFrame and drives the Canvas-to-Metal pipeline.
• src/runtime/window_view_runtime.zig — Wires ViewKind.gpu_surface into the window hierarchy and links the platform MTKView to the runtime.
• src/tooling/templates.zig — Generates manifest boilerplate for GPU surfaces.
• src/tooling/manifest.zig — Parses gpu_surface capabilities and view properties from the app manifest.

Summary

• Declare a .gpu_surface view with .gpu_backend = .metal and pixel-format options in your shell manifest to allocate a native MTKView.
• The platform emits gpu_surface_frame, gpu_surface_resized, and gpu_surface_input events that your Zig app handles to drive animations and resource updates.
• The runtime prepares a CanvasFrame for each view and converts it to a Metal command buffer inside dispatchGpuSurfaceFrame in src/runtime/gpu_surface_events.zig.
• Advanced integrations can interleave custom Metal commands into that same command buffer before presentation, keeping your renderer in sync with the SDK's built-in canvas and WebView compositing.

Frequently Asked Questions

Which platforms support GPU surfaces in the Native SDK?

The Native SDK maps ViewKind.gpu_surface to an AppKit MTKView on macOS and the equivalent Metal view on iOS. Because the surface is a first-class native view, it composites automatically with sibling WebViews, toolbars, and split panes in the same window hierarchy.

How do I configure the Metal pixel format and presentation mode?

In the shell manifest, set fields such as .gpu_pixel_format = .bgra8_unorm, .gpu_present_mode = .timer, and .gpu_vsync = true on the gpu_surface view. These values are parsed by src/tooling/manifest.zig and honored by the platform layer when it creates the MTKView.

Can I mix the built-in Canvas API with my own Metal rendering?

Yes. The runtime drives the full frame lifecycle. It first builds a CanvasFrame and translates canvas commands into Metal via CanvasFrameMethods().planCanvasFrameForView. You can then inject additional encoder commands inside dispatchGpuSurfaceFrame in src/runtime/gpu_surface_events.zig before the command buffer is committed.

Where does the Native SDK create the actual Metal view?

The macOS platform bridge creates the underlying MTKView implicitly and attaches it to the window. The runtime side of this connection is managed in src/runtime/window_view_runtime.zig, which adds the gpu_surface to the view hierarchy and ensures frame events reach RuntimeGpuSurfaceEvents.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}