Can JSAR Be Integrated with Unity or Unreal Engine? A Complete Technical Guide

Yes, JSAR integrates with Unity through a production-ready SDK and supports Unreal Engine via its native C++ embedder API, enabling seamless mixed-reality rendering in both engines.

The JSAR Runtime (Spatial Web Browser Engine) from m-creativelab/jsar-runtime is architected as an embeddable library designed to render spatial web content inside 3D game engines. While Unity developers benefit from a fully-featured UPM package with C# bindings, Unreal Engine developers can leverage the engine-agnostic native API to achieve the same framebuffer-shared rendering capabilities.

Unity Integration (Production-Ready)

Unity developers can integrate JSAR immediately using the official Unity SDK, which provides a high-level C# bridge and ready-to-use prefabs.

The Transmute Component and C# Bridge

The Unity SDK exposes functionality through the Transmute MonoBehaviour class. This component handles lifecycle management and provides static methods to marshal input events from Unity into the JSAR runtime.

According to the integration guide in docs/manual/runtime/integrate-runtime-with-unity.md, the Transmute script offers methods like DispatchMouseMoveEvent and DispatchHandTracking that convert Unity's input data into JSAR's internal event system. The accompanying TransmuteBow.prefab provides a pre-configured GameObject that developers can drag into scenes to get started immediately.

Platform Support and Rendering Pipeline

The Unity integration currently supports Windows, macOS (both Intel and Apple Silicon), and Android arm64. It works with Unity's Built-in Render Pipeline, with Universal Render Pipeline (URP) support planned for future releases.

The rendering architecture uses a framebuffer-shared approach where JSAR batches all draw calls into ≤10 GPU commands and renders directly into the host engine's framebuffer. This design enables correct depth testing between DOM elements and native Unity geometry.

Unreal Engine Integration (Native API)

While Unity enjoys a dedicated loader, Unreal Engine integration requires working directly with the native embedder layer—though the architectural path is clearly documented.

Using the Engine-Agnostic Native Embedder

As noted in docs/manual/references/embedder-guides/native-apis.md, the native C++ embedder API explicitly mentions Unreal Engine support. The repository's README lists jsar-loader-unreal as Not started, meaning no high-level plugin exists yet, but the underlying infrastructure supports implementation today.

The embedder API provides a RenderAPI factory that creates platform-specific rendering backends. For Unreal integration, developers instantiate this API using constants like kUnrealGfxRendererOpenGLCore (or Vulkan/Metal equivalents) and integrate the update loop into Unreal's Tick() system.

Implementing the RenderAPI for Unreal

The core integration pattern requires three steps:

1. Create the RenderAPI: Call RenderAPI::Create() with the appropriate graphics renderer constant for your Unreal project
2. Initialize the Runtime: Instantiate JSARRuntime with the API and load XSML content using runtime.LoadXSML()
3. Per-Frame Updates: Invoke api->Update() inside Unreal's game loop to synchronize spatial web rendering with native frames

The factory implementation in src/renderer/render_api.cpp handles the creation of platform-specific backends including OpenGL ES 3, Vulkan, Metal, and Direct3D variants.

Architectural Overview: Shared Framebuffer Design

JSAR's embedding architecture follows a four-layer model that remains consistent across both engines:

• Engine Host: Provides graphics context (OpenGL ES 3, Metal, Vulkan) and per-frame hooks. In Unity, the Transmute component calls JSARRuntime.Update() inside MonoBehaviour.Update(). In Unreal, developers invoke api->Update() within Tick().

• JSAR Core: Handles Web standards (HTML, CSS, WebGL/WebGPU, WebXR) and spatializes DOM elements. This layer compiles into native libraries (libjsar_runtime.so or jsar_runtime.dll).

• Bridge Layer: Marshals input events (mouse, hand-tracking, rays) between engine and runtime. Unity uses static C# methods; Unreal uses direct C++ calls on the RenderAPI instance.

• Rendering Backend: Batches draw calls and shares the host framebuffer. Both engines can render opaque and transparent passes together, enabling mixed-reality scenes where HTML divs correctly occlude native 3D objects and vice versa.

Code Examples

Unity C# Implementation with Transmute

using UnityEngine;
using JSAR;  // Namespace from the UPM package

public class JsarExample : MonoBehaviour
{
    public string xsmlPath = "https://example.com/app.xsml";

    void Start()
    {
        // Load JSAR's HTML-like format at runtime
        Transmute.ScriptName = xsmlPath;
        Transmute.AutoStart = true;
    }

    void Update()
    {
        // Forward Unity mouse input to JSAR
        Vector3 mouse = Input.mousePosition;
        Transmute.DispatchMouseMoveEvent(mouse);
    }

    void SendHandTracking()
    {
        Vector3[] joints = new Vector3[21]; // Populate with tracking data
        Pose handPose = new Pose(Vector3.zero, Quaternion.identity);
        Transmute.DispatchHandTracking(0, joints, handPose, 1, 0, false);
    }
}

The Transmute.cs script referenced above ships with the Unity SDK package under Scripts/Transmute.cs, while TransmuteBow.prefab provides a ready-to-use GameObject configuration.

Unreal Engine C++ Integration

#include "RenderAPI.h"
#include "JSARRuntime.h"

int main()
{
    // Create RenderAPI for Unreal's graphics backend
    RenderAPI* api = RenderAPI::Create(
        kUnrealGfxRendererOpenGLCore, 
        GetUnrealConstellation()
    );

    // Initialize JSAR runtime
    JSARRuntime runtime(api);
    runtime.LoadXSML("https://example.com/app.xsml");
    runtime.Start();

    while (engineIsRunning())
    {
        // Called each Tick() in Unreal
        runtime.Update();
        
        // Optional: Dispatch raw input
        // runtime.DispatchMouseMove(...);
    }

    delete api;
}

This pattern matches the embedder documentation in docs/manual/references/embedder-guides/native-apis.md, which specifies that developers must supply the framebuffer handle to JSAR using the same mechanism as the Unity integration.

Summary

• Unity integration is production-ready via the official UPM package, offering the Transmute component, prefab-based setup, and support for Windows, macOS, and Android arm64.
• Unreal Engine integration requires using the native C++ embedder API—specifically RenderAPI::Create() and manual Update() calls—since the high-level jsar-loader-unreal remains unimplemented.
• Both engines utilize framebuffer-shared rendering, enabling correct depth testing and blending between spatial web content and native 3D geometry.
• The architecture separates engine-specific input marshaling from the core runtime, allowing consistent Web standards support across platforms.

Frequently Asked Questions

Is there an official Unreal Engine plugin for JSAR?

No, as of the current codebase, jsar-loader-unreal is listed as Not started in the repository README. However, the native C++ embedder API explicitly supports Unreal Engine integration, and developers can embed JSAR today by implementing the RenderAPI interface and calling api->Update() within Unreal's game loop.

What rendering pipelines does JSAR support in Unity?

JSAR currently supports Unity's Built-in Render Pipeline. Support for the Universal Render Pipeline (URP) is planned but not yet available. The integration works on Windows, macOS (Intel and Apple Silicon), and Android arm64 platforms.

Can JSAR content depth-test against native 3D objects?

Yes. JSAR uses a shared framebuffer architecture that batches all spatial web draw calls into ≤10 GPU commands and renders directly into the host engine's framebuffer. This allows DOM elements to correctly occlude native Unity or Unreal geometry and vice versa, essential for mixed-reality applications.

Which platforms are supported for Unity integration?

The Unity SDK supports Windows, macOS (both Intel and Apple Silicon architectures), and Android arm64. The native embedder API supports additional graphics backends including OpenGL ES 3, Vulkan, Metal, and Direct3D, suggesting future platform expansion for Unreal Engine implementations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}