# How the 3D Graph Visualization UI Renders and Interacts with the Knowledge Graph

> Learn how the 3D graph visualization UI renders an interactive knowledge graph using React Three Fiber. Explore nodes, edges, and camera animations at DeusData codebase-memory-mcp.

- Repository: [Martin Vogel/codebase-memory-mcp](https://github.com/DeusData/codebase-memory-mcp)
- Tags: internals
- Published: 2026-07-17

---

**The 3D graph visualization UI renders the knowledge graph using React Three Fiber to create an interactive WebGL scene where nodes appear as glowing instanced spheres and edges render as additive line segments, with camera animation and hover interactions handled through a coordinated React component architecture.**

The knowledge graph in `DeusData/codebase-memory-mcp` represents code relationships as interconnected nodes and edges that mirror your repository's structure. The frontend transforms this structured JSON data into an immersive 3D visualization using a React-based Three.js implementation. This article explains exactly how the 3D graph visualization UI interacts with the knowledge graph through its component hierarchy, data flow, and rendering pipeline.

## Data Loading and State Management

The interaction between the UI and the knowledge graph begins with data ingestion. The `useGraphData` hook in [`graph-ui/src/hooks/useGraphData.ts`](https://github.com/DeusData/codebase-memory-mcp/blob/main/graph-ui/src/hooks/useGraphData.ts) fetches the layout JSON from the backend endpoint `/api/layout` and stores it in React state.

```typescript
// Inside graph-ui/src/components/GraphTab.tsx
const { data } = useGraphData();               // Fetch JSON from /api/layout
const filtered = computeFiltered(data);        // Apply label and edge filters

```

The `GraphTab` component ([`graph-ui/src/components/GraphTab.tsx`](https://github.com/DeusData/codebase-memory-mcp/blob/main/graph-ui/src/components/GraphTab.tsx)) orchestrates this data flow, managing filter state, selection handling, and camera targeting before passing the processed `GraphData` to the rendering layer. This separation ensures the visualization components remain pure rendering functions while the container handles business logic.

## The Rendering Pipeline Architecture

The visualization uses **React Three Fiber** as the core renderer, with supporting utilities from `@react-three/drei` and `@react-three/postprocessing`. The pipeline follows this sequence:

1. **Canvas Creation** – `GraphScene` sets up the WebGL context
2. **Edge Rendering** – `EdgeLines` draws relationship lines
3. **Node Rendering** – `NodeCloud` instances sphere geometries
4. **Label Overlay** – `NodeLabels` adds billboarded text
5. **Post-Processing** – Bloom effects create the glowing aesthetic

### Scene Setup and Configuration

The `GraphScene` component in [`graph-ui/src/components/GraphScene.tsx`](https://github.com/DeusData/codebase-memory-mcp/blob/main/graph-ui/src/components/GraphScene.tsx) initializes the `<Canvas>` with a dark background, ambient lighting, point lights, and the `EffectComposer` for post-processing. It also instantiates `CameraAnimator` and `IdleAutoRotate` for interactive camera behavior.

```tsx
<Canvas ...>
  <ambientLight intensity={0.5} />
  <pointLight position={[10, 10, 10]} />
  <EdgeLines nodes={data.nodes} edges={data.edges} />
  <NodeCloud nodes={data.nodes} highlightedIds={highlightedIds} />
  {showLabels && <NodeLabels nodes={data.nodes} />}
  <EffectComposer><Bloom ... /></EffectComposer>
</Canvas>

```

### Edge Visualization with BufferGeometry

Edges render as additive line segments for a glowing effect. The `EdgeLines` component in [`graph-ui/src/components/EdgeLines.tsx`](https://github.com/DeusData/codebase-memory-mcp/blob/main/graph-ui/src/components/EdgeLines.tsx) constructs a `Float32Array` of vertex positions and colors, creates a `THREE.BufferGeometry`, and renders with `<lineSegments>` using `THREE.AdditiveBlending`.

```tsx
// From graph-ui/src/components/EdgeLines.tsx
const geometry = useMemo(() => {
  const positions = new Float32Array(edges.length * 6); // 2 vertices * 3 coords
  const colors = new Float32Array(edges.length * 6);
  
  edges.forEach((edge, i) => {
    // Map edge type to color (e.g., CALLS → #1DA27E)
    const color = new THREE.Color(getEdgeColor(edge.type));
    // ... populate positions and colors arrays
  });
  
  const geo = new THREE.BufferGeometry();
  geo.setAttribute('position', new THREE.BufferAttribute(positions, 3));
  geo.setAttribute('color', new THREE.BufferAttribute(colors, 3));
  return geo;
}, [edges]);

```

This approach handles normal edges, cross-project edges, and opacity adjustments while maintaining high performance even with thousands of connections.

### Node Visualization with InstancedMesh

Nodes render as glowing "stars" using a single `InstancedMesh` of spheres. The `NodeCloud` component in [`graph-ui/src/components/NodeCloud.tsx`](https://github.com/DeusData/codebase-memory-mcp/blob/main/graph-ui/src/components/NodeCloud.tsx) sets position, scale, and per-instance color attributes each frame.

```tsx
// From graph-ui/src/components/NodeCloud.tsx
const meshRef = useRef<THREE.InstancedMesh>(null);

useFrame(() => {
  if (!meshRef.current) return;
  
  for (let i = 0; i < nodes.length; i++) {
    const node = nodes[i];
    dummy.position.set(node.x, node.y, node.z);
    dummy.scale.setScalar(node.size || 1);
    dummy.updateMatrix();
    meshRef.current.setMatrixAt(i, dummy.matrix);
    
    // Apply bloom boost for highlighted nodes
    const brightness = highlightedIds.has(node.id) ? 1.5 : 1.0;
    tempColor.set(node.color).multiplyScalar(brightness);
    meshRef.current.setColorAt(i, tempColor);
  }
  
  meshRef.current.instanceMatrix.needsUpdate = true;
  if (meshRef.current.instanceColor) {
    meshRef.current.instanceColor.needsUpdate = true;
  }
});

```

The instanced approach ensures the UI remains responsive even when rendering tens of thousands of nodes, as implemented in the `DeusData/codebase-memory-mcp` source code.

### Label Rendering with Billboards

The `NodeLabels` component in [`graph-ui/src/components/NodeLabels.tsx`](https://github.com/DeusData/codebase-memory-mcp/blob/main/graph-ui/src/components/NodeLabels.tsx) renders text labels for selected or largest nodes using Drei's `<Billboard>` helper, ensuring labels always face the camera regardless of rotation.

```tsx
// From graph-ui/src/components/NodeLabels.tsx
<Billboard>
  <Text
    fontSize={0.5}
    outlineWidth={0.05}
    outlineColor="#000000"
  >
    {node.label}
  </Text>
</Billboard>

```

Labels include a black outline for readability against the glowing background.

## Interaction Systems and Camera Control

The 3D graph visualization UI supports multiple interaction modes through specialized components within [`GraphScene.tsx`](https://github.com/DeusData/codebase-memory-mcp/blob/main/GraphScene.tsx).

**Hover Detection**: The `NodeTooltip` component in [`graph-ui/src/components/NodeTooltip.tsx`](https://github.com/DeusData/codebase-memory-mcp/blob/main/graph-ui/src/components/NodeTooltip.tsx) displays a floating info card when the pointer intersects a node instance, using raycasting against the instanced mesh.

**Camera Animation**: The `CameraAnimator` interpolates the camera's `position` and `lookAt` toward targets computed by `computeCameraTarget`, which averages selected node positions and derives appropriate viewing distances. This enables smooth transitions when users click nodes in the sidebar.

**Idle Rotation**: The `IdleAutoRotate` component enables automatic slow rotation after a period of inactivity, keeping the visualization alive while users read documentation.

## Customizing Node Appearance

You can extend the visualization by modifying the color generation logic in `NodeCloud`. For example, to color every node whose label starts with "Auth" in orange:

```tsx
// In graph-ui/src/components/NodeCloud.tsx
const colors = useMemo(() => {
  const arr = new Float32Array(nodes.length * 3);
  for (let i = 0; i < nodes.length; i++) {
    const node = nodes[i];
    tempColor.set(node.color);
    
    // Custom rule
    if (node.label.startsWith('Auth')) {
      tempColor.set('#ff8800');          // Orange override
    }
    
    // Apply highlight dimming/bloom boost
    const opacity = highlightedIds.size > 0 && !highlightedIds.has(node.id) ? 0.3 : 1.0;
    arr[i * 3] = tempColor.r * opacity;
    arr[i * 3 + 1] = tempColor.g * opacity;
    arr[i * 3 + 2] = tempColor.b * opacity;
  }
  return arr;
}, [nodes, highlightedIds]);

```

Because the component updates per-frame via `useFrame`, changes appear instantly without requiring additional re-renders.

## Summary

- **Data Flow**: The `useGraphData` hook fetches layout JSON from `/api/layout`, processed by `GraphTab` and rendered by `GraphScene`.
- **Edge Rendering**: `EdgeLines` uses `BufferGeometry` with `AdditiveBlending` to create glowing relationship lines mapped from edge types.
- **Node Rendering**: `NodeCloud` employs a single `InstancedMesh` with per-instance color buffers to render thousands of nodes efficiently.
- **Post-Processing**: The `EffectComposer` with `Bloom` creates the characteristic glowing aesthetic for highlighted nodes.
- **Interactions**: `CameraAnimator` handles smooth transitions to selected nodes, while `NodeTooltip` provides hover context.
- **Performance**: The architecture maintains 60fps even with large knowledge graphs through instanced rendering and efficient geometry updates.

## Frequently Asked Questions

### What technology stack powers the 3D graph visualization?

The visualization uses **React Three Fiber** as the React renderer for **Three.js**, with **@react-three/drei** providing utility components like `Billboard` and `Text`, and **@react-three/postprocessing** handling the bloom effects. The backend provides layout data as JSON through the `/api/layout` endpoint, defined in [`graph-ui/src/lib/types.ts`](https://github.com/DeusData/codebase-memory-mcp/blob/main/graph-ui/src/lib/types.ts).

### How does the UI handle large knowledge graphs with thousands of nodes?

The system uses **instanced rendering** via `InstancedMesh` in [`NodeCloud.tsx`](https://github.com/DeusData/codebase-memory-mcp/blob/main/NodeCloud.tsx) to draw all nodes with a single GPU draw call rather than individual meshes. Edges use merged `BufferGeometry` in [`EdgeLines.tsx`](https://github.com/DeusData/codebase-memory-mcp/blob/main/EdgeLines.tsx). This architecture ensures the UI remains responsive even when visualizing tens of thousands of nodes and edges.

### How are node colors and edge colors determined?

Edge colors map to relationship types (e.g., `CALLS` edges render as `#1DA27E`) as implemented in [`EdgeLines.tsx`](https://github.com/DeusData/codebase-memory-mcp/blob/main/EdgeLines.tsx). Node colors come from the backend layout data, with brightness multipliers applied in [`NodeCloud.tsx`](https://github.com/DeusData/codebase-memory-mcp/blob/main/NodeCloud.tsx) to create bloom effects for highlighted nodes. You can customize these by modifying the color generation logic before the instanced buffer attributes are populated.

### Can the camera automatically focus on specific nodes?

Yes. The `CameraAnimator` component in [`GraphScene.tsx`](https://github.com/DeusData/codebase-memory-mcp/blob/main/GraphScene.tsx) interpolates the camera position toward targets computed by `computeCameraTarget`. When users select nodes via the `Sidebar` component or click directly in the scene, the camera smoothly transitions to center those nodes in the viewport, calculating appropriate distance based on the selection's bounding sphere.