# How Voicebox Stories Editor Timeline Composition Works: A Deep Dive

> Understand Voicebox stories editor timeline composition. Learn how millisecond audio durations convert to pixel coordinates for synchronized playback using Zustand and React Query.

- Repository: [Jamie Pine/voicebox](https://github.com/jamiepine/voicebox)
- Tags: deep-dive
- Published: 2026-04-14

---

**The Voicebox stories editor timeline composition converts millisecond-based audio durations into pixel coordinates using a configurable `pixelsPerSecond` scale, rendering clips on vertical tracks with synchronized playback via a Zustand store and React Query mutations.**

The Voicebox repository (`jamiepine/voicebox`) implements a browser-based timeline editor for arranging voice-generated clips into structured stories. The stories editor timeline composition relies on a tightly-coupled React architecture that maps temporal audio data to spatial UI coordinates, enabling complex editing operations like trimming, splitting, and drag-and-drop rearrangement.

## Core Architecture Components

The timeline implementation consists of four primary layers working in concert:

- **`StoryTrackEditor`** – The main UI component located in [`app/src/components/StoriesTab/StoryTrackEditor.tsx`](https://github.com/jamiepine/voicebox/blob/main/app/src/components/StoriesTab/StoryTrackEditor.tsx) that orchestrates layout, zoom handling, mouse interactions, and playhead rendering.
- **`ClipWaveform`** – An inner component (lines 38-70) that instantiates Wavesurfer.js instances to render waveforms with trim offsets applied via CSS transforms.
- **`useStoryStore`** – A Zustand store defined in [`app/src/stores/storyStore.ts`](https://github.com/jamiepine/voicebox/blob/main/app/src/stores/storyStore.ts) that manages reactive state for clip selection, editor height, and playback timing anchors.
- **`useStories` hooks** – React Query wrappers in [`app/src/lib/hooks/useStories.ts`](https://github.com/jamiepine/voicebox/blob/main/app/src/lib/hooks/useStories.ts) that expose mutations for CRUD operations including `useMoveStoryItem`, `useTrimStoryItem`, and `useSplitStoryItem`.

## Timeline Geometry and Coordinate Mapping

### Pixel-Per-Second Scaling

The editor establishes a bidirectional mapping between milliseconds and pixels through a configurable scale factor:

```typescript
const [pixelsPerSecond, setPixelsPerSecond] = useState(DEFAULT_PIXELS_PER_SECOND);
// Where DEFAULT_PIXELS_PER_SECOND = 50 (line 30)

const handleZoomIn = () => setPixelsPerSecond(p => Math.min(p * 1.5, MAX_PIXELS_PER_SECOND));
const handleZoomOut = () => setPixelsPerSecond(p => Math.max(p / 1.5, MIN_PIXELS_PER_SECOND));

```

Zoom handlers multiply or divide the current scale by 1.5, clamping to defined minimum and maximum boundaries. This value drives all spatial calculations in the stories editor timeline composition.

### Time-to-Pixel Conversion Functions

The component memoizes conversion helpers to translate between temporal and spatial units:

```typescript
const msToPixels = useCallback((ms) => (ms / 1000) * pixelsPerSecond, [pixelsPerSecond]);
const pixelsToMs = useCallback((px) => (px / pixelsPerSecond) * 1000, [pixelsPerSecond]);

```

These functions calculate CSS `left` and `width` properties for clip positioning and convert mouse coordinates back to millisecond values during seek operations.

### Dynamic Timeline Width Calculation

The total scrollable width derives from the longest clip endpoint plus padding:

```typescript
const totalDurationMs = useMemo(() => {
  if (items.length === 0) return 10000; // default 10s
  return Math.max(...items.map(i => i.start_time_ms + getEffectiveDuration(i)), 10000);
}, [items, getEffectiveDuration]);

const contentWidth = (totalDurationMs / 1000) * pixelsPerSecond + 200; // extra padding
const timelineWidth = Math.max(contentWidth, containerWidth);

```

The `getEffectiveDuration` function accounts for trim offsets, ensuring the timeline extends beyond the last clip’s audible content.

## Track Layout Management

### Track Enumeration and Sorting

The editor maintains a vertical stack of tracks using a computed Set-based enumeration:

```typescript
const tracks = useMemo(() => {
  const trackSet = new Set([...DEFAULT_TRACKS, ...items.map(i => i.track)]);
  return Array.from(trackSet).sort((a, b) => b - a); // higher index = top
}, [items]);

```

`DEFAULT_TRACKS = [1, 0, -1]` ensures three visible rows exist even in empty projects. Tracks render with a fixed `TRACK_HEIGHT = 48` pixels (line 26).

### Vertical Positioning

Each clip calculates its vertical offset based on track index:

```typescript
const trackIndex = getTrackIndex(item.track);
const top = trackIndex * TRACK_HEIGHT;

```

The track index mapper converts track identifiers into zero-based indices for CSS `top` positioning, creating the layered visual layout characteristic of the Voicebox stories editor timeline composition.

## Clip Rendering and Waveform Visualization

### Effective Duration Calculations

Trim handles modify the audible portion of clips through duration arithmetic:

```typescript
const getEffectiveDuration = (item) =>
  item.duration * 1000 - (item.trim_start_ms || 0) - (item.trim_end_ms || 0);

```

This value determines the rendered width via `msToPixels`, while the original duration determines the waveform container size.

### Waveform Offset Handling

The `ClipWaveform` component renders full waveforms then offsets them to hide trimmed sections:

```tsx
<div
  ref={waveformRef}
  style={{ 
    width: `${fullWaveformWidth}px`, 
    transform: `translateX(-${offsetX}px)` 
  }}
/>

```

Where `offsetX` equals the pixel value of `trim_start_ms`, effectively masking silent portions without regenerating the waveform.

## User Interaction Handling

The timeline supports complex editing gestures through mouse event handlers:

- **Drag-and-drop moving** – `handleDragStart` (lines 47-62) captures mouse offsets relative to the clip; `handleDragMove` (lines 64-73) updates temporary `dragPosition` state for live visual feedback; `handleDragEnd` commits changes via the `useMoveStoryItem` mutation.
- **Trimming** – Trim handles appear on selected clips only. `handleTrimMove` updates `tempTrimValues` state for real-time width preview, while `handleTrimEnd` persists changes through `useTrimStoryItem`.
- **Splitting** – The `S` key or Split button triggers `handleSplit` (lines 502-525), calculating the split point relative to the playhead position and invoking `useSplitStoryItem` to create two clips from one.
- **Keyboard shortcuts** – `Cmd/Ctrl+D` duplicates via `useDuplicateStoryItem`, while Delete/Backspace removes clips through `useRemoveStoryItem`.

All mutations invalidate the `stories` React Query cache, ensuring UI synchronization with the server state.

## Playback Synchronization and Auto-Scrolling

The store manages playback timing through anchored references:

```typescript
// From storyStore.ts (line 56 onwards)
const play = () => {
  setPlaybackStartContextTime(audioContext.currentTime);
  setPlaybackStartStoryTime(currentTimeMs);
  setIsPlaying(true);
};

```

A `useEffect` hook (lines 47-61 in `StoryTrackEditor`) handles auto-scrolling during playback:

```typescript
useEffect(() => {
  if (!isCurrentlyPlaying || !tracksRef.current) return;
  const container = tracksRef.current;
  const halfway = container.scrollLeft + container.clientWidth / 2;
  if (playheadLeft > halfway) {
    container.scrollLeft = playheadLeft - container.clientWidth / 2;
  }
}, [isCurrentlyPlaying, playheadLeft]);

```

This keeps the playhead centered in the viewport while audio plays, maintaining visual context during story review.

## Implementation Example

Embed the editor in a parent view by providing the story ID and items array:

```tsx
import { StoryTrackEditor } from '@/components/StoriesTab/StoryTrackEditor';
import { useStory } from '@/lib/hooks/useStories';

export function StoriesTab({ storyId }: { storyId: string }) {
  const { data: story } = useStory(storyId);
  
  if (!story) return null;
  
  return (
    <StoryTrackEditor
      storyId={story.id}
      items={story.items}
    />
  );
}

```

The component self-contains all geometry calculations, interaction handlers, and waveform generation, requiring only the story metadata and item array to render the full stories editor timeline composition.

## Summary

- **Coordinate system** – Milliseconds convert to pixels via `pixelsPerSecond` scaling with configurable zoom bounds.
- **Track layout** – Vertical tracks render at 48-pixel heights, enumerated from a Set that merges default tracks with existing clip assignments.
- **Waveform rendering** – `ClipWaveform` uses Wavesurfer.js with CSS transforms to visually trim audio without processing.
- **State management** – Zustand handles UI state (selection, playback) while React Query manages server synchronization for mutations.
- **Playback integration** – Auto-scrolling centers the playhead during playback using timing anchors from the audio context.

## Frequently Asked Questions

### How does Voicebox convert between time and pixel coordinates?

The editor uses two memoized conversion functions, `msToPixels` and `pixelsToMs`, defined in [`StoryTrackEditor.tsx`](https://github.com/jamiepine/voicebox/blob/main/StoryTrackEditor.tsx). These functions multiply or divide milliseconds by the current `pixelsPerSecond` value (default 50) divided by 1000, establishing a scalable coordinate system that updates when users zoom in or out.

### What handles the drag-and-drop logic in the timeline?

Drag interactions follow a three-phase pattern: `handleDragStart` records initial mouse offsets and clip positions; `handleDragMove` updates temporary drag state for visual feedback; and `handleDragEnd` calculates the final millisecond destination and track index before calling the `useMoveStoryItem` mutation from [`useStories.ts`](https://github.com/jamiepine/voicebox/blob/main/useStories.ts).

### How does the timeline handle audio trimming visually?

Trimming manipulates the `transform: translateX` CSS property on the waveform container. The `ClipWaveform` component renders the full waveform at original duration width, then offsets it by the `trim_start_ms` value converted to pixels, effectively hiding trimmed sections without regenerating the waveform data.

### Where is the playback state managed in Voicebox?

Playback state resides in the Zustand store at [`app/src/stores/storyStore.ts`](https://github.com/jamiepine/voicebox/blob/main/app/src/stores/storyStore.ts), which maintains `currentTimeMs`, `isPlaying`, and timing anchors. The `StoryTrackEditor` reads these values to position the playhead and trigger auto-scrolling, while audio synchronization occurs through a separate hook that updates the store on each animation frame.