# How Meetily Handles Incremental Transcript Processing During Live Recording

> Discover how Meetily processes live transcripts incrementally using Whisper and Tauri events. Learn about real-time audio capture, VAD, and fault tolerance for seamless recordings.

- Repository: [Zackriya Solutions/meetily](https://github.com/Zackriya-Solutions/meetily)
- Tags: internals
- Published: 2026-07-13

---

**Meetily captures audio in real-time, applies Voice Activity Detection (VAD) to isolate speech segments, processes them through the Whisper engine, and emits partial transcripts incrementally via Tauri events while simultaneously persisting checkpoints to disk for fault tolerance.**

Meetily is an open-source meeting transcription application developed by Zackriya-Solutions that transcribes speech live during recording sessions. Unlike traditional batch processing that delays transcription until the meeting ends, this architecture streams partial results to the user interface immediately while maintaining durable checkpoints. This approach ensures users see live captions within seconds of speaking and protects data against application crashes.

## The Four-Stage Incremental Pipeline

The incremental transcript processing system consists of four tightly-coupled components that operate concurrently during a live recording session.

### Stage 1: Audio Capture and VAD Filtering

The process begins in [`src/audio/pipeline.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/src/audio/pipeline.rs), where the **Pipeline** receives PCM frames from the audio capture modules located in `src/audio/capture/*`. The pipeline buffers the incoming microphone stream and applies a **Voice Activity Detection (VAD)** filter implemented in [`src/audio/vad.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/src/audio/vad.rs). This filter identifies speech-only segments and discards silence, producing structured `AudioChunk` objects that contain only relevant audio data. By filtering out non-speech audio before transcription, the system dramatically reduces the computational load on the heavy transcription model.

### Stage 2: Async Chunk Dispatch

Each `AudioChunk` is handed to the **transcription worker** defined in [`src/audio/transcription/worker.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/src/audio/transcription/worker.rs). This worker operates on a separate async task to prevent blocking the main audio capture thread. The worker forwards the chunk to the configured transcription provider, which can be either Whisper-cpp or Parakeet, depending on the system configuration. This asynchronous dispatch ensures that audio capture continues uninterrupted while transcription proceeds in parallel.

### Stage 3: Whisper Transcription Processing

The provider—either [`whisper_provider.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/whisper_provider.rs) or [`parakeet_provider.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/parakeet_provider.rs)—calls the native Whisper library to process the speech chunk. The engine returns a **partial transcript** for the received audio segment. The provider wraps this result in a `TranscriptUpdate` struct and sends it back to the pipeline through an async channel. This immediate return of partial results, rather than waiting for the entire meeting to conclude, is the core mechanism that enables live captioning.

### Stage 4: Real-Time Emission and Persistence

Upon receiving the `TranscriptUpdate`, the pipeline performs two critical operations simultaneously. First, it emits a Tauri event named `transcript-update` that the React frontend listens for, allowing the UI to append new text to the live transcript view immediately. Second, it passes the same update to the **incremental audio-saver** in [`src/audio/incremental_saver.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/src/audio/incremental_saver.rs), which stores a checkpoint containing both the raw audio segment and its associated transcript fragment in the `.checkpoints/` directory. This dual-path emission ensures that even if the application crashes, the partially-saved transcript can be recovered from disk.

## Why Incremental Processing Matters

The incremental architecture provides three distinct advantages over batch transcription:

- **Responsiveness**: Users see captions appear seconds after they speak, mirroring the experience of professional live-captioning services.
- **Fault-tolerance**: Checkpoints containing audio and transcript fragments are written to disk as soon as they are produced, ensuring that a crash does not discard work already completed.
- **Resource Efficiency**: The VAD system filters out silence before sending data to Whisper, significantly reducing the amount of audio the transcription model must process and lowering CPU usage.

## Implementation Code Examples

### Starting a Recording Session

The frontend initiates the incremental pipeline by invoking the Tauri command `start_recording`, which creates a `RecordingManager` instance that spawns the audio pipeline described above.

```typescript
await invoke('start_recording', {
  mic_device_name: 'Built-in Microphone',
  system_device_name: 'BlackHole 2ch',
  meeting_name: 'Team Stand-up',
});

```

### Listening for Live Transcript Updates

The React frontend listens for `transcript-update` events to append new text to the live view as it becomes available.

```typescript
listen<TranscriptUpdate>('transcript-update', (event) => {
  setTranscripts(prev => [...prev, event.payload.text]);
});

```

Each `TranscriptUpdate` contains the freshly-transcribed text for the latest speech chunk, enabling real-time UI updates.

### Recovering from Checkpoints

If the application needs to recover from a crash or when finalizing the recording, the system uses [`incremental_saver.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/incremental_saver.rs) to merge checkpoints into the final output.

```rust
let checkpoints = audio::incremental_saver::recover_audio_from_checkpoints(meeting_path)?;
for chk in checkpoints {
    println!("Chunk {} – {} bytes, {} transcript chars", chk.id, chk.audio.len(), chk.transcript.len());
}

```

When the user stops the recording, the `incremental_saver` merges all checkpoints into the final `audio.mp4` file and a complete transcript file, ensuring no data is lost.

## Summary

- **Incremental transcript processing** in Meetily enables real-time captioning by processing audio chunks as they are captured rather than waiting for recording completion.
- The **Voice Activity Detection** system in [`src/audio/vad.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/src/audio/vad.rs) filters silence before transcription, optimizing resource usage.
- **Async workers** in [`src/audio/transcription/worker.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/src/audio/transcription/worker.rs) handle transcription without blocking the audio capture thread.
- **Dual-path emission** sends transcript updates to both the UI via Tauri events and to disk via [`incremental_saver.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/incremental_saver.rs) for fault tolerance.
- **Checkpoint-based persistence** ensures that partial meeting transcripts survive application crashes and can be recovered automatically.

## Frequently Asked Questions

### How does Meetily handle crashes during live recording?

Meetily writes **incremental checkpoints** to disk immediately upon receiving each transcript update. These checkpoints, stored in the `.checkpoints/` directory by [`src/audio/incremental_saver.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/src/audio/incremental_saver.rs), contain both the raw audio segment and its corresponding transcript fragment. If the application crashes, the `recover_audio_from_checkpoints` function can reconstruct the meeting data up to the last successful checkpoint, preventing data loss.

### What is the role of Voice Activity Detection in the pipeline?

**Voice Activity Detection (VAD)** serves as a pre-filter that identifies speech segments within the raw audio stream and discards silence or non-speech noise. Implemented in [`src/audio/vad.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/src/audio/vad.rs) and applied in [`src/audio/pipeline.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/src/audio/pipeline.rs), VAD ensures that only relevant `AudioChunk` objects are sent to the transcription engine. This reduces CPU usage and improves transcription accuracy by eliminating unnecessary processing of silent periods.

### How does the frontend receive real-time transcript updates?

The frontend receives updates through **Tauri's event system**. When the Rust backend in [`src/audio/pipeline.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/src/audio/pipeline.rs) receives a `TranscriptUpdate` from the transcription worker, it emits a `transcript-update` event. The React frontend listens for this event using the `listen<TranscriptUpdate>()` function and immediately appends the new text to the live transcript view, creating a seamless real-time captioning experience.

### Where are incremental checkpoints stored and how are they finalized?

Checkpoints are stored in the `.checkpoints/` directory within the meeting's folder path by [`src/audio/incremental_saver.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/src/audio/incremental_saver.rs). Each checkpoint contains a serialized audio segment and its transcript. When the user stops the recording via the `stop_recording` command, the `incremental_saver` merges all checkpoint files into the final `audio.mp4` and a complete transcript file, then cleans up the temporary checkpoint directory.