# How DesktopCommanderMCP Implements Streaming Search Results with Pagination in startSearch

> Discover how DesktopCommanderMCP's startSearch streams results with pagination. Learn about its ripgrep process, JSON parsing, and offset-based access for efficient searching.

- Repository: [Eduard Ruzga/DesktopCommanderMCP](https://github.com/wonderwhy-er/DesktopCommanderMCP)
- Tags: how-to-guide
- Published: 2026-07-09

---

**`startSearch` implements streaming results with pagination by spawning a ripgrep process that incrementally parses JSON output into a session-specific buffer, returning a session ID immediately while `readSearchResults` provides offset-based access to the accumulating results.**

The `SearchManager` class in [wonderwhy-er/DesktopCommanderMCP](https://github.com/wonderwhy-er/DesktopCommanderMCP) delivers high-performance file searching through a **streaming results with pagination** architecture. By leveraging Node.js child processes and ripgrep's JSON output mode, the system balances immediate responsiveness with complete result set availability.

## The Architecture Behind Streaming Search

In [`src/search-manager.ts`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts), the `startSearch` method (lines 59-120) orchestrates a non-blocking search operation by wrapping ripgrep in a Node.js child process. Rather than waiting for the entire search to complete, the method immediately returns a session identifier while the process continues executing in the background.

### Spawning the Ripgrep Process

The implementation begins by calling `buildRipgrepArgs` (lines 333-410) to construct a ripgrep command with JSON output formatting. The method then spawns the process and wires up event handlers via `setupProcessHandlers` (lines 813-886), which establishes the streaming pipeline between the process stdout and the session storage.

### Incremental Output Parsing

As ripgrep writes JSON lines to stdout, the `process.on('data')` handler appends chunks to a session-wide buffer. The `processBufferedOutput` routine splits this buffer on newlines, passing complete lines to `parseLine` for deserialization into `SearchResult` objects. These objects are immediately pushed into the session's `results` array, making them available for pagination before the search completes.

## The First-Chunk Response Pattern

To provide immediate feedback while maintaining the streaming architecture, `startSearch` creates a first-chunk promise that resolves when the initial data event arrives or after a 40ms timeout. This allows the method to return the current session state—including any results parsed up to that moment—while the ripgrep process continues executing. The returned payload includes the `sessionId`, initial results array, and metadata indicating whether the search is complete.

## Pagination with readSearchResults

Once a session is established, clients retrieve additional pages via `readSearchResults` (lines 222-286), which implements offset-based pagination against the in-memory results array.

### Offset-Based Slicing

The method signature `readSearchResults(sessionId, offset, length)` accepts a starting offset and page length, returning a slice of the stored results along with metadata including `totalResults`, `hasMoreResults`, and `isComplete`. This design allows efficient random access to any portion of the result set without re-executing the search.

### Tail Mode with Negative Offsets

For monitoring recent matches, the API supports negative offset values that trigger "tail" behavior. When passed a negative offset, `readSearchResults` calculates the position from the end of the results array, returning the last N entries regardless of total result count. This enables real-time log monitoring use cases where only the most recent matches are relevant.

## Session State and Lifecycle Management

Each search session maintains a comprehensive state object tracking `lastReadTime`, `totalMatches`, `totalContextLines`, and boolean flags for `isComplete` and `wasIncomplete`. The `close` handler finalizes the session when the ripgrep process exits, recording whether the search terminated due to permission errors. A periodic cleanup interval removes stale completed sessions to prevent memory leaks.

## Code Examples

```typescript
// Initiate a streaming search
const { sessionId, results } = await searchManager.startSearch({
  rootPath: '/my/project',
  pattern: 'TODO',
  searchType: 'content',
  ignoreCase: true,
  contextLines: 2,
  maxResults: 100
});

console.log('Initial batch:', results);

// Retrieve page 2 (items 100-199)
const page2 = searchManager.readSearchResults(sessionId, 100, 100);
console.log('Page 2:', page2.results);

// Tail mode: get last 10 results
const recent = searchManager.readSearchResults(sessionId, -10);
console.log('Recent matches:', recent.results);

```

## Summary

- **`startSearch`** launches ripgrep as a background process and returns immediately with a session ID and initial results.
- **Streaming** occurs via stdout event handlers that parse JSON lines incrementally into the session's `results` array.
- **First-chunk resolution** provides sub-100ms response times through a promise that resolves on the first data event or 40ms timeout.
- **`readSearchResults`** supports standard pagination with offset/length parameters and tail-mode with negative offsets.
- **Session cleanup** automatically removes completed sessions based on inactivity timestamps and process exit status.

## Frequently Asked Questions

### How does startSearch handle large result sets without blocking?

The method uses a streaming architecture where ripgrep runs in a separate process, with stdout data processed incrementally. This keeps the event loop unblocked while results accumulate in memory, allowing the initial response to return within milliseconds while hundreds of thousands of matches continue processing in the background.

### What happens if I request an offset beyond the current results count?

`readSearchResults` returns the available results up to the requested length, along with `hasMoreResults: true` and `isComplete: false` flags. Clients should poll the method until `isComplete` becomes true or `totalResults` stabilizes, indicating the ripgrep process has finished writing to the buffer.

### Can I stream results in real-time without using pagination?

While `startSearch` returns the initial batch immediately, the session object in [`src/search-manager.ts`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts) exposes the underlying `results` array that grows as new data arrives. For true real-time streaming, you would need to poll `readSearchResults` with appropriate offsets or access the session object directly, though the latter is not recommended as it bypasses the tail-mode calculation and boundary checking.

### Why does the first-chunk timeout default to 40ms?

The 40ms timeout balances immediate responsiveness with result completeness. According to the source code, this duration ensures the method returns promptly even for searches with few matches, while still capturing the initial burst of results from ripgrep's highly optimized regex engine.