How Search Sessions Stream Results with Pagination in DesktopCommanderMCP

DesktopCommanderMCP implements a search-session layer that mirrors the terminal-session API but works with ripgrep to provide progressive, stream-able results and offset-based pagination.

DesktopCommanderMCP uses a sophisticated search-session architecture to handle large codebase searches without blocking the main event loop. This system wraps ripgrep in a streaming session that delivers results incrementally while supporting traditional pagination patterns. Understanding how search sessions stream results with pagination reveals why the tool remains responsive even when searching millions of lines of code.

Search Session Architecture and Lifecycle

The foundation of streaming search results lies in the SearchManager class implemented in src/search-manager.ts. This manager maintains a registry of active sessions and handles the lifecycle of ripgrep child processes.

Creating a Search Session

When a client initiates a search, the startSearch method creates a SearchSession object, spawns a ripgrep child process, and registers the session in this.sessions【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L59-L71】. Unlike synchronous search implementations, this method returns immediately with a session identifier while the search continues in the background.

Spawning the Ripgrep Process

The manager resolves the ripgrep binary path through src/utils/ripgrep-resolver.js and launches the process with streaming JSON output enabled. This allows the system to parse results line-by-line as they are discovered rather than waiting for the entire search to complete.

Streaming Results in Real-Time

The streaming mechanism relies on a first-data-chunk strategy combined with continuous buffering to provide immediate feedback to users.

The First Data Chunk Strategy

Instead of waiting for the complete ripgrep run, startSearch waits for the first data chunk or a 40ms cap, whichever comes first, before returning the session ID together with any results collected so far【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L98-L110】. This gives the caller a streaming hint that more data will arrive, allowing the UI to display initial matches while the search continues.

Buffering JSON Output

As ripgrep writes JSON lines to stdout, setupProcessHandlers appends them to session.buffer and calls processBufferedOutput【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L13-L20】. Each complete line is parsed via parseLine and pushed onto session.results【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L21-L53】. This buffering approach ensures that results are available for immediate pagination even while the underlying process continues to run.

Offset-Based Pagination Implementation

The readSearchResults(sessionId, offset?, length?) method implements the pagination logic, supporting both forward traversal and tail-based access patterns.

Range Reading with Positive Offsets

When provided with positive offsets, the method returns a slice of the accumulated results starting from the specified offset【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L226-L232】. This allows clients to fetch results in predictable chunks (e.g., 100 results at a time) while the buffer continues to grow in the background.

Tail Behavior with Negative Offsets

Negative offsets are interpreted as "tail" behavior, returning the last |offset| results regardless of how many results have been collected【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L52-L58】. This mirrors the file-read API in DesktopCommanderMCP and is particularly useful for log-file searching or when you need the most recent matches.

The hasMoreResults Flag

Every pagination response includes a hasMoreResults boolean that indicates whether additional data will arrive later【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L69-L73】. Clients use this flag to determine when to stop polling for new results and to display progress indicators in the UI.

Early Termination and Session Cleanup

The system includes optimizations for specific search patterns and automatic cleanup to prevent memory leaks.

Exact Filename Optimization

For exact-filename searches, the implementation sets a short default timeout of 1500ms and includes an early-termination check that aborts the ripgrep process as soon as the exact match is seen【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L14-L16】【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L31-L49】. This prevents unnecessary disk scanning when the target file has already been located.

Automatic Session Expiration

A global interval calls searchManager.cleanupSessions() every 5 minutes, purging completed sessions that have been idle longer than the configured maximum age【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L10-L16】. This ensures that memory-intensive search buffers do not accumulate indefinitely.

Working with Search Sessions

The following example demonstrates the complete workflow for initiating a streaming search and retrieving paginated results:

import { searchManager } from './src/search-manager.js';

// 1️⃣ Start a file-search session (streaming response)
const start = await searchManager.startSearch({
  rootPath: '/Users/alex/projects',
  pattern: 'README',                // substring search → will be wrapped with wildcards
  searchType: 'files',
  includeHidden: false,
  maxResults: 200,
});
console.log('Session ID:', start.sessionId);
console.log('First chunk:', start.results);

// 2️⃣ Pull the next page (offset-based pagination)
const page2 = searchManager.readSearchResults(start.sessionId, 100, 100);
console.log('Next 100 results:', page2.results);
console.log('More results available?', page2.hasMoreResults);

// 3️⃣ Tail-read – get the last 20 results irrespective of offset
const tail = searchManager.readSearchResults(start.sessionId, -20);
console.log('Last 20 results:', tail.results);

// 4️⃣ Stop the search early (e.g. user cancelled)
searchManager.terminateSearch(start.sessionId);

// 5️⃣ List active sessions (useful for UI "search in progress" panels)
const active = searchManager.listSearchSessions();
console.log('Active searches:', active);

The RPC-style schemas defined in src/tools/schemas.ts (start_search, get_more_search_results, stop_search, list_sessions) provide the interface that front-end clients use to invoke these methods.

Summary

• DesktopCommanderMCP uses a SearchSession pattern in src/search-manager.ts to wrap ripgrep in a streaming, non-blocking interface.
• Streaming results are achieved by returning the first data chunk within 40ms while the process continues running in the background.
• Offset-based pagination supports both positive offsets (range reading) and negative offsets (tail behavior) through the readSearchResults method.
• The hasMoreResults flag signals when additional data is still being processed by the ripgrep child process.
• Early termination optimizes exact filename searches with a 1500ms timeout and immediate abort on match.
• Automatic cleanup runs every 5 minutes to remove idle sessions and free memory.

Frequently Asked Questions

How does DesktopCommanderMCP stream search results without blocking the main thread?

DesktopCommanderMCP spawns ripgrep as a child process and buffers JSON output asynchronously. The startSearch method returns a session ID immediately after the first data chunk arrives (or after 40ms), allowing the main event loop to continue while setupProcessHandlers processes incoming lines in the background【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L13-L20】【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L98-L110】.

What is the difference between positive and negative offsets in search pagination?

Positive offsets return results starting from the specified index (range reading), which is useful for traditional pagination. Negative offsets trigger "tail" behavior, returning the last |offset| results from the current buffer, similar to the Unix tail command【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L52-L58】【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L226-L232】.

How does the search session handle early termination for exact filename searches?

When searching for exact filenames, the system sets a 1500ms timeout and monitors the output stream. If the exact match is located before the timeout expires, the ripgrep process is terminated immediately to avoid unnecessary disk scanning【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L14-L16】【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L31-L49】.

What happens to search sessions that are no longer active?

A global cleanup interval runs every 5 minutes to purge completed sessions that have exceeded the configured maximum idle age. This prevents memory leaks from abandoned search buffers that are no longer being polled by clients【https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/search-manager.ts#L10-L16】.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}