# DesktopCommanderMCP A/B Testing Infrastructure: How shouldShowMcpUiPreviews Controls Feature Rollouts

> Discover how DesktopCommanderMCP's A/B testing infrastructure uses shouldShowMcpUiPreviews to control feature rollouts with deterministic client assignments and safe error fallbacks for UI previews.

- Repository: [Eduard Ruzga/DesktopCommanderMCP](https://github.com/wonderwhy-er/DesktopCommanderMCP)
- Tags: deep-dive
- Published: 2026-07-10

---

**DesktopCommanderMCP uses a lightweight client-side A/B testing system driven by remote feature-flag JSON files, where `shouldShowMcpUiPreviews` deterministically assigns users to variants using a persistent `clientId` while safely falling back to enabled UI previews on errors.**

The DesktopCommanderMCP repository implements a deterministic, remotely configurable A/B testing infrastructure that controls feature exposure without requiring server-side state. This system relies on remote JSON configuration, consistent client-side hashing, and local persistence to manage experiments like the MCP UI Previews rollout. Understanding how `shouldShowMcpUiPreviews` evaluates user eligibility reveals a robust pattern for client-side feature flagging in TypeScript applications.

## How the A/B Testing Infrastructure Works

### Remote Feature Flag Management

The **feature-flag manager** in [`src/utils/feature-flags.ts`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/utils/feature-flags.ts) drives the entire system by loading JSON configuration from `https://desktopcommander.app/flags/v2/production.json`. On startup, it immediately reads a cached copy stored locally as [`feature-flags.json`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/feature-flags.json) to ensure responsiveness, then fetches a fresh copy in the background to synchronize with remote changes.

The manager exposes two critical methods for downstream consumers:
- **`wasLoadedFromCache()`** – indicates whether current flags are from local storage
- **`waitForFreshFlags()`** – returns a promise that resolves when the remote fetch completes

This dual-phase loading ensures the application remains functional during startup while eventually converging on the latest remote configuration.

### Experiment Configuration Format

Experiments are defined in the remote JSON using a V2 format consumed by [`src/utils/ab-test.ts`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/utils/ab-test.ts). Each experiment specifies weighted variants that determine user assignment probabilities:

```json
{
  "flags": {
    "experiments": {
      "McpUiPreviews": {
        "variants": [
          { "name": "showMCPUi", "weight": 80 },
          { "name": "notShowMCPUi", "weight": 20 }
        ]
      }
    }
  }
}

```

The client reads these definitions through `featureFlagManager.get('experiments')` without hardcoding experiment logic, allowing dynamic configuration changes without redeployment.

## Deterministic Variant Assignment

The A/B testing infrastructure guarantees **consistent user experiences** through deterministic variant selection. When a user first encounters an experiment, the system generates a hash from their persistent `clientId` (stored in [`config.json`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/config.json)) combined with the experiment name.

The selection process in [`src/utils/ab-test.ts`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/utils/ab-test.ts) follows these steps:

1. Hash the concatenation of `clientId + experimentName`
2. Select a variant respecting the configured percentage weights (e.g., 80% vs 20%)
3. Persist the chosen variant under the key `abTest_<experimentName>` using `configManager.setValue`
4. Return the same variant on subsequent calls for that user

This approach ensures that refreshing the application or restarting the computer does not change the user's assigned experience, eliminating flickering UI states and maintaining experimental validity.

## How shouldShowMcpUiPreviews Evaluates User Eligibility

The `shouldShowMcpUiPreviews()` function exported from [`src/utils/mcp-ui-ab-test.ts`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/utils/mcp-ui-ab-test.ts) serves as the decision layer for the MCP UI Previews experiment. Rather than implementing logic directly, it delegates to `resolveMcpUiPreviewDecision` with injected dependencies for testability:

```typescript
export async function shouldShowMcpUiPreviews(): Promise<boolean> {
  return resolveMcpUiPreviewDecision({
    getExistingAssignment: () => configManager.getValue(`abTest_${MCP_UI_EXPERIMENT_NAME}`),
    isFirstRun: () => configManager.isFirstRun(),
    wasLoadedFromCache: () => featureFlagManager.wasLoadedFromCache(),
    waitForFreshFlags: () => featureFlagManager.waitForFreshFlags(),
    getABTestVariant,
    capture,
  });
}

```

The resolution logic implements a three-phase decision tree:

### Existing Assignment Lookup

First, the function checks for a previously persisted variant using `configManager.getValue`. If an assignment exists, it evaluates whether the cached decision came from stale flags. When `wasLoadedFromCache()` returns true, the system optionally waits for fresh flags via `waitForFreshFlags()` to determine if remote overrides should replace the local assignment before returning the boolean value (`showMCPUi` yields `true`, `notShowMCPUi` yields `false`).

### First-Run Variant Resolution

For fresh installations where `configManager.isFirstRun()` returns true, the system fetches the current remote variant before making any UI promises. If the remote variant is recognized (either `showMCPUi` or `notShowMCPUi`), the function immediately **captures the decision for telemetry** using `capture('server_mcp_ui_ab_decision', …)` and returns the corresponding boolean. If the remote variant is missing or malformed, the UI defaults to enabled previews as a safe fallback.

### Error Handling and Telemetry

Any unexpected exceptions during flag retrieval or variant resolution trigger a catch-all fallback returning `true`, ensuring the MCP UI previews remain visible even during infrastructure failures. This **fail-open** philosophy prioritizes user experience over experimental rigor when system health is compromised.

## Server Integration

The A/B testing decision integrates directly into the MCP server lifecycle in [`src/server.ts`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/server.ts) (lines 297-318). When constructing UI metadata for tools requiring previews, the server awaits the boolean result:

```typescript
const showMcpUiPreviews = await shouldShowMcpUiPreviews();
const uiMetadata = buildUiToolMeta(toolName, toolParams, showMcpUiPreviews);

```

This integration ensures that tool previews respect the experiment configuration without forcing individual tools to understand A/B testing mechanics.

## Practical Implementation Examples

### Retrieving a Variant for Any Experiment

```typescript
import { getABTestVariant } from './utils/ab-test.js';

// Returns "showMCPUi", "notShowMCPUi", or null if unassigned
const variant = await getABTestVariant('McpUiPreviews');

```

### Checking Feature Availability Directly

```typescript
import { hasFeature } from './utils/ab-test.js';

if (await hasFeature('showMCPUi')) {
  // Execute preview-specific code path
}

```

### Using the MCP UI Preview Helper

```typescript
import { shouldShowMcpUiPreviews } from './utils/mcp-ui-ab-test.js';

const uiEnabled = await shouldShowMcpUiPreviews();
if (uiEnabled) {
  // Initialize MCP UI preview components
}

```

### Refreshing Feature Flags Manually

```typescript
import { featureFlagManager } from './utils/feature-flags.js';

// Force immediate remote refresh; resolves when complete
await featureFlagManager.refresh();

```

## Summary

- **Remote-first configuration**: Feature flags load from `https://desktopcommander.app/flags/v2/production.json` with local caching in [`src/utils/feature-flags.ts`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/utils/feature-flags.ts) to balance startup speed with configuration freshness.
- **Deterministic assignment**: Variants are selected via hash of `clientId + experimentName` in [`src/utils/ab-test.ts`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/utils/ab-test.ts) and persisted in [`config.json`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/config.json) under keys matching `abTest_<experimentName>`.
- **Fail-safe defaults**: `shouldShowMcpUiPreviews` in [`src/utils/mcp-ui-ab-test.ts`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/utils/mcp-ui-ab-test.ts) returns `true` on errors and when remote variants are unrecognized, ensuring UI previews remain available during outages.
- **First-run telemetry**: Fresh installations report their A/B assignment via the `capture` function, enabling accurate experiment participation tracking without exposing personal data.
- **Server-side consumption**: The decision integrates at the server level in [`src/server.ts`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/server.ts), passing the boolean to `buildUiToolMeta` to control preview rendering.

## Frequently Asked Questions

### How does DesktopCommanderMCP ensure users always see the same A/B test variant?

The system combines a stable `clientId` from [`config.json`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/config.json) with the experiment name, hashes the concatenation, and caches the resulting variant selection under a persistent key. This deterministic approach prevents variant switching when the application restarts or flags refresh.

### What happens if the remote feature flag file cannot be loaded?

If the remote JSON at `https://desktopcommander.app/flags/v2/production.json` fails to load, the system relies on the locally cached [`feature-flags.json`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/feature-flags.json) file. For the specific MCP UI previews experiment, any error or missing configuration causes `shouldShowMcpUiPreviews` to default to `true`, ensuring the feature remains visible rather than hidden.

### How does the A/B testing infrastructure distinguish between new and returning users?

The `configManager.isFirstRun()` method identifies fresh installations. During first run, the system waits for fresh remote flags before assigning a variant and reports the decision via telemetry. Returning users immediately load their cached assignment from [`config.json`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/config.json), skipping the telemetry capture and remote wait unless the cache is stale.

### Where are the core A/B testing files located in the repository?

The primary files are [`src/utils/feature-flags.ts`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/utils/feature-flags.ts) for remote flag fetching, [`src/utils/ab-test.ts`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/utils/ab-test.ts) for core assignment logic, and [`src/utils/mcp-ui-ab-test.ts`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/utils/mcp-ui-ab-test.ts) for the specific preview decision implementation. Integration occurs in [`src/server.ts`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/src/server.ts), and the full system is validated by [`test/ab-test.test.js`](https://github.com/wonderwhy-er/DesktopCommanderMCP/blob/main/test/ab-test.test.js).