How Cypress Implements Test Retries and Flake Detection: A Deep Dive into the Source Code

Cypress extends Mocha's native retry mechanism with experimental strategies that analyze multiple attempt results to distinguish between genuine failures and flaky tests, applying configurable pass thresholds to determine final status.

Cypress builds its retry and flake-detection capabilities on top of Mocha's native retry mechanism, extending it with experimental strategies that can differentiate between genuine failures and flaky tests. The implementation spans the driver, server, and configuration packages within the cypress-io/cypress repository. Understanding how these components interact reveals how Cypress automatically retries failing tests while intelligently flagging flaky behavior.

Mocha Integration Foundation

Cypress loads Mocha directly in the browser and extracts its core constructors (Test, Suite, Hook, Runner) to maintain full control over the test runner. In packages/driver/src/cypress/mocha.ts, the driver keeps references to the original Mocha methods including Test.prototype.retries, Test.prototype.clone, and Suite.prototype.retries—these serve as the entry points for the retry API that users configure via cypress.config.{js,ts} or per-test options.

After extracting these constructors, Cypress removes the original Mocha globals from the window object (delete window.mocha) to prevent conflicts and ensure complete control over the test execution lifecycle. This wrapping occurs in the initialization logic found in packages/driver/src/cypress/mocha.ts between lines 18 and 30.

Configuration Schema and Validation

The retry configuration supports two distinct modes: a stable numeric shorthand and an experimental object-based strategy. The validation logic in packages/config/src/validation.ts (lines 121-148) ensures that when users specify the experimental format, the object contains valid experimentalStrategy and experimentalOptions keys.

The public retries option is declared in packages/config/src/options.ts (line 402), exposing both the simple numeric interface and the complex object interface to users. When using the experimental strategies, the configuration is normalized into a NormalizedRetriesConfig structure containing strategy, maxRetries, passesRequired, and stopIfAnyPassed fields.

Experimental Flake Detection Strategies

Cypress provides two distinct experimental strategies that change how the runner interprets retry attempts. These strategies normalize into a consistent internal representation before the test executes.

detect-flake-and-pass-on-threshold

This strategy allows a test to pass as soon as a configurable number of attempts succeed, even if earlier attempts failed. The passesRequired option (defaulting to 1) determines how many successful attempts are necessary before the runner marks the test as passed and stops retrying. This is particularly useful for tests that occasionally fail due to timing issues but consistently pass on retry.

detect-flake-but-always-fail

This strategy keeps retrying the test according to the maximum retry count, but the final status always reports as failed regardless of whether any attempt passed. The optional stopIfAnyPassed boolean can halt retries early if any single attempt passes, allowing you to detect flakiness without masking the failure. This helps identify unstable tests that require investigation rather than automatically accepting them as passing.

The calculateTestStatus Algorithm

The core retry logic resides in the calculateTestStatus function within packages/driver/src/cypress/mocha.ts (lines 57-95 and 101-119). This function inspects the current retry count, the list of previous attempts stored in test.prevAttempts, and the normalized experimental configuration to make three critical decisions:

• Whether attempts should continue: The shouldAttemptsContinue boolean determines if the runner should execute another retry based on the strategy rules and remaining attempts.
• Outer status reporting: The algorithm sets the final reported status to passed or failed according to the experimental strategy's logic.
• Finality determination: The test.final flag indicates when no further attempts will occur, handling edge cases such as forcing the state to passed on the last attempt when using detect-flake-but-always-fail.

The IRetries interface exposed to users via cy.retries() is implemented in packages/driver/src/cypress/cy.ts, providing the programmatic bridge to these internal calculations.

Reporter Integration and Output Formatting

The Reporter class in packages/server/lib/reporter.ts (lines 400-525) listens to Mocha events including retry and test:after:run to format output appropriately. When no experimental strategy is configured, the reporter prints the classic ✖(Attempt X of Y) line. However, when a strategy is active, it prints detailed attempt results for each retry and ensures the final status reflects the experimental logic rather than the raw last-attempt result.

This separation between the driver's status calculation and the reporter's formatting ensures that the experimental retry logic remains consistent across different output formats and integrations.

Configuration Examples

Global Configuration with Numeric Retries

// cypress.config.ts
export default defineConfig({
  retries: 2,  // equivalent to { runMode: 2, openMode: 2 }
})

Global Configuration with Flake Detection

// cypress.config.ts
export default defineConfig({
  retries: {
    experimentalStrategy: 'detect-flake-and-pass-on-threshold',
    experimentalOptions: { passesRequired: 2 },
  },
})

Per-Test Numeric Override

it('may fail a few times', { retries: 4 }, () => {
  // test body
})

Per-Test Experimental Strategy

it('detects flake, stops after first pass', {
  retries: {
    experimentalStrategy: 'detect-flake-but-always-fail',
    experimentalOptions: { stopIfAnyPassed: true },
  },
}, () => {
  // test body
})

Summary

• Cypress wraps Mocha's native retry mechanism in packages/driver/src/cypress/mocha.ts by extracting core constructors and removing global Mocha references to maintain control over test execution.
• Experimental strategies provide two distinct approaches: detect-flake-and-pass-on-threshold for allowing passes after multiple successes, and detect-flake-but-always-fail for identifying flaky tests while maintaining a failed status.
• The calculateTestStatus function implements the core decision logic, analyzing test.prevAttempts and the normalized configuration to determine whether to continue retries and what final status to report.
• Configuration validation occurs in packages/config/src/validation.ts, ensuring that experimental options like passesRequired and stopIfAnyPassed meet expected schemas.
• The reporter in packages/server/lib/reporter.ts formats retry output differently based on whether experimental strategies are active, ensuring accurate logging of attempt histories.

Frequently Asked Questions

How does Cypress's retry mechanism differ from Mocha's native implementation?

Cypress retains Mocha's core retry API (Test.prototype.retries) but wraps it with additional logic in packages/driver/src/cypress/mocha.ts that supports experimental flake detection strategies. While Mocha only supports a simple numeric retry count, Cypress extends this with the calculateTestStatus function that can interpret multiple attempt results and apply complex pass/fail rules based on the experimentalStrategy configuration.

What is the difference between detect-flake-and-pass-on-threshold and detect-flake-but-always-fail?

The detect-flake-and-pass-on-threshold strategy (configured with passesRequired) marks a test as passed as soon as the specified number of attempts succeed, even if previous attempts failed. Conversely, detect-flake-but-always-fail runs the full retry sequence but always reports the test as failed, optionally stopping early if stopIfAnyPassed is true, which helps identify flaky tests without masking them as passing.

Can I configure different retry strategies for individual tests?

Yes, you can override global retry settings at the individual test level by passing a configuration object to the test definition. Both numeric retries ({ retries: 3 }) and experimental strategies ({ retries: { experimentalStrategy: '...', experimentalOptions: {...} } }) can be specified inline in the it() or describe() block, allowing granular control over flake detection behavior for specific test cases.

Where does Cypress store the results of previous retry attempts?

Cypress stores the history of retry attempts in the test.prevAttempts array, which the calculateTestStatus function in packages/driver/src/cypress/mocha.ts inspects to determine the final outcome. This array accumulates the results of each execution attempt, enabling the experimental strategies to analyze patterns across retries rather than considering only the most recent attempt.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}