How no-mistakes Handles Flaky Test and Lint Commands with Timeouts and Retries

no-mistakes handles flaky commands by wrapping shell execution in a process-group-aware timeout and retrying failed steps up to three times when marked as auto-fixable.

The no-mistakes open-source pipeline tool provides resilient execution of test and lint steps through configurable timeouts and intelligent retry logic that distinguishes between transient failures and permanent errors. When a command hangs or fails intermittently, the executor automatically manages retries while ensuring hung processes cannot block the pipeline indefinitely.

Timeout Handling with Context Deadlines

Every command in no-mistakes runs through the runStepShellCommand function in internal/pipeline/steps/step_shell.go. This function creates a context with a deadline using context.WithTimeout, defaulting to five minutes from sctx.Config.StepTimeout.

The shellenv.ConfigureShellCommand wrapper sets up a process group for the entire command tree, allowing the system to kill all child processes if the deadline expires. When a timeout occurs, the step records the error with needsApproval = true and autoFixable = false, preventing retries on genuine timeouts while surfacing the failure for user review.

// internal/pipeline/steps/step_shell.go
func runStepShellCommand(sctx *pipeline.StepContext, cmd string) (output string, exitCode int, err error) {
    timeout := sctx.Config.StepTimeout // e.g. 5 min
    ctx, cancel := context.WithTimeout(sctx.Ctx, timeout)
    defer cancel()
    // shellenv.ConfigureShellCommand creates a process group and attaches the ctx
    execCmd := shellenv.ConfigureShellCommand(ctx, exec.Command("sh", "-c", cmd))
    out, err := execCmd.CombinedOutput()
    // parse exit code, handle ctx.Err() == deadline exceeded as timeout
    // ...
}

Auto-Fix Retry Logic for Flaky Steps

When a lint or test step fails with a non-zero exit code but appears recoverable, the step marks the outcome with AutoFixable: true. This triggers the retry loop in internal/pipeline/executor.go, which respects the maxAttempts value from configuration (defaulting to three).

The executor iterates through attempts, logging each retry and re-running the step with a fresh StepContext to clear any transient state. If a retry succeeds, the FixSummary is committed and the pipeline proceeds; otherwise, the step requires manual approval after exhausting all attempts.

// internal/pipeline/executor.go
maxAttempts := cfg.StepRetry // default 3
for attempt := 1; attempt <= maxAttempts; attempt++ {
    outcome, err := step.Execute(ctx)
    if err != nil { return err }
    if !outcome.AutoFixable {               // no retry needed
        break
    }
    // log attempt number – visible in test output
    logger.Infof("auto-fixing %s (attempt %d/%d)…", step.Name(), attempt, maxAttempts)
}

Lint and Test Step Implementation

In internal/pipeline/steps/lint.go (lines 155-158) and internal/pipeline/steps/test.go, failing commands set AutoFixable: true alongside NeedsApproval: true. This signals to the executor that the failure might resolve on retry, unlike timeouts which are not auto-fixable.

// internal/pipeline/steps/lint.go
if exitCode != 0 {
    findings := Findings{
        Items: []Finding{{Severity: "warning",
            Description: fmt.Sprintf("linter found issues (exit code %d)", exitCode)}},
        Summary: output,
    }
    findingsJSON, _ := json.Marshal(findings)
    return &pipeline.StepOutcome{
        NeedsApproval: true,
        AutoFixable:   true,               // <-- enables retries
        Findings:      string(findingsJSON),
        ExitCode:      exitCode,
        FixSummary:    fixSummary,
    }, nil
}

Configuration Options

The timeout and retry behavior are configurable through config.yaml or the internal configuration struct defined in internal/config/config.go. The StepTimeout default is five minutes, while StepRetry defaults to three attempts. These values apply globally to all pipeline steps that support auto-fix operations.

Summary

• Process-group timeouts: Commands execute with a context deadline that kills the entire process tree if exceeded, preventing zombie processes from hanging the pipeline.
• Auto-fixable retries: Steps that return non-zero exit codes can be retried up to three times (configurable) when marked with AutoFixable: true in the outcome.
• Clear separation: Timeouts surface as non-retryable failures requiring approval, while flaky command failures trigger the retry loop in internal/pipeline/executor.go.
• Fresh context: Each retry receives a new StepContext, ensuring transient state does not persist between attempts.

Frequently Asked Questions

How does no-mistakes prevent a flaky command from hanging indefinitely?

no-mistakes wraps every shell command in a context with a configurable timeout (default five minutes) using context.WithTimeout in internal/pipeline/steps/step_shell.go. If the deadline expires, shellenv.ConfigureShellCommand terminates the entire process group and returns a timeout error, preventing any single command from blocking the pipeline indefinitely.

What is the maximum number of retries for a failing test or lint step?

By default, no-mistakes retries auto-fixable steps up to three times. This value is controlled by the StepRetry configuration in config.yaml (or internal/config/config.go), which the executor reads in internal/pipeline/executor.go before entering the retry loop.

How does the executor distinguish between a timeout and a flaky failure?

Timeouts set autoFixable = false in the step outcome, indicating the command did not complete and cannot be retried. Flaky failures (non-zero exit codes) set autoFixable = true, signaling to the executor in internal/pipeline/executor.go that the step should be retried according to the maxAttempts configuration.

Can I configure different timeout values for specific steps?

The timeout value comes from sctx.Config.StepTimeout, which is a global configuration setting defined in internal/config/config.go. While the current implementation uses a single global timeout, the StepContext pattern allows for future per-step configuration extensions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}