How to Integrate the Retry Package with Custom Backoff Logic in Go

You can integrate the retry package with custom backoff logic by implementing the backoff.BackOff interface and passing it to retry.Retry(), or by configuring the protobuf-backed backoff.Backoff struct and using retry.NewBackOff() to generate a concrete implementation.

The retry package in aperturerobotics/util provides a robust, context-aware retry loop for Go applications. To integrate the retry package with custom backoff logic, you can either configure the built-in exponential backoff parameters or implement a bespoke BackOff interface that controls interval calculation and reset behavior.

Architecture of the Retry and Backoff Components

Understanding how the components interact is essential before implementing custom logic.

Core Retry Loop

The entry point is retry.Retry in retry/retry.go. Its signature is:

func Retry(ctx context.Context, le *logrus.Entry, f func(context.Context, func()) error, bo backoff.BackOff) error

The function f receives a success callback. Invoking this callback triggers bo.Reset(), which resets the backoff timer to its initial state.

Backoff Interface Contract

The interface definition resides in backoff/cbackoff/backoff.go:

type BackOff interface {
    NextBackOff() time.Duration
    Reset()
}

Any type implementing these two methods can be passed to retry.Retry.

Configuration Struct

For convenience, the repository provides a protobuf-generated configuration struct in backoff/backoff.pb.go:

type Backoff struct {
    InitialInterval     time.Duration
    RandomizationFactor float64
    Multiplier          float64
    MaxInterval         time.Duration
    MaxElapsedTime      time.Duration
}

Method 1: Configure Custom Backoff Using Protobuf Config

The simplest way to customize backoff behavior is to populate the backoff.Backoff struct and pass it to retry.NewBackOff.

package main

import (
    "time"
    
    "github.com/aperturerobotics/util/backoff"
    "github.com/aperturerobotics/util/retry"
)

func createCustomBackoff() backoff.BackOff {
    conf := &backoff.Backoff{
        InitialInterval:     500 * time.Millisecond,
        RandomizationFactor: 0.25,
        Multiplier:          1.5,
        MaxInterval:         10 * time.Second,
        MaxElapsedTime:      2 * time.Minute,
    }
    
    return retry.NewBackOff(conf)
}

retry.NewBackOff returns a backoff.BackOff implementation that uses exponential backoff with the parameters you specified.

Method 2: Implement a Custom BackOff Interface

For algorithms not supported by the default exponential implementation, implement the interface directly. Below is a linear backoff example that increases the wait time by a fixed step on each failure.

package main

import (
    "time"
    
    "github.com/aperturerobotics/util/backoff"
)

type linearBackOff struct {
    step time.Duration
    max  time.Duration
    cur  time.Duration
}

func (b *linearBackOff) NextBackOff() time.Duration {
    if b.cur == 0 {
        b.cur = b.step
    } else {
        b.cur += b.step
        if b.cur > b.max {
            b.cur = b.max
        }
    }
    return b.cur
}

func (b *linearBackOff) Reset() {
    b.cur = 0
}

// Ensure interface compliance at compile time.
var _ backoff.BackOff = (*linearBackOff)(nil)

This implementation satisfies the backoff.BackOff interface and can be passed directly to retry.Retry without using retry.NewBackOff.

Wiring It All Together

To use your custom backoff, wire it into the retry loop along with your operation. The operation function must accept a success callback to reset the backoff timer upon success.

package main

import (
    "context"
    "fmt"
    "net/http"
    "time"

    "github.com/aperturerobotics/util/backoff"
    "github.com/aperturerobotics/util/retry"
    "github.com/sirupsen/logrus"
)

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
    defer cancel()

    logger := logrus.New().WithField("service", "api-client")

    // Use the linear backoff from Method 2
    bo := &linearBackOff{
        step: 1 * time.Second,
        max:  30 * time.Second,
    }

    err := retry.Retry(ctx, logger, func(ctx context.Context, success func()) error {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.example.com/v1/status", nil)
        if err != nil {
            return err
        }

        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return err
        }
        defer resp.Body.Close()

        if resp.StatusCode == http.StatusOK {
            success() // Reset backoff timer
            return nil
        }
        return fmt.Errorf("unexpected status: %d", resp.StatusCode)
    }, bo)

    if err != nil {
        logger.WithError(err).Fatal("operation failed")
    }
    logger.Info("operation succeeded")
}

Key Source Files in the Repository

• retry/retry.go – Contains Retry(), NewBackOff(), and DefaultBackoff(). View source
• backoff/cbackoff/backoff.go – Defines the BackOff interface (NextBackOff, Reset). View source
• backoff/backoff.pb.go – Protobuf-generated Backoff configuration struct. View source
• backoff/cbackoff/exponential.go – Exponential backoff implementation used by NewBackOff. View source
• backoff/cbackoff/tries.go – Fixed-tries backoff for limited attempt counts. View source

Summary

• The retry package in aperturerobotics/util accepts any type implementing the backoff.BackOff interface, allowing you to integrate the retry package with custom backoff logic seamlessly.
• Use retry.NewBackOff() to convert a protobuf backoff.Backoff configuration into a concrete exponential backoff implementation.
• Implement NextBackOff() and Reset() directly for bespoke algorithms like linear backoff or token-bucket rate limiting.
• Always invoke the success callback inside your operation function to reset the backoff timer when the operation succeeds.
• The retry loop respects context cancellation and logs each attempt using the provided logrus.Entry.

Frequently Asked Questions

What interface must my custom backoff implement to work with the retry package?

Your custom backoff must implement the backoff.BackOff interface defined in backoff/cbackoff/backoff.go. This interface requires two methods: NextBackOff() time.Duration, which returns the duration to wait before the next retry, and Reset(), which resets the backoff state to its initial values.

How do I reset the backoff timer after a successful operation?

The operation function passed to retry.Retry receives a success callback as its second argument. When your operation succeeds, invoke this callback (success()), which internally calls Reset() on your backoff instance. This ensures that the next failure starts from the initial interval rather than continuing from the previous backoff duration.

Can I use a fixed number of retries instead of time-based backoff?

Yes. The repository includes a fixed-tries implementation in backoff/cbackoff/tries.go. You can use this implementation or create your own that tracks attempt counts in the NextBackOff() method and returns backoff.Stop (a zero duration or specific sentinel value depending on implementation) when the maximum number of attempts is reached.

Where is the main retry loop implementation located?

The main retry loop is implemented in retry/retry.go within the aperturerobotics/util repository. This file contains the Retry() function that orchestrates the execution of your operation, handles the backoff intervals, manages context cancellation, and logs retry attempts using the provided logrus.Entry.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}