# How to Handle Tool Failures and Implement Retries for Agent Actions in Embabel

> Learn to handle tool failures and implement retries for agent actions in Embabel. Discover built-in failure detection and configurable retry policies for robust agent behavior.

- Repository: [Embabel/embabel-agent](https://github.com/embabel/embabel-agent)
- Tags: how-to-guide
- Published: 2026-08-10

---

**Embabel Agent provides built-in failure detection via `ToolCallOutcomes` and configurable retry policies through the `ActionRetryPolicy` enum and Spring Expression Language (SpEL) expressions.**

The `embabel/embabel-agent` framework offers robust primitives to handle tool failures and implement retries for agent actions in Embabel. By leveraging event listeners and quality-of-service (QoS) policies, you can ensure your agents gracefully recover from external tool failures without manual intervention.

## Detecting Tool Failures with Event Listeners

Every tool invocation generates a `ToolCallRequestEvent` and a corresponding `ToolCallResponseEvent`. The static helper `ToolCallOutcomes` extracts results or errors from these responses using `ToolCallOutcomes.error(event)`.

In [`EmbabelSpanEventListener.java`](https://github.com/embabel/embabel-agent/blob/main/EmbabelSpanEventListener.java), the framework inspects this error to determine whether an agent should re-plan or retry an action. When a tool call throws an exception, the listener records the failure and adds the `"embabel.replan.reason"` attribute with the value `"stuck, retrying with new plan"`, signaling the planner to create a fresh execution plan.

## Configuring Built-In Retry Policies

The `ActionRetryPolicy` enum in [`ActionRetryPolicy.kt`](https://github.com/embabel/embabel-agent/blob/main/ActionRetryPolicy.kt) defines two built-in strategies:

- **FIRE_ONCE**: Maps to `maxAttempts = 1`, executing the action exactly once regardless of failure.
- **DEFAULT**: Applies standard QoS settings with `maxAttempts = 5` and exponential backoff starting at 10 seconds.

The [`DefaultActionQosProvider.kt`](https://github.com/embabel/embabel-agent/blob/main/DefaultActionQosProvider.kt) consults this policy when building `ActionQos` for `@Action` annotated methods. If you specify `FIRE_ONCE`, the generated QoS forces a single attempt; otherwise, the framework applies the default retry configuration defined in [`ActionQos.kt`](https://github.com/embabel/embabel-agent/blob/main/ActionQos.kt).

## Customizing Retries with SpEL Expressions

For externalized configuration, use the `actionRetryPolicyExpression` attribute on the `@Action` annotation. This SpEL expression resolves against the Spring `Environment`, allowing you to define retry parameters in [`application.yml`](https://github.com/embabel/embabel-agent/blob/main/application.yml) or other property sources.

As demonstrated in [`ActionQosPropertyProviderTest.kt`](https://github.com/embabel/embabel-agent/blob/main/ActionQosPropertyProviderTest.kt) (lines 176-179), you can reference fully qualified QoS configurations:

```kotlin
@Action(actionRetryPolicyExpression = "\${retry-twice}")
fun fetchData(context: AgentProcess) {
    val result = context.toolCall("DataService", mapOf("id" to "123"))
}

```

With corresponding YAML configuration:

```yaml
retry-twice:
  max-attempts: 2
  backoff-millis: 5000
  backoff-multiplier: 2.0
  backoff-max-interval: 20000

```

## The Re-Planning Mechanism

When `ToolCallOutcomes.error(event)` returns an exception, the `EmbabelSpanEventListener` adds the `embabel.replan.reason` attribute to the span. As verified in [`EmbabelSpanEventListenerTest.java`](https://github.com/embabel/embabel-agent/blob/main/EmbabelSpanEventListenerTest.java) (lines 531-537), this attribute triggers the planner to abort the current plan and create a new one that respects the action's retry policy. The new plan generation occurs automatically without requiring custom error handling code in your agent logic.

## Practical Implementation Examples

### Basic Fire-Once Policy

Use `FIRE_ONCE` for actions that should not retry under any circumstances:

```kotlin
@Agent(description = "Demo agent")
class SimpleAgent {

    @Action(actionRetryPolicy = ActionRetryPolicy.FIRE_ONCE)
    fun riskyToolCall(context: AgentProcess) {
        val result = context.toolCall("UnreliableWebSearch", mapOf("query" to "latest news"))
        println("Result: $result")
    }
}

```

### Externalized Configuration

Reference external properties for flexible deployment-specific retry logic:

```kotlin
@Agent(description = "Configurable retry example")
class ConfigurableAgent {

    @Action(actionRetryPolicyExpression = "\${my.retry.policy}")
    fun fetchWeather(context: AgentProcess) {
        val weather = context.toolCall("WeatherService", mapOf("city" to "London"))
        println("Weather: $weather")
    }
}

```

```yaml
my:
  retry:
    policy:
      max-attempts: 3
      backoff-millis: 2000
      backoff-multiplier: 2.0
      backoff-max-interval: 15000
      idempotent: true

```

### Custom Failure Listener

Implement `ToolCallListener` to add custom observability or alerting:

```java
public class MyToolFailureListener implements ToolCallListener {
    
    @Override
    public void onToolCallResponse(ToolCallResponseEvent event) {
        Throwable error = ToolCallOutcomes.error(event);
        if (error != null) {
            log.warn("Tool '{}' failed: {}", event.getRequest().getToolName(), error.getMessage());
            // Additional custom logic here
        }
    }
}

```

## Summary

- **Tool failure detection** relies on `ToolCallOutcomes.error(event)` within [`EmbabelSpanEventListener.java`](https://github.com/embabel/embabel-agent/blob/main/EmbabelSpanEventListener.java) to extract exceptions from `ToolCallResponseEvent` instances.
- **Retry policies** are controlled via the `ActionRetryPolicy` enum (`FIRE_ONCE` or `DEFAULT`) processed by [`DefaultActionQosProvider.kt`](https://github.com/embabel/embabel-agent/blob/main/DefaultActionQosProvider.kt).
- **Custom configurations** use SpEL expressions (`actionRetryPolicyExpression`) to externalize `ActionQos` parameters in [`application.yml`](https://github.com/embabel/embabel-agent/blob/main/application.yml) or properties files.
- **Re-planning** occurs automatically when the framework sets the `embabel.replan.reason` attribute after detecting tool errors, triggering the planner to create a fresh execution strategy.

## Frequently Asked Questions

### How does Embabel detect when a tool call fails?

Embabel generates `ToolCallRequestEvent` and `ToolCallResponseEvent` for every invocation. The `ToolCallOutcomes.error(event)` static method extracts any exception from the response event. The [`EmbabelSpanEventListener.java`](https://github.com/embabel/embabel-agent/blob/main/EmbabelSpanEventListener.java) inspects this result to trigger retry or re-planning logic when errors are present.

### What is the difference between FIRE_ONCE and DEFAULT retry policies?

`FIRE_ONCE` forces exactly one execution attempt regardless of outcome, while `DEFAULT` applies the standard QoS configuration with up to 5 retry attempts and exponential backoff starting at 10 seconds. These policies are defined in [`ActionRetryPolicy.kt`](https://github.com/embabel/embabel-agent/blob/main/ActionRetryPolicy.kt) and applied by [`DefaultActionQosProvider.kt`](https://github.com/embabel/embabel-agent/blob/main/DefaultActionQosProvider.kt) when constructing the `ActionQos` for annotated methods.

### Can I configure retry parameters in application.properties instead of code?

Yes. Use the `actionRetryPolicyExpression` attribute on `@Action` to reference a SpEL expression that resolves to an `ActionQos` configuration defined in your Spring properties. This allows externalized configuration of `max-attempts`, `backoff-millis`, and other parameters without modifying source code, as demonstrated in [`ActionQosPropertyProviderTest.kt`](https://github.com/embabel/embabel-agent/blob/main/ActionQosPropertyProviderTest.kt).

### What happens when all retry attempts are exhausted?

When the maximum attempts defined in the `ActionQos` are exhausted, the framework sets the `embabel.replan.reason` attribute to `"stuck, retrying with new plan"` and triggers the planner to create a fresh execution plan. This behavior is verified in [`EmbabelSpanEventListenerTest.java`](https://github.com/embabel/embabel-agent/blob/main/EmbabelSpanEventListenerTest.java) (lines 531-537), ensuring that stuck actions receive new planning rather than failing permanently.