# How HITL (Human-in-the-Loop) Works in the AskdataAgent Workflow

> Understand how HITL in AskdataAgent works by pausing LLM execution for user input or tool approval via real time stream events. Learn more about asynchronous CompletableFuture primitives.

- Repository: [Junjie.M/dat](https://github.com/junjiem/dat)
- Tags: deep-dive
- Published: 2026-03-05

---

**HITL in AskdataAgent leverages asynchronous `CompletableFuture` primitives to pause LLM execution, enabling users to provide missing data via the `askUser` tool or approve sensitive tool invocations through real-time stream events like `HITL_TOOL_APPROVAL`.**

The junjiem/dat repository implements a robust Human-in-the-Loop (HITL) mechanism that allows Askdata agents to operate either fully autonomously or with strategic human oversight. This implementation centers on two primary classes—`AbstractHitlAskdataAgent` in the core module and `AgenticAskdataAgent` in the agentic module—which together provide the asynchronous infrastructure for blocking execution and capturing user input through REST endpoints.

## Core Architecture Components

### AbstractHitlAskdataAgent

The abstract base class located at [`dat-core/src/main/java/ai/dat/core/agent/AbstractHitlAskdataAgent.java`](https://github.com/junjiem/dat/blob/main/dat-core/src/main/java/ai/dat/core/agent/AbstractHitlAskdataAgent.java) manages the low-level asynchronous infrastructure for HITL interactions. It maintains `CompletableFuture` instances for both user responses and approvals, providing blocking methods such as `waitForUserResponse()` and `waitForUserApproval()` that the agent invokes when human input is required.

### AgenticAskdataAgent

The concrete implementation in [`dat-agents/dat-agent-agentic/src/main/java/ai/dat/agent/agentic/AgenticAskdataAgent.java`](https://github.com/junjiem/dat/blob/main/dat-agents/dat-agent-agentic/src/main/java/ai/dat/agent/agentic/AgenticAskdataAgent.java) wires HITL capabilities into the LangChain4j tool-execution pipeline. This class handles tool registration, approval wrappers, and event emission, bridging the gap between the asynchronous HITL primitives and the streaming AI workflow.

## The HITL Execution Flow

### 1. Configuration and Initialization

During agent construction via `AgenticAskdataAgent.Builder` (lines 110-138), boolean flags determine which HITL features are active:

- `humanInTheLoop`: Master switch enabling HITL capabilities
- `humanInTheLoopAskUser`: Activates the `askUser` tool for gathering missing information
- `humanInTheLoopToolApproval`: Requires explicit approval before executing sensitive tools
- `humanInTheLoopToolNotApprovalAndFeedback`: Captures user feedback when rejecting tool executions

### 2. Tool Registration and the askUser Tool

When `createTools()` executes (lines 508-569), the agent conditionally registers the `askUser` tool if `humanInTheLoop && humanInTheLoopAskUser` is true. The tool's executor calls `waitForUserResponse()` from the parent `AbstractHitlAskdataAgent`, creating a blocking point where the LLM thread pauses until the user provides input.

### 3. Tool Approval and Execution Hooks

The `beforeToolExecution()` method (lines 165-188) intercepts every tool invocation through LangChain4j. When HITL is enabled:

- For `askUser` tool invocations, it emits a `HITL_AI_REQUEST` stream event containing the AI's question text
- For other tools when `humanInTheLoopToolApproval` is true, it emits a `HITL_TOOL_APPROVAL` event prompting the UI to request explicit confirmation

### 4. Asynchronous User Response Handling

User interactions complete through REST endpoints that invoke `userResponse(String)` or `userApproval(Boolean)` on `AbstractHitlAskdataAgent` (lines 41-54). These methods complete the corresponding `CompletableFuture` instances stored in the agent, unblocking `waitForUserResponse()` or `waitForUserApproval()` and allowing the tool to return its result to the LLM.

### 5. Tool Execution After Approval

In `createToolProvider()` (lines 333-362), a `toolWrapper` consults `isToolApproval()` before delegating to the actual `ToolExecutor`. If the user approved, the original tool executes normally. If rejected, the wrapper returns `TOOL_NOT_APPROVAL_MESSAGE` (optionally augmented with user feedback if `humanInTheLoopToolNotApprovalAndFeedback` is enabled), allowing the LLM to see the refusal and adjust its strategy.

## Practical Implementation Examples

### Enabling HITL in Agent Configuration

```java
import ai.dat.agent.agentic.AgenticAskdataAgent;
import ai.dat.core.agent.AskdataAgent;

// Build a HITL-enabled agent
AskdataAgent agent = AgenticAskdataAgent.builder()
        .contentStore(myContentStore)
        .databaseAdapter(myDbAdapter)
        .defaultModel(myChatModel)
        .defaultStreamingModel(myStreamingModel)
        .text2sqlModel(myText2SqlModel)
        // Enable all HITL features
        .humanInTheLoop(true)                     // master switch
        .humanInTheLoopAskUser(true)              // askUser tool active
        .humanInTheLoopToolApproval(true)         // require approval before any tool
        .humanInTheLoopToolNotApprovalAndFeedback(true) // ask for feedback on rejection
        .build();

```

### Triggering the askUser Tool Flow

When the LLM determines it needs missing information, it generates a tool invocation:

```json
{
  "name": "askUser",
  "arguments": {
    "request": "What date range should the sales report cover?"
  }
}

```

LangChain4j forwards this to `AgenticAskdataAgent.createTools()`. Because `humanInTheLoopAskUser` is true, the tool executes `waitForUserResponse()`, blocking the LLM thread. Simultaneously, `beforeToolExecution()` emits a `HITL_AI_REQUEST` stream event to the UI. The user submits their answer via the `/user-response` endpoint, which calls `userResponse(String)` to complete the `CompletableFuture`, returning the text to the LLM and resuming execution.

### Handling Tool Approval with Feedback

For sensitive operations like `sendEmail`:

```json
{
  "name": "sendEmail",
  "arguments": {
    "recipients": "alice@example.com,bob@example.com",
    "subject": "Monthly Sales",
    "content": "Please find attached...",
    "isHtml": false
  }
}

```

With `humanInTheLoopToolApproval` enabled, `beforeToolExecution()` emits a `HITL_TOOL_APPROVAL` event. The UI presents an approval dialog. If the user rejects and `humanInTheLoopToolNotApprovalAndFeedback` is true, the system waits up to 30 seconds for feedback text, then returns a refusal message containing that feedback to the LLM via the tool wrapper in `createToolProvider()`.

## Key Source Files and Implementation Details

| File | Role | Key Methods |
|------|------|-------------|
| [`dat-core/src/main/java/ai/dat/core/agent/AbstractHitlAskdataAgent.java`](https://github.com/junjiem/dat/blob/main/dat-core/src/main/java/ai/dat/core/agent/AbstractHitlAskdataAgent.java) | Base class managing `CompletableFuture` instances for async user interaction | `waitForUserResponse()`, `waitForUserApproval()`, `userResponse(String)`, `userApproval(Boolean)` |
| [`dat-agents/dat-agent-agentic/src/main/java/ai/dat/agent/agentic/AgenticAskdataAgent.java`](https://github.com/junjiem/dat/blob/main/dat-agents/dat-agent-agentic/src/main/java/ai/dat/agent/agentic/AgenticAskdataAgent.java) | Concrete implementation wiring HITL into LangChain4j | `createTools()`, `beforeToolExecution()`, `createToolProvider()`, `isToolApproval()` |
| [`dat-core/src/main/java/ai/dat/core/factories/AskdataAgentFactory.java`](https://github.com/junjiem/dat/blob/main/dat-core/src/main/java/ai/dat/core/factories/AskdataAgentFactory.java) | Factory interface for agent instantiation | Factory methods supporting HITL configuration |
| [`dat-core/src/main/java/ai/dat/core/utils/FactoryUtil.java`](https://github.com/junjiem/dat/blob/main/dat-core/src/main/java/ai/dat/core/utils/FactoryUtil.java) | Helper creating agents from configuration | `createAskdataAgent()` passing HITL flags from YAML |
| [`dat-sdk/src/main/java/ai/dat/boot/utils/ProjectUtil.java`](https://github.com/junjiem/dat/blob/main/dat-sdk/src/main/java/ai/dat/boot/utils/ProjectUtil.java) | High-level entry point for CLI/server | Calls `FactoryUtil.createAskdataAgent` |
| [`dat-core/src/main/java/ai/dat/core/agent/data/StreamAction.java`](https://github.com/junjiem/dat/blob/main/dat-core/src/main/java/ai/dat/core/agent/data/StreamAction.java) | Captures streaming events for UI rendering | `action.add()` for HITL events |

## Summary

- **HITL is opt-in** via boolean configuration flags (`humanInTheLoop`, `humanInTheLoopAskUser`, `humanInTheLoopToolApproval`) set during agent construction in `AgenticAskdataAgent.Builder`.
- **Asynchronous blocking** is implemented through `CompletableFuture` in `AbstractHitlAskdataAgent`, allowing the LLM to pause without consuming resources while awaiting human input via REST endpoints.
- **Dual interaction modes** support both information gathering via the `askUser` tool (emitting `HITL_AI_REQUEST` events) and safety controls via tool approval (emitting `HITL_TOOL_APPROVAL` events).
- **Feedback loops** can be enabled to capture user comments when rejecting tool executions, enriching the refusal message returned to the LLM through the tool wrapper in `createToolProvider()`.
- **Stream events** drive the UI through `StreamAction`, ensuring real-time rendering of approval dialogs and question prompts without polling.

## Frequently Asked Questions

### What is the difference between the `askUser` tool and tool approval in AskdataAgent?

The `askUser` tool is designed for **information gathering**—when the LLM needs missing data to complete a task, it invokes this tool to ask the user a specific question and waits for a text response. Tool approval, conversely, is a **safety mechanism** that intercepts potentially sensitive tool executions (like sending emails or running MCP tools) and requires explicit user confirmation before proceeding. While `askUser` returns user input to the LLM as data, tool approval returns a boolean decision that either allows execution or returns a refusal message to the LLM.

### How does the agent handle timeouts when waiting for user input?

The `AbstractHitlAskdataAgent` class provides timeout-aware methods such as `waitForUserResponse(long timeout, TimeUnit unit)` and `waitForUserApproval(long timeout, TimeUnit unit)`. If the user does not respond within the specified duration, these methods throw a `TimeoutException`, allowing the agent to catch the exception and continue with a default fallback path or error handling logic rather than blocking indefinitely.

### Can HITL be enabled for specific tools only, or is it all-or-nothing?

Currently, the implementation uses global boolean flags (`humanInTheLoopToolApproval`) that apply to all tools when enabled. However, the architecture in `createToolProvider()` (lines 333-362) uses a `toolWrapper` that consults `isToolApproval()` before execution. This design allows for future extension where specific tool types or names could be filtered, though the current `AgenticAskdataAgent` implementation treats the flag as a global setting for all non-`askUser` tools.

### What happens to the LLM conversation when a tool is rejected by the user?

When a user rejects a tool execution via the approval mechanism, the `toolWrapper` in `createToolProvider()` returns a static `TOOL_NOT_APPROVAL_MESSAGE` (or an augmented version containing user feedback if `humanInTheLoopToolNotApprovalAndFeedback` is enabled). This message is returned to the LLM as the tool's execution result, allowing the LLM to see that the operation was denied and potentially adjust its strategy, ask for clarification, or proceed with alternative approaches based on the refusal context.