How HITL (Human-in-the-Loop) Works in the AskdataAgent Workflow

HITL in AskdataAgent leverages asynchronous CompletableFuture primitives to pause LLM execution, enabling users to provide missing data via the askUser tool or approve sensitive tool invocations through real-time stream events like HITL_TOOL_APPROVAL.

The junjiem/dat repository implements a robust Human-in-the-Loop (HITL) mechanism that allows Askdata agents to operate either fully autonomously or with strategic human oversight. This implementation centers on two primary classes—AbstractHitlAskdataAgent in the core module and AgenticAskdataAgent in the agentic module—which together provide the asynchronous infrastructure for blocking execution and capturing user input through REST endpoints.

Core Architecture Components

AbstractHitlAskdataAgent

The abstract base class located at dat-core/src/main/java/ai/dat/core/agent/AbstractHitlAskdataAgent.java manages the low-level asynchronous infrastructure for HITL interactions. It maintains CompletableFuture instances for both user responses and approvals, providing blocking methods such as waitForUserResponse() and waitForUserApproval() that the agent invokes when human input is required.

AgenticAskdataAgent

The concrete implementation in dat-agents/dat-agent-agentic/src/main/java/ai/dat/agent/agentic/AgenticAskdataAgent.java wires HITL capabilities into the LangChain4j tool-execution pipeline. This class handles tool registration, approval wrappers, and event emission, bridging the gap between the asynchronous HITL primitives and the streaming AI workflow.

The HITL Execution Flow

1. Configuration and Initialization

During agent construction via AgenticAskdataAgent.Builder (lines 110-138), boolean flags determine which HITL features are active:

• humanInTheLoop: Master switch enabling HITL capabilities
• humanInTheLoopAskUser: Activates the askUser tool for gathering missing information
• humanInTheLoopToolApproval: Requires explicit approval before executing sensitive tools
• humanInTheLoopToolNotApprovalAndFeedback: Captures user feedback when rejecting tool executions

2. Tool Registration and the askUser Tool

When createTools() executes (lines 508-569), the agent conditionally registers the askUser tool if humanInTheLoop && humanInTheLoopAskUser is true. The tool's executor calls waitForUserResponse() from the parent AbstractHitlAskdataAgent, creating a blocking point where the LLM thread pauses until the user provides input.

3. Tool Approval and Execution Hooks

The beforeToolExecution() method (lines 165-188) intercepts every tool invocation through LangChain4j. When HITL is enabled:

• For askUser tool invocations, it emits a HITL_AI_REQUEST stream event containing the AI's question text
• For other tools when humanInTheLoopToolApproval is true, it emits a HITL_TOOL_APPROVAL event prompting the UI to request explicit confirmation

4. Asynchronous User Response Handling

User interactions complete through REST endpoints that invoke userResponse(String) or userApproval(Boolean) on AbstractHitlAskdataAgent (lines 41-54). These methods complete the corresponding CompletableFuture instances stored in the agent, unblocking waitForUserResponse() or waitForUserApproval() and allowing the tool to return its result to the LLM.

5. Tool Execution After Approval

In createToolProvider() (lines 333-362), a toolWrapper consults isToolApproval() before delegating to the actual ToolExecutor. If the user approved, the original tool executes normally. If rejected, the wrapper returns TOOL_NOT_APPROVAL_MESSAGE (optionally augmented with user feedback if humanInTheLoopToolNotApprovalAndFeedback is enabled), allowing the LLM to see the refusal and adjust its strategy.

Practical Implementation Examples

Enabling HITL in Agent Configuration

import ai.dat.agent.agentic.AgenticAskdataAgent;
import ai.dat.core.agent.AskdataAgent;

// Build a HITL-enabled agent
AskdataAgent agent = AgenticAskdataAgent.builder()
        .contentStore(myContentStore)
        .databaseAdapter(myDbAdapter)
        .defaultModel(myChatModel)
        .defaultStreamingModel(myStreamingModel)
        .text2sqlModel(myText2SqlModel)
        // Enable all HITL features
        .humanInTheLoop(true)                     // master switch
        .humanInTheLoopAskUser(true)              // askUser tool active
        .humanInTheLoopToolApproval(true)         // require approval before any tool
        .humanInTheLoopToolNotApprovalAndFeedback(true) // ask for feedback on rejection
        .build();

Triggering the askUser Tool Flow

When the LLM determines it needs missing information, it generates a tool invocation:

{
  "name": "askUser",
  "arguments": {
    "request": "What date range should the sales report cover?"
  }
}

LangChain4j forwards this to AgenticAskdataAgent.createTools(). Because humanInTheLoopAskUser is true, the tool executes waitForUserResponse(), blocking the LLM thread. Simultaneously, beforeToolExecution() emits a HITL_AI_REQUEST stream event to the UI. The user submits their answer via the /user-response endpoint, which calls userResponse(String) to complete the CompletableFuture, returning the text to the LLM and resuming execution.

Handling Tool Approval with Feedback

For sensitive operations like sendEmail:

{
  "name": "sendEmail",
  "arguments": {
    "recipients": "alice@example.com,bob@example.com",
    "subject": "Monthly Sales",
    "content": "Please find attached...",
    "isHtml": false
  }
}

With humanInTheLoopToolApproval enabled, beforeToolExecution() emits a HITL_TOOL_APPROVAL event. The UI presents an approval dialog. If the user rejects and humanInTheLoopToolNotApprovalAndFeedback is true, the system waits up to 30 seconds for feedback text, then returns a refusal message containing that feedback to the LLM via the tool wrapper in createToolProvider().

Key Source Files and Implementation Details

File	Role	Key Methods
dat-core/src/main/java/ai/dat/core/agent/AbstractHitlAskdataAgent.java	Base class managing CompletableFuture instances for async user interaction	waitForUserResponse(), waitForUserApproval(), userResponse(String), userApproval(Boolean)
dat-agents/dat-agent-agentic/src/main/java/ai/dat/agent/agentic/AgenticAskdataAgent.java	Concrete implementation wiring HITL into LangChain4j	createTools(), beforeToolExecution(), createToolProvider(), isToolApproval()
dat-core/src/main/java/ai/dat/core/factories/AskdataAgentFactory.java	Factory interface for agent instantiation	Factory methods supporting HITL configuration
dat-core/src/main/java/ai/dat/core/utils/FactoryUtil.java	Helper creating agents from configuration	createAskdataAgent() passing HITL flags from YAML
dat-sdk/src/main/java/ai/dat/boot/utils/ProjectUtil.java	High-level entry point for CLI/server	Calls FactoryUtil.createAskdataAgent
dat-core/src/main/java/ai/dat/core/agent/data/StreamAction.java	Captures streaming events for UI rendering	action.add() for HITL events

Summary

• HITL is opt-in via boolean configuration flags (humanInTheLoop, humanInTheLoopAskUser, humanInTheLoopToolApproval) set during agent construction in AgenticAskdataAgent.Builder.
• Asynchronous blocking is implemented through CompletableFuture in AbstractHitlAskdataAgent, allowing the LLM to pause without consuming resources while awaiting human input via REST endpoints.
• Dual interaction modes support both information gathering via the askUser tool (emitting HITL_AI_REQUEST events) and safety controls via tool approval (emitting HITL_TOOL_APPROVAL events).
• Feedback loops can be enabled to capture user comments when rejecting tool executions, enriching the refusal message returned to the LLM through the tool wrapper in createToolProvider().
• Stream events drive the UI through StreamAction, ensuring real-time rendering of approval dialogs and question prompts without polling.

Frequently Asked Questions

What is the difference between the askUser tool and tool approval in AskdataAgent?

The askUser tool is designed for information gathering—when the LLM needs missing data to complete a task, it invokes this tool to ask the user a specific question and waits for a text response. Tool approval, conversely, is a safety mechanism that intercepts potentially sensitive tool executions (like sending emails or running MCP tools) and requires explicit user confirmation before proceeding. While askUser returns user input to the LLM as data, tool approval returns a boolean decision that either allows execution or returns a refusal message to the LLM.

How does the agent handle timeouts when waiting for user input?

The AbstractHitlAskdataAgent class provides timeout-aware methods such as waitForUserResponse(long timeout, TimeUnit unit) and waitForUserApproval(long timeout, TimeUnit unit). If the user does not respond within the specified duration, these methods throw a TimeoutException, allowing the agent to catch the exception and continue with a default fallback path or error handling logic rather than blocking indefinitely.

Can HITL be enabled for specific tools only, or is it all-or-nothing?

Currently, the implementation uses global boolean flags (humanInTheLoopToolApproval) that apply to all tools when enabled. However, the architecture in createToolProvider() (lines 333-362) uses a toolWrapper that consults isToolApproval() before execution. This design allows for future extension where specific tool types or names could be filtered, though the current AgenticAskdataAgent implementation treats the flag as a global setting for all non-askUser tools.

What happens to the LLM conversation when a tool is rejected by the user?

When a user rejects a tool execution via the approval mechanism, the toolWrapper in createToolProvider() returns a static TOOL_NOT_APPROVAL_MESSAGE (or an augmented version containing user feedback if humanInTheLoopToolNotApprovalAndFeedback is enabled). This message is returned to the LLM as the tool's execution result, allowing the LLM to see that the operation was denied and potentially adjust its strategy, ask for clarification, or proceed with alternative approaches based on the refusal context.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}