How Flows Copilot Agents Propose Automation Graphs in OpenHuman

OpenHuman's Flows copilot agents, WorkflowBuilder and FlowDiscovery, analyze user intent and contextual data to automatically generate or retrieve automation graphs, validating them through the TinyFlows engine before presenting them as runnable workflow suggestions.

The OpenHuman platform enables intelligent automation through its Flows feature, which relies on specialized copilot agents to suggest workflow graphs. These agents—WorkflowBuilder and FlowDiscovery—operate within the src/openhuman/flows/ domain to transform natural language prompts into executable automation graphs. Understanding how these flows copilot agents propose automation graphs reveals the sophisticated orchestration between context gathering, graph construction, and safety validation in the OpenHuman architecture.

Architecture of the Flow Copilot System

The copilot functionality centers on two distinct agents that collaborate with the TinyFlows Engine to produce valid workflow definitions. Both agents interface with the core system through JSON-RPC controllers and share a unified execution harness.

WorkflowBuilder Agent

The WorkflowBuilder agent specializes in creating new automation graphs from scratch. Located in src/openhuman/flows/agents/workflowbuilder.rs, this agent accepts natural language prompts, gathers relevant tool and memory data, and constructs a fresh WorkflowDefinition using the TinyFlows DSL. It annotates each node with capability requirements—such as "requires webhook tool"—and packages the result with a descriptive summary before returning it as a suggested workflow.

FlowDiscovery Agent

The FlowDiscovery agent focuses on retrieving and ranking existing workflows that match current user needs. Implemented in src/openhuman/flows/agents/flowdiscovery.rs, this agent scans the user's workspace history, recent thread contexts, and the skill catalog stored in the TinyFlows database. It loads workflow drafts and scores them against the current context using similarity functions defined in src/openhuman/flows/ops.rs, returning the highest-ranked candidates for potential reuse or adaptation.

TinyFlows Engine and Safety Layer

Both agents rely on the TinyFlows Engine (src/openhuman/flows/tinyflows/engine.rs) to execute and validate graph definitions. This engine manages workflow revisions, stores execution state, and enforces safety constraints through the tinyflows::caps:: modules. Before any suggested graph reaches the user, the engine checks for disallowed side effects or insecure operations, stripping violations or falling back to safe templates when necessary.

How the Agents Construct Automation Graphs

The process of proposing automation graphs follows a structured pipeline that transforms user intent into validated workflow definitions. The Agent Harness (src/openhuman/agent/harness.rs) provides the unified "turn" execution loop that coordinates these steps.

Gathering Context Through Tool Calls

When the frontend invokes openhuman.flows_suggest or openhuman.flows_discover via the JSON-RPC dispatcher in src/core/jsonrpc.rs, the harness creates a TurnContext and initiates the appropriate agent. The agents then execute tool calls to gather contextual data:

• WorkflowBuilder typically calls memory_search, skill_list, or tool_catalog to understand available capabilities and relevant historical actions.
• FlowDiscovery queries the workflow store and memory domain to retrieve recent usage patterns and existing graph structures.

These tools feed raw data back into the agent's context window, enabling informed decision-making about graph composition.

Building New Graphs with WorkflowBuilder

After collecting context, the WorkflowBuilder agent constructs the automation graph programmatically. It instantiates a WorkflowDefinition struct from the tinyflows crate, adds nodes representing specific actions, connects edges to define execution flow, and validates that each node references existing tools or skills. The agent serializes the completed graph to JSON using serde_json and wraps it in a suggestion payload that includes a natural language description of what the workflow accomplishes.

Scoring and Selecting Existing Flows

Rather than generating new graphs, FlowDiscovery leverages the scoring function in src/openhuman/flows/ops.rs to evaluate existing workflows. It compares stored workflows against the current search context—considering factors like recent usage, semantic similarity to the user's prompt, and compatibility with available tools. The agent selects the top-ranked candidates and returns them as adaptation candidates, allowing users to modify existing automation rather than building from zero.

Safety and Policy Validation

Before returning any proposal to the frontend, the agents invoke flow-policy checks through the TinyFlows safety caps. This validation layer examines the graph for unauthorized side effects, permission violations, or deprecated tool usage. Graphs that fail validation are either sanitized—removing problematic nodes—or discarded in favor of conservative default templates. This ensures that users receive suggestions that are not only relevant but also safe to execute within their permission scope.

Implementation Examples

Developers can interact with these copilot agents through the JSON-RPC interface or directly via Rust APIs for testing and custom integrations.

Triggering Suggestions from the Frontend

The React frontend communicates with WorkflowBuilder through the core RPC client:

import { coreRpcClient } from '@/services/coreRpcClient';

async function suggestWorkflow(prompt: string) {
  const result = await coreRpcClient.call('openhuman.flows_suggest', {
    user_prompt: prompt,
    max_suggestions: 3,
  });

  return result.suggestions;
}

This returns an array of WorkflowDefinition objects ready for rendering in the Flows UI.

Direct Agent Invocation in Rust

For backend testing or custom automation, invoke the agent directly through the Harness:

use openhuman_core::harness::Harness;

#[tokio::test]
async fn test_workflowbuilder_suggest() {
    let harness = Harness::builder()
        .provider(openhuman_core::provider::mock())
        .access(openhuman_core::access::full())
        .build()
        .await
        .unwrap();

    let resp = harness
        .turn("Suggest a workflow to back‑up my photos to the cloud.")
        .agent("workflowbuilder")
        .send()
        .await
        .unwrap();

    let workflow: openhuman_flows::tinyflows::WorkflowDefinition =
        serde_json::from_str(&resp.content).unwrap();
    println!("Proposed workflow: {}", workflow.name);
}

Scoring Existing Workflows

When implementing custom discovery logic, use the FlowStore and scoring functions:

use openhuman_flows::tinyflows::store::FlowStore;
use openhuman_flows::tinyflows::ops::score_flow;

fn discover_best_match(context: &SearchContext) -> Option<WorkflowDefinition> {
    let store = FlowStore::open(context.workspace)?;
    let all = store.list_all()?;
    all.into_iter()
        .max_by_key(|flow| score_flow(flow, context))
}

Summary

• WorkflowBuilder and FlowDiscovery serve distinct roles: the former generates novel automation graphs while the latter retrieves relevant existing workflows.
• Both agents operate through the unified Agent Harness in src/openhuman/agent/harness.rs, executing tool calls to gather context before proposing graphs.
• The TinyFlows Engine validates all suggestions against safety caps, ensuring suggested workflows lack disallowed side effects.
• Frontend applications invoke these capabilities through JSON-RPC methods openhuman.flows_suggest and openhuman.flows_discover.
• All workflow definitions follow the WorkflowDefinition schema from the tinyflows crate, enabling consistent serialization and execution across the platform.

Frequently Asked Questions

What distinguishes WorkflowBuilder from FlowDiscovery agents?

WorkflowBuilder generates entirely new automation graphs based on user prompts and available tools, constructing nodes and edges programmatically in src/openhuman/flows/agents/workflowbuilder.rs. FlowDiscovery searches existing workflow repositories to find reusable patterns, ranking them by contextual relevance using the scoring logic in src/openhuman/flows/ops.rs.

How do the agents ensure proposed workflows are safe to execute?

Before returning suggestions, the agents run flow-policy checks via the TinyFlows safety modules (tinyflows::caps::). These validations scan for unauthorized side effects, permission mismatches, and insecure operations. Violations trigger automatic sanitization or fallback to conservative templates, ensuring users receive only secure automation graphs.

Can developers invoke these copilot agents outside the OpenHuman UI?

Yes, developers can call the agents programmatically through the JSON-RPC interface using methods like openhuman.flows_suggest, or directly instantiate the Agent Harness in Rust for testing and server-side automation. The harness supports mock providers and full access controls, enabling safe integration into custom applications.

What data sources inform the agents' workflow suggestions?

Both agents query multiple context sources: the memory domain for recent user actions, the skill catalog for available tools, and the TinyFlows store for existing workflow definitions. WorkflowBuilder emphasizes tool compatibility and user intent, while FlowDiscovery prioritizes historical usage patterns and semantic similarity to current tasks.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}