Core Components of the OpenHuman Architecture

OpenHuman is built as a React and Tauri v2 desktop application embedding a Rust core, utilizing JSON-RPC communication between layers and featuring a Memory Tree for local data storage, an Agent Harness for LLM turn management, and native tool integrations.

OpenHuman is an open-source AI desktop application developed by tinyhumansai that keeps all user data local while providing powerful agentic capabilities. Understanding the core components of the OpenHuman architecture is essential for developers who want to extend the system or integrate custom providers. The codebase is organized into distinct layers that separate presentation, platform integration, and business logic.

Architecture Overview

The system follows a three-layer architecture where a thin frontend delegates all business logic to an embedded Rust core via a well-defined JSON-RPC interface:

┌──────────────────────────────────────────────────┐
│ Tauri Shell (app/src-tauri/)                     │
│ • Windowing, OS integration, sidecar lifecycle    │
│ • CEF child webviews for external providers       │
└──────────────────────────────────────────────────┘
          │ JSON-RPC (HTTP) ↕
┌──────────────────────────────────────────────────┐
│ Rust Core (src/)                                 │
│ • Memory Tree pipeline                           │
│ • Integration adapters + auto-fetch scheduler     │
│ • Provider router (model routing)                 │
│ • TokenJuice compression                          │
│ • Native tools (search, fetch, fs, git, ...)      │
│ • Voice processing (STT in, TTS out)            │
└──────────────────────────────────────────────────┘
          │
┌──────────────────────────────────────────────────┐
│ React Frontend (app/src/)                        │
│ • Screens and navigation                          │
│ • coreRpcClient for core communication            │
│ • Presentation-only layer (no business logic)     │
└──────────────────────────────────────────────────┘

The Tauri Shell Layer

The Tauri v2 shell (app/src-tauri/) serves as the cross-platform desktop host that manages the application lifecycle and bridges the gap between the web frontend and the operating system.

Key responsibilities include:

• Platform-specific window handling including deep-link registration and overlay windows
• Core process management via commands like start_core_process and restart functionality
• Authentication token provision through core_rpc_token which returns the bearer token for the embedded core
• Global shortcuts including conflict-aware dictation and push-to-talk hotkeys via register_dictation_hotkey
• Overlay management using mascot_window_show for floating UI elements (macOS NSPanel)
• Auto-update flow through check_app_update, download, and apply functions

The Rust Core Engine

The Rust core (src/) contains the authoritative business logic and runs as a sidecar process to the Tauri shell.

The entry point at [src/main.rs](https://github.com/tinyhumansai/openhuman/blob/main/src/main.rs#L11-L18) initializes Sentry, loads environment variables, and forwards CLI arguments to the core library. The core is structured into several specialized modules:

Memory Tree

The Memory Tree is a local-first data pipeline that stores all user data in a SQLite database and Obsidian Markdown vault. According to the architecture documentation, the pipeline follows six stages:

1. Auto-fetch pulls raw provider data every 20 minutes
2. Canonicalise normalizes data to provenance-tagged Markdown
3. Chunk splits Markdown into ≤3k-token pieces
4. Store persists chunks in <workspace>/memory_tree/chunks.db and as .md files in <workspace>/wiki/
5. Score runs embeddings and hotness scoring
6. Summarise builds source/topic/global summary trees

Agent Harness

The Agent Harness (src/openhuman/agent/) implements the turn engine that processes every user interaction. As detailed in the agent harness documentation, the system handles:

• Trigger triage classifying inbound events (user, webhook, cron) to decide whether to drop, notify, or spawn an orchestrator turn
• Turn engine via Agent::turn which resumes transcripts, builds system prompts, injects memory, and enters the tool-call loop (max 10 iterations by default) as described here
• Sub-agents specialist agents (planner, researcher, code executor) that the orchestrator spawns with narrow system prompts and filtered tool sets as documented
• Hooks including mid-turn stop-hooks for budget caps and post-turn hooks for archivist, learning, and cost logging operations see reference

Supporting Systems

• Integration adapters native providers (Gmail, Slack, etc.) with periodic auto-fetch scheduling
• Provider router selects appropriate LLM provider/model hints for each task
• TokenJuice compresses large tool outputs before they enter the LLM context
• Native tools implementations for search, fetch, file system, and git operations

The React Frontend

The React frontend (app/src/) is a presentation-only layer that contains screens, navigation, and UI components. It contains no business logic, delegating all operations to the Rust core via the coreRpcClient.

The frontend communicates through [app/src/services/rpcMethods.ts](https://github.com/tinyhumansai/openhuman/blob/main/app/src/services/rpcMethods.ts), which provides a thin wrapper around Tauri's invoke function:

// Example from rpcMethods.ts
export async function coreRpcRelay(method: string, params: any) {
  return await window.__TAURI__.invoke('core_rpc_relay', { method, params });
}

Integration and Communication Patterns

JSON-RPC Interface

The frontend and core communicate via JSON-RPC over HTTP. The Tauri shell acts as a proxy, forwarding requests from the React app to the Rust core process.

Calling Core Methods from the Frontend

To execute operations in the core from the UI, use the core_rpc_relay invoke:

import { coreRpcRelay } '@/services/rpcMethods';

// Ask the core to run a turn with the current user message
async function askAssistant(message: string) {
  const response = await coreRpcRelay('agent.turn', { userMessage: message });
  return response;
}

Starting the Core Process

The frontend initiates the core sidecar using Tauri commands:

import { invoke } from '@tauri-apps/api';

async function bootCore() {
  await invoke('start_core_process');
  // Core is now ready to accept JSON-RPC calls
}

Registering Global Shortcuts

Configure system-wide hotkeys through the Tauri layer:

import { invoke } from '@tauri-apps/api';

async function setupDictationHotkey(shortcut: string) {
  await invoke('register_dictation_hotkey', { shortcut });
}

Running Sub-Agents in Rust

Within the core, orchestrators can spawn specialized sub-agents:

use openhuman::agent::agents::loader::AgentDefinition;

// Load the "researcher" archetype and execute as sub-agent
let def = AgentDefinition::load("researcher")?;
let result = def.run_subagent("Find recent papers on LLM safety")?;

Key Source Files

File	Description
[gitbooks/developing/architecture/README.md](https://github.com/tinyhumansai/openhuman/blob/main/gitbooks/developing/architecture/README.md)	High-level system diagram and data-flow overview
[app/src-tauri/src/lib.rs](https://github.com/tinyhumansai/openhuman/blob/main/app/src-tauri/src/lib.rs)	Tauri command implementations including core RPC token, process management, hotkeys, and update flow
[src/main.rs](https://github.com/tinyhumansai/openhuman/blob/main/src/main.rs)	Binary entry point for Sentry initialization and CLI dispatch
[src/openhuman/agent/agent_harness.md](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/agent/agent_harness.md)	Agent Harness documentation covering turn engine, sub-agents, and hooks
[app/src/services/rpcMethods.ts](https://github.com/tinyhumansai/openhuman/blob/main/app/src/services/rpcMethods.ts)	Frontend wrapper for core RPC communication
src/openhuman/agent/harness/	Turn engine implementation, tool-loop logic, and sub-agent runner
src/openhuman/memory/	Memory Tree pipeline implementation (canonicalization, chunking, storage)
src/openhuman/agent/tools/	Native tool implementations for search, fetch, and file operations
src/openhuman/provider/router/	Model routing and provider selection logic

Summary

• OpenHuman uses a three-tier architecture: React frontend, Tauri v2 shell, and Rust core.
• Communication occurs via JSON-RPC over HTTP, with the Tauri shell proxying requests and managing the core sidecar lifecycle.
• Agent Harness (src/openhuman/agent/) implements the turn engine with trigger triage, tool-call loops (max 10 iterations), sub-agent spawning, and hooks for budget and logging.
• Memory Tree stores all data locally using SQLite and Markdown vaults, processing data through canonicalization, chunking, embedding, and summarization stages.
• TokenJuice compresses tool outputs before they enter the LLM context to optimize token usage.
• Native tools provide filesystem, search, fetch, and git capabilities available to the agent harness.
• Key integration points include core_rpc_relay for frontend-to-core calls, start_core_process for lifecycle management, and AgentDefinition::load for sub-agent orchestration.

Frequently Asked Questions

How does the frontend communicate with the Rust core?

The React frontend communicates with the Rust core via JSON-RPC over HTTP proxied through Tauri's invoke mechanism. The frontend calls core_rpc_relay with a method name and parameters, which the Tauri shell forwards to the embedded core process. The [app/src/services/rpcMethods.ts](https://github.com/tinyhumansai/openhuman/blob/main/app/src/services/rpcMethods.ts) file implements this thin wrapper around the Tauri invoke API.

What is the Agent Harness responsible for?

The Agent Harness is the turn engine that manages every interaction cycle (turn). It classifies incoming triggers, resumes conversation transcripts, builds system prompts with injected memory, and runs the tool-call loop where the LLM selects tools and observes results for up to 10 iterations. It also manages sub-agent spawning for specialized tasks and executes hooks for budget enforcement and post-turn archival.

Where is user data stored in OpenHuman?

All user data is stored locally in the Memory Tree, which uses a SQLite database at <workspace>/memory_tree/chunks.db and Markdown files in <workspace>/wiki/. The system automatically fetches data from integrated providers every 20 minutes, canonicalizes it to Markdown, chunks it into ≤3k-token pieces, and stores it with embeddings and hotness scores.

What is TokenJuice and why is it used?

TokenJuice is a compression system within the Rust core that reduces the token size of large tool outputs before they enter the LLM context window. This optimization allows the agent to work with substantial data from file reads, search results, or git diffs without exceeding model context limits, ensuring efficient token usage during the tool-call loop.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}