# TinyMemory Core Crate Architecture: Modules and Components Explained

> Explore the modular TinyMemory Core crate architecture in OpenHuman. Understand its key modules: singleton initialization, chunk storage, job queues, and audit logging for efficient memory management.

- Repository: [Tiny Humans/openhuman](https://github.com/tinyhumansai/openhuman)
- Tags: architecture
- Published: 2026-08-31

---

**The TinyMemory Core crate provides a modular Rust memory subsystem for OpenHuman, exposing singleton initialization, hierarchical chunk storage, background job queues, and comprehensive audit logging through a set of well-defined modules.**

The TinyMemory Core crate (`tinymemory_core`) serves as the central library powering OpenHuman’s persistent memory layer. It organizes functionality into discrete modules with clear separation of concerns, from low-level chunk storage in `src/openhuman/memory/store/chunks/` to high-level content summarization and external synchronization. Below is a complete architectural breakdown of the crate’s components, including actual source paths and implementation details from the `tinyhumansai/openhuman` repository.

## Global Initialization and Client Management

The `global` module in [`src/openhuman/memory/global.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/global.rs) provides the entry point for the entire memory system. It exposes a singleton pattern through the `global::init` function, which bootstraps the `MemoryClient` with a workspace directory, and `global::client` to retrieve the handle for subsequent operations.

```rust
use tinymemory_core::global;

// Initialise the global client with a workspace directory.
global::init("/path/to/workspace").expect("failed to initialise memory");

// Retrieve a handle to the client for subsequent calls.
let client = global::client();

```

This pattern ensures that all modules share a single, initialized client instance throughout the process lifecycle.

## The Unified Storage Interface

At the heart of the crate lies the `store` module ([`src/openhuman/memory/store/mod.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/store/mod.rs)), which exposes a high-level façade through types like `UnifiedMemory`, `NamespaceStore`, and the `MemoryClient`. This layer abstracts the underlying storage backends, providing a consistent API for writing and retrieving memory objects regardless of their specific format.

### Chunk Management (store::chunks)

The `store::chunks` submodule handles fine-grained text fragments and their metadata. Defined in [`src/openhuman/memory/store/chunks/types.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/store/chunks/types.rs), the core `Chunk` struct carries text content, optional embeddings, and metadata including `SourceKind` (e.g., Chat, Document) and `SourceRef`. The companion storage logic in [`src/openhuman/memory/store/chunks/store.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/store/chunks/store.rs) provides `upsert_chunks` for atomic insert-or-update operations and `with_connection` for transaction-scoped access.

```rust
use tinymemory_core::store::chunks::store::upsert_chunks;
use tinymemory_core::store::chunks::types::{Chunk, Metadata, SourceKind, SourceRef};

let chunk = Chunk {
    id: "c1".into(),
    text: "Hello, world!".into(),
    embedding: None,
    metadata: Metadata {
        source_ref: SourceRef::new("source-123".into()),
        source_kind: SourceKind::Chat,
        ..Default::default()
    },
    ..Default::default()
};

upsert_chunks(&client, vec![chunk]).expect("failed to upsert chunk");

```

### Hierarchical Tree Structures (store::trees)

While chunks store raw data, the `store::trees` module organizes them into hierarchical structures. Located in [`src/openhuman/memory/store/trees/types.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/store/trees/types.rs), it defines the `Tree` struct with variants like `TreeKind::Topic` and `TreeKind::Session`, along with `SummaryNode` for aggregated representations. Persistence functions in [`src/openhuman/memory/store/trees/store.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/store/trees/store.rs) expose `upsert_tree` for creating these organizational structures.

```rust
use tinymemory_core::store::trees::store::upsert_tree;
use tinymemory_core::store::trees::types::{Tree, TreeKind, SummaryNode};

let tree = Tree {
    id: "t1".into(),
    kind: TreeKind::Topic,
    summary: SummaryNode::default(),
    ..Default::default()
};

upsert_tree(&client, vec![tree]).expect("failed to create tree");

```

### Content Operations (store::content)

Higher-level content processing resides in [`src/openhuman/memory/store/content/mod.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/store/content/mod.rs). This module handles summarization workflows through types like `SummaryComposeInput` and `SummaryTreeKind`, and manages raw content variants via `raw::RawKind`. It bridges the gap between raw chunks and semantic understanding.

### Identity Mapping (store::identity)

The `store::identity` module ([`src/openhuman/memory/store/identity/mod.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/store/identity/mod.rs)) manages user and tool identity within the memory graph. It provides the `IdentityKind` enum and helper functions like `is_self_identity_any_toolkit` to distinguish between human users and autonomous tools when attributing memory entries.

## Background Job Processing

The `queue` module ([`src/openhuman/memory/queue/mod.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/queue/mod.rs)) implements asynchronous background processing for computationally expensive tasks. It defines payloads such as `ReembedBackfillPayload` for recalculating embeddings and exposes `enqueue_backfill` to schedule work without blocking the main execution flow.

```rust
use tinymemory_core::queue::types::ReembedBackfillPayload;
use tinymemory_core::queue::enqueue_backfill;

let payload = ReembedBackfillPayload {
    chunk_id: "c1".into(),
    model: "openai_text-embedding-ada-002".into(),
};

enqueue_backfill(&client, payload).expect("failed to enqueue backfill");

```

## Audit Logging with TinyCortex

All mutations within the TinyMemory Core crate are recorded by the `tinycortex` subsystem ([`src/openhuman/memory/tinycortex/mod.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/tinycortex/mod.rs)). This audit-log module provides `read_audit_log` for retrieving historical operations and `run_github_sync` for external synchronization, enabling reproducible debugging and event replay.

```rust
use tinymemory_core::tinycortex::read_audit_log;

let entries = read_audit_log(&client, 100).expect("audit read failed");
for entry in entries {
    println!("{:?}", entry);
}

```

## External Service Synchronization

The `sync` module ([`src/openhuman/memory/sync/mod.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/sync/mod.rs)) bridges the core memory system to external providers. Current implementations include Composio integration, accessed through paths like `sync::composio::providers::user_scopes`, allowing automated ingestion of external tool interactions into the memory graph.

## Source Registry and Tree Policies

Three specialized modules handle metadata management and lifecycle policies:

- **sources** ([`src/openhuman/memory/sources/registry.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/sources/registry.rs)): Maintains a registry of `SourceKind` descriptors and provides `replace_sources_in` for batch source updates.
- **tree_source** ([`src/openhuman/memory/tree_source.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/tree_source.rs)): Offers `get_or_create_source_tree` to locate or instantiate source-trees that group related chunks under common origins.
- **tree_policy** ([`src/openhuman/memory/tree_policy.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/tree_policy.rs)): Implements `TreePolicy` structs that drive pruning, retention budgeting, and hierarchical cleanup strategies.

## How the Components Integrate

When an application interacts with the TinyMemory Core crate, the data flows through a predictable pipeline:

1. **Bootstrapping**: `global::init` establishes the singleton `MemoryClient`.
2. **Data Ingestion**: Callers use `upsert_chunks` or `upsert_tree` to write low-level objects, associating `IdentityKind` and `SourceKind` metadata.
3. **Organization**: The **tree_source** and **tree_policy** modules organize chunks into trees and enforce retention rules.
4. **Async Processing**: Expensive operations route through the **queue** module for background execution.
5. **Auditing**: The **tinycortex** module records every mutation for later inspection.
6. **External Sync**: The **sync** module pulls external data (e.g., from Composio) and injects it through the standard store APIs.

## Summary

- The **TinyMemory Core crate** organizes OpenHuman's memory layer into twelve distinct modules with clear separation of concerns.
- **Global initialization** via `global::init` creates a singleton `MemoryClient` shared across all operations.
- **Storage** splits between atomic **chunks** (text fragments) and hierarchical **trees** (organizational structures), managed by the unified `store` façade.
- **Identity** metadata distinguishes between users and tools, while the **sources** registry tracks data provenance.
- **Background jobs** and **audit logging** run asynchronously through the `queue` and `tinycortex` modules.
- **External synchronization** integrates third-party services like Composio into the memory graph.

## Frequently Asked Questions

### What is the difference between chunks and trees in the TinyMemory Core crate?

**Chunks** represent atomic text fragments with embeddings and metadata, handled by the `store::chunks` module. **Trees** provide hierarchical organization for these chunks, managed by `store::trees`, allowing grouping by topic, session, or custom taxonomies through the `Tree` and `TreeKind` types.

### How do I initialize the memory client in a Rust application?

Call `global::init("/path/to/workspace")` once at startup, then retrieve the handle with `global::client()`. This singleton pattern, defined in [`src/openhuman/memory/global.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/global.rs), ensures all modules share a single initialized `MemoryClient` instance.

### What is the purpose of the TinyCortex module?

The **TinyCortex** module ([`src/openhuman/memory/tinycortex/mod.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/tinycortex/mod.rs)) serves as the immutable audit log for the memory system. It records all mutations via `read_audit_log` and supports event replay through functions like `run_github_sync`, enabling debugging and historical analysis.

### How does the queue module handle background tasks?

The **queue** module ([`src/openhuman/memory/queue/mod.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/memory/queue/mod.rs)) offloads expensive operations—such as recomputing vector embeddings via `ReembedBackfillPayload`—to background workers using `enqueue_backfill`. This prevents blocking the main thread during heavy computational tasks.