Understanding the Memory Seam Between tinymemory and tinycortex in OpenHuman

The memory seam is a deliberate architectural boundary that separates the public contract types (tinymemory-api) used for cross-process communication from the internal engine implementation (tinymemory-core), enforced through explicit re-exports in src/openhuman/memory/api.rs and compile-time verification tests.

OpenHuman employs a dual-crate memory architecture to balance modular extensibility with internal performance. This design creates a strict memory seam that determines which types can traverse the TinyBus wire to external modules versus which remain confined to the core process.

The Two Sides of the Memory Architecture

The OpenHuman core interacts with long-term memory through two distinct crates that serve complementary but strictly separated roles.

tinymemory-api defines the contract—the minimal set of types and messages that travel over TinyBus between the core and any external memory module. This crate represents the public surface that both the core and dynamically loaded modules must understand.

tinymemory-core provides the in-process engine that the core calls directly when operating without an external module. It implements the actual storage, indexing, and retrieval logic with richer internal APIs that never cross the process boundary.

How the Seam is Implemented

The core repository stitches these two worlds together through explicit architectural constraints in specific source files.

Contract Re-export Layer

The file src/openhuman/memory/api.rs re-exports only the items belonging to the bus contract. This creates the canonical public interface that both the core and any loaded module consume:

// src/openhuman/memory/api.rs
pub use tinymemory_api::{Chunk, MemoryQuery, StoreResult};

By restricting public exports to tinymemory-api types only, the core ensures that any type visible to external modules comes from the version-locked contract.

Engine-Only Imports

Rather than re-exporting the entire tinymemory-core crate (which would blur the contract boundary), the core imports engine-specific types in isolated locations explicitly marked as "engine only". Key integration points include:

• src/openhuman/memory/modules/memory.rs – Contains the ModuleMemoryProvider implementation that serializes contract types for bus transmission
• src/openhuman/memory/modules/memory_host.rs – Host-side helpers for module management that remain strictly within the core process

These files use tinymemory_core types directly for fast, internal operations:

// src/openhuman/memory/modules/memory.rs (internal usage)
use tinymemory_core::engine::MemoryEngine;

let engine = MemoryEngine::new(config);
engine.store_chunk(chunk);

Compile-Time Verification

The seam's integrity is guarded by src/openhuman/memory/direct_engine_refs_tests.rs, a suite of compile-time tests that guarantees the two sides stay synchronized. The test suite verifies that every type used by the core either originates from the contract (tinymemory_api) or is explicitly categorized as engine-only.

If the contract changes without updating dependent core code, these tests fail immediately, surfacing type mismatches before runtime. This mechanism prevents accidental leakage of internal engine types into the public API surface.

Crossing the Boundary: Module vs In-Process

When a memory module is loaded, the core communicates only via contract types. All calls crossing the process boundary—such as ModuleMemoryProvider::call and ModuleMemoryProvider::push—serialize and deserialize the contract structs (tinymemory_api::Chunk, etc.) over TinyBus.

Calls that remain inside the process can utilize the richer tinymemory-core API directly, avoiding the overhead of bus marshalling. This distinction allows the core to fall back to the in-process engine when no module is present, while supporting external modules for large-scale storage or custom back-ends without architectural changes.

// Cross-process (contract)
use tinymemory_api::Chunk;
use openhuman::memory::modules::memory::ModuleMemoryProvider;

let provider = ModuleMemoryProvider::new(handle);
provider.push_chunk(Chunk { id, data, metadata })?;

// In-process (engine)
use tinymemory_core::engine::MemoryEngine;
let local_engine = MemoryEngine::default();
local_engine.batch_store(chunks)?;

Why the Memory Seam Matters

This architectural split provides three critical guarantees for the OpenHuman system:

Safety – The contract represents the only data exchangeable with third-party modules. Keeping a clean seam prevents accidental leakage of internal types that external modules cannot understand or deserialize.

Stability – Changes to the contract (e.g., adding a field to Chunk) force compilation errors in any code using the old definition. This ensures that the core, loaded modules, and future releases maintain binary compatibility across the TinyBus interface.

Flexibility – The core can seamlessly fallback to tinymemory-core for local development or small deployments, while production environments can load specialized memory modules for distributed storage, all without changing the calling code.

Key Implementation Files

File	Purpose
src/openhuman/memory/api.rs	Re-exports the contract (tinymemory_api) as the public interface
src/openhuman/memory/modules/memory.rs	Core implementation communicating with loaded modules via the bus
src/openhuman/memory/modules/memory_host.rs	Host-side module loading helpers using contract types
src/openhuman/memory/direct_engine_refs_tests.rs	Compile-time verification enforcing the contract/engine split
vendor/tinymemory/crates/tinymemory-api/	Upstream contract crate (version-locked)
vendor/tinymemory/crates/tinymemory-core/	Full in-process engine used when no module is loaded

Summary

• The memory seam strictly separates tinymemory-api (public contract) from tinymemory-core (internal engine).
• src/openhuman/memory/api.rs re-exports only contract types, creating the boundary external modules see.
• Engine-only code in memory.rs and memory_host.rs imports tinymemory_core directly for in-process operations.
• direct_engine_refs_tests.rs provides compile-time guarantees that internal types never leak across the boundary.
• Cross-process calls use ModuleMemoryProvider methods with serialized contract types, while local operations use the native MemoryEngine API.

Frequently Asked Questions

What happens if I accidentally use tinymemory_core types in the public API?

The project will fail to compile. The test suite in src/openhuman/memory/direct_engine_refs_tests.rs specifically checks that only tinymemory_api types appear in public interfaces. Attempting to expose tinymemory_core::Chunk or other engine types through src/openhuman/memory/api.rs triggers a compilation error, preventing runtime type mismatches between the core and external modules.

Can external memory modules access tinymemory_core functionality?

No. External modules communicate exclusively through the tinymemory-api contract over TinyBus. They receive serialized contract structs like Chunk and MemoryQuery, never the raw engine types. This restriction ensures that modules written in other languages or compiled against different versions can safely interoperate with the core without needing the full Rust engine implementation.

How does the core decide between using the engine directly versus calling a loaded module?

The core checks for the presence of a loaded module at initialization. If ModuleMemoryProvider detects an active memory module, it routes all storage operations through the TinyBus using contract serialization. If no module is present, the system instantiates MemoryEngine from tinymemory-core directly, avoiding serialization overhead while maintaining identical logical behavior through the shared contract types.

Where is the memory seam documented in the source code?

The seam is physically manifested in src/openhuman/memory/api.rs, which serves as the single point of re-export for contract types. The boundary is enforced programmatically in src/openhuman/memory/direct_engine_refs_tests.rs, and utilized in src/openhuman/memory/modules/memory.rs where the core decides whether to serialize for the bus or call the local engine.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}