How Magnitude's ICN Differentiates MemoryLocation from MemoryBreakdown

Magnitude's ICN (Inference Compute Network) uses MemoryLocation to identify where memory resides and MemoryBreakdown to describe what memory is being used, with resolution occurring before accounting to enable precise memory tracking across heterogeneous compute.

In the magnitudedev/magnitude repository, the ICN memory system is architected around two complementary abstractions that work in sequence. This design, documented in info/icn/memory-abstractions.md, enables fine-grained memory accounting across distributed GPUs, CPUs, and hosts.

MemoryLocation: The "Where" of Memory

A MemoryLocation is a logical identifier that describes the physical or virtual placement of memory. It answers the question: where does this memory live?

The ICN resolves MemoryLocation through the MemoryTopology layer. When the system needs to allocate or query memory, it first invokes MemoryTopology.resolve(location) to map the logical location to concrete hardware. This resolution step produces a physical device descriptor that downstream components consume.

Key characteristics of MemoryLocation:

• Contains hostId and device information (GPU/CPU index)
• Resolved via MemoryTopology.resolve() before any accounting occurs
• Serves as the anchor for all subsequent memory operations

MemoryBreakdown: The "What" of Memory

A MemoryBreakdown provides a hierarchical decomposition of memory consumption at a resolved location. It answers the question: what constitutes this memory usage?

The MemoryAccountant produces a MemoryBreakdown after location resolution completes. This breakdown categorizes memory into components such as model parameters, activation buffers, and temporary buffers. The breakdown then feeds into MemoryAccounting, which aggregates category totals for budgeting and runtime limit enforcement.

Key characteristics of MemoryBreakdown:

• Captures categorical composition of memory usage
• Only exists in relation to a resolved MemoryLocation
• Provides single source of truth for per-category counters

The Resolution Order: Location Before Breakdown

The ICN enforces strict sequencing between these abstractions:

1. Resolve location — MemoryTopology.resolve(loc) maps logical to physical
2. Compute breakdown — MemoryAccountant generates categorical breakdown
3. Record charge — MemoryAccountant.recordCharge(charge) persists both

This ordering ensures that breakdowns are always grounded in actual hardware locations. No breakdown exists independently of its resolved location.

import * as S from '@magnitudedev/icn-protocol/schemas';

// 1. Create logical MemoryLocation
const loc: S.MemoryLocation = {
  hostId: 'alpha',
  device: { kind: 'gpu', index: 0 },
};

// 2. Resolve to concrete topology
const topology = await MemoryTopology.resolve(loc);

// 3. Construct charge with location and breakdown
const charge: S.MemoryCharge = {
  location: topology.location,
  breakdown: {
    parameters: 120_000_000,
    activations: 80_000_000,
    buffers: 30_000_000,
  },
};

// 4. Record to accountant
await MemoryAccountant.recordCharge(charge);

Source File References

The memory abstractions are defined across several files in the repository:

• info/icn/memory-abstractions.md — Design document defining MemoryLocation and MemoryBreakdown semantics
• packages/icn-protocol/schemas.ts — Generated TypeScript schemas for runtime use
• packages/sdk/src/inference-client.ts — Client implementation creating memory-related RPCs

Summary

• MemoryLocation identifies where memory lives; MemoryBreakdown describes what memory contains
• Resolution order is strict: location resolution precedes breakdown generation
• MemoryAccountant maintains category totals exclusively from breakdown data, ensuring single source of truth
• All operations in info/icn/memory-abstractions.md follow this two-phase pattern for distributed memory safety

Frequently Asked Questions

Can a MemoryBreakdown exist without a MemoryLocation?

No. According to the ICN design in info/icn/memory-abstractions.md, a MemoryBreakdown is always produced in relation to a resolved MemoryLocation. The breakdown describes memory composition at a specific location, so the two concepts are inherently coupled in the accounting flow.

How does MemoryTopology handle resolution failures?

When MemoryTopology.resolve() cannot map a logical MemoryLocation to available hardware, the resolution fails before any MemoryBreakdown is computed. This prevents phantom accounting entries and ensures that all recorded memory charges correspond to verified physical resources.

What categories appear in a typical MemoryBreakdown?

The schema in packages/icn-protocol/schemas.ts defines standard categories including parameters (model weights), activations (layer outputs), and buffers (temporary storage). The exact set is extensible, but all categories roll up through MemoryAccounting to enforce runtime budgets and limits.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}