How the Model Slot Controller Manages Local Inference Hardware in Magnitude

The Model Slot Controller orchestrates local inference hardware by reconciling persisted slot selections with live local model offerings, computing real-time availability and residency states, and exposing UI actions for direct hardware management.

The Model Slot Controller serves as the central arbiter for inference slot allocation in the Magnitude agent framework. It determines whether primary and secondary slots utilize remote APIs or local hardware, continuously synchronizing persisted user preferences with the dynamic state of installed models. Understanding how this controller manages local inference hardware reveals the architecture behind Magnitude's hybrid cloud-edge inference capabilities.

Three-Stage Slot Reconciliation Workflow

The controller operates through a defined lifecycle in packages/acn/src/model-slot-controller.ts, progressing through distinct phases to maintain accurate hardware state.

Stage 1: Loading Current Selections and Catalogs

The process begins in ModelSlotControllerLive by gathering prerequisite data from multiple sources. According to the source code at lines 21-30, the controller retrieves persisted slot selections via modelSelection.get, reads the provider-model catalog from catalog.state, and fetches locally installed configurations through localOfferings.list. This aggregation ensures the controller has a complete view of both declared intent and actual hardware availability before making allocation decisions.

Stage 2: Building Slot Descriptions with Local Awareness

For each inference slot (primary or secondary), the controller invokes the buildSlot function to determine the concrete ModelSlot state. The implementation at lines 63-84 handles four distinct configurations:

• Unassigned: Preserves previous state when no selection exists.
• Resolving: Waits for catalog data or local offering resolution.
• ConfiguredRemote: Activates when selection.providerId !== LOCAL_PROVIDER_ID.
• ConfiguredLocal: Triggers when selection.providerId === LOCAL_PROVIDER_ID (imported from @magnitudedev/icn/provider), initiating local-specific availability and residency checks.

Stage 3: Committing and Publishing State Changes

After building slot descriptions, the commit function (lines 56-88) compares the new ModelSlotsState against the previous aggregate using ModelSlotsStateSchema for comparison. When changes are detected, the controller updates the subscription reference, publishes events via changes.publish, and recomputes the agent-wide configuration through buildConfigStateFromSlots.

Local Inference Hardware Lifecycle Management

Managing local hardware requires precise tracking of model availability and runtime status through specialized projection functions.

Detecting Local Model Availability

When processing a local selection in buildSlot (lines 86-104), the controller calls localModelSlotAvailability defined in packages/acn/src/model-slot-projection.ts (lines 8-30). This function evaluates three critical flags from the projection layer:

1. Whether the catalog identity is still pending.
2. Whether the local offering list is ready.
3. Whether the specific offering exists in the installed catalog.

The result manifests as a ModelSlotAvailability enumeration with values Pending, Unavailable, or Available, determining whether the hardware can service the requested model.

Determining Inference Residency

For available local models, the controller inspects running ICN instances via instances.get to locate the latest model instance matching the selected ID. At lines 110-115 of model-slot-controller.ts, the projectInferenceResidency function maps this instance to an InferenceResidency state:

• Loaded: The model is active in local memory and ready for inference.
• Unloaded: The model is installed but dormant, requiring activation before use.

Generating Hardware Management Actions

Based on calculated availability and residency, the controller constructs actionable UI commands through modelSlotActions (lines 119-122). These actions enable direct hardware control including Load, Unload, and Download operations, bridging the gap between software state and physical inference hardware.

State Persistence and Slot Updates

When users modify slots through updateModelSlot (lines 137-158), the controller executes a validation and persistence pipeline:

1. Validates the selection structure and provider compatibility.
2. Ensures local offering presence via requireSelectedLocalOffering.
3. Persists changes through modelSelection.updateSlot.
4. Triggers a state rebuild to refresh availability metrics and residency status.

Integration with Magnitude's Architecture

The controller operates within a layered dependency structure wired together in ModelSlotControllerLive with session-scoped lifetimes (Scope.Scope):

• LocalModels: Provides discovery, assessment, and ranking of locally discovered models.
• LocalProviderOfferings: Exposes the list of installed configurations and resolves specific offerings via localOfferings.resolve.
• ProviderModelCatalog: Supplies capabilities and availability data for both remote and local providers.
• IcnInstances: Maintains live ICN model instances that enable residency computation.
• AcnChanges: Publishes change events for reactive UI updates.

Practical Implementation Examples

The following examples demonstrate interacting with the Model Slot Controller using the Effect library:

Reading Current Slot State

import { Effect, Layer } from "effect"
import { ModelSlotController } from "@magnitudedev/acn"

const getSlotState = Effect.gen(function* () {
  const controller = yield* ModelSlotController
  const state = yield* controller.state // ModelSlotsState
  console.log(state.slots.primary)   // -> ModelSlot (e.g., ConfiguredLocal)
})

// Run in a layer that provides the required dependencies
Layer.scoped(
  ModelSlotController,
  // ... other deps like ModelSelection, LocalProviderOfferings, etc.
).provide(getSlotState)

Switching to Local Inference Hardware

import { Effect, Option } from "effect"
import { ModelSlotController } from "@magnitudedev/acn"

const switchToLocal = Effect.gen(function* () {
  const controller = yield* ModelSlotController
  // Local model ID from the installed offerings
  const localModelId = "llama-2-7b"
  const selection = {
    providerId: "local",               // LOCAL_PROVIDER_ID
    providerModelId: localModelId,
    reasoningEffort: "none",
  }
  // Update secondary slot to use local hardware
  yield* controller.updateModelSlot(
    "secondary",
    Option.some(selection),
  )
})

Managing Model Favorites

import { Effect } from "effect"
import { ModelSlotController } from "@magnitudedev/acn"

const favoriteRemote = Effect.gen(function* () {
  const controller = yield* ModelSlotController
  const model = {
    providerId: "openai",
    providerModelId: "gpt-4o",
  }
  yield* controller.setModelFavorite(model, true)
})

Summary

• The Model Slot Controller manages local inference hardware through a three-stage reconciliation process: loading dependencies, building slot descriptions, and committing state changes.
• Local models are identified by comparing providerId against LOCAL_PROVIDER_ID, triggering specialized availability checks via localModelSlotAvailability.
• Inference residency tracks whether models are Loaded (active in memory) or Unloaded (installed but inactive) by inspecting ICN instances.
• Hardware management actions (Load, Unload, Download) are generated based on real-time availability and residency states.
• State changes persist through updateModelSlot, which validates offerings and triggers immediate state rebuilds.

Frequently Asked Questions

How does the controller distinguish between local and remote models?

The controller checks if selection.providerId === LOCAL_PROVIDER_ID (imported from @magnitudedev/icn/provider) within the buildSlot function. When this condition is true, it treats the model as local and queries localOfferings.list for the specific offering rather than consulting remote catalog endpoints.

What determines if a local model is available for inference?

Availability is determined by localModelSlotAvailability in packages/acn/src/model-slot-projection.ts, which verifies three conditions: the catalog identity is resolved, the local offering list has loaded, and the specific model offering exists in the installation directory. The result is one of three states: Pending, Unavailable, or Available.

What is the difference between a model being "available" and "loaded"?

Availability indicates whether the model files are installed and ready to be served (ModelSlotAvailability), while residency indicates the runtime state of the model instance (InferenceResidency). A model can be Available but Unloaded (installed but not in memory) or Available and Loaded (actively running in the ICN instance manager).

How are hardware actions like Load and Unload triggered?

The controller generates UI action lists via modelSlotActions(availability, residency) based on the current slot state. When a user selects a local model that is Available but Unloaded, the controller includes a Load action; conversely, an Available and Loaded model receives an Unload action to free hardware resources.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}