How Magnitude's Model Catalog System Recommends Models for Hardware

Magnitude's model catalog system profiles your hardware to calculate safe memory headroom, filters available models against those constraints, and ranks the survivors by your preferred optimization strategy (speed, accuracy, or balanced).

The open-source Magnitude framework (magnitudedev/magnitude) simplifies local AI deployment by automatically matching models to your machine's capabilities. Its model catalog system eliminates manual compatibility checks by executing a type-safe, effect-driven pipeline that ensures recommended models fit within your CPU, GPU, and RAM limits.

Hardware Profiling and Resource Assessment

The recommendation process begins with a comprehensive hardware assessment that establishes strict resource boundaries before any model evaluation occurs.

The ICN Hardware Scanner

The ICN hardware scanner inspects your host's CPU, GPU, and total RAM to compute two critical values: a recommended_working_set and an application_head_room value. According to the implementation in packages/icn-protocol/src/schemas/model-state.ts (lines 613–617), the headroom calculation follows the formula H = max(20% of T, 4 GiB), where T represents total system memory. This ensures the inference engine reserves sufficient resources for the operating system and concurrent applications.

// Conceptual implementation based on model-state.ts
const totalMemory = getSystemRAM(); // T in bytes
const applicationHeadRoom = Math.max(totalMemory * 0.20, 4 * 1024 ** 3); // 4 GiB minimum
const recommendedWorkingSet = totalMemory - applicationHeadRoom;

The HardwareRecommendation Contract

The scanner exposes its findings through the HardwareRecommendation enum defined in the ICN protocol contracts. This interface communicates calculated constraints—such as maximum allocable memory and GPU availability—to downstream components, ensuring that hardware limits are respected throughout the recommendation pipeline.

Catalog Retrieval and Model Enrichment

Once hardware limits are established, the system queries provider-specific catalogs to fetch available models and augment them with metadata required for compatibility filtering.

Provider-Specific Catalog Implementations

The Magnitude provider implementation in packages/providers/src/magnitude/catalog.ts (lines 55–72) demonstrates how raw model descriptors are fetched from remote APIs and processed. Each model is classified into a family and tagged with hardware-assessment derived constraints. The createMagnitudeCatalog function handles this enrichment, attaching family IDs and compatibility metadata to every model descriptor before caching.

// From magnitude/catalog.ts - enriching model metadata
const enrichedModels = rawModels.map(model => ({
  ...model,
  familyId: classifyModelFamily(model),
  maxMemoryUsage: estimateMemoryFootprint(model),
  compatibleWith: hardwareProfile.recommendedWorkingSet
}));

Caching and Contract Operations

The ModelCatalog contract standardizes operations across providers, exposing list, get, and refresh methods. To optimize performance, the catalog implementation caches the model list with a configurable ttlMs parameter, reducing redundant API calls while ensuring the recommendation set remains current.

Filtering and Ranking Recommendations

With hardware constraints defined and model metadata enriched, the system executes the core recommendation logic through the catalog aggregator.

Hardware Constraint Filtering

The catalog-aggregator in packages/providers/src/catalog-aggregator.ts (lines 16–30) merges catalogs from all enabled providers and applies strict hardware filters. Any model exceeding the calculated memory limits or violating the application_head_room policy is immediately excluded from consideration. This filtering occurs before ranking to ensure only viable, hardware-compatible options proceed.

// Hardware filtering logic from catalog-aggregator.ts
const compatibleModels = enrichedCatalog.filter(model => 
  model.memoryRequirements <= hardwareContext.recommendedWorkingSet &&
  model.memoryRequirements <= (totalMemory - hardwareContext.applicationHeadRoom)
);

Preference-Based Ranking

The remaining models are ranked according to user-selected preferences: balanced, speed, or accuracy. The ranking algorithm evaluates each model's reasoningEffort level, vision capability flags, and other metadata to generate an ordered list. For example, selecting speed prioritizes models with lower reasoningEffort and faster inference times, while accuracy weights higher capability scores and larger parameter counts.

CLI Integration and User Presentation

The recommendation pipeline surfaces results through the Magnitude CLI, providing actionable, hardware-validated options.

When users execute magnitude catalog recommendations, the CLI triggers the hardware scanner and consumes the aggregated catalog state. As implemented in cli/src/commands/inference.ts (lines 21–24), the command displays the top 10 recommendations by default, each annotated with compatibility badges derived from the HardwareRecommendation assessment. The output includes direct links to documentation explaining the recommendation methodology.

# Retrieve top hardware-compatible models optimized for speed

magnitude catalog recommendations --preference speed --top 5

Users can then select from the filtered, ranked list to install models via the Harness onboarding flow, confident that their choice will run efficiently on their specific hardware configuration.

Summary

• Hardware-first assessment: The ICN hardware scanner calculates safe memory headroom using H = max(20% of T, 4 GiB) before evaluating any models, ensuring system stability.
• Type-safe constraint propagation: The HardwareRecommendation enum in packages/icn-protocol/src/schemas/model-state.ts communicates limits to the catalog layer.
• Provider enrichment: createMagnitudeCatalog in packages/providers/src/magnitude/catalog.ts classifies models into families and attaches hardware constraints to raw API data.
• Constraint-aware ranking: The catalog-aggregator filters over-committed models and ranks survivors by speed, accuracy, or balanced preferences using reasoningEffort and vision metadata.
• Actionable CLI output: The magnitude catalog recommendations command presents the top 10 hardware-compatible options with documentation links.

Frequently Asked Questions

How does Magnitude calculate available memory for model recommendations?

Magnitude calculates available memory by first determining total system RAM, then reserving the application_head_room (the greater of 20% of total memory or 4 GiB). The remaining recommended_working_set represents the safe capacity for inference workloads, ensuring the OS and other applications retain sufficient resources.

What criteria does the catalog aggregator use to rank models?

The catalog aggregator ranks models based on the user-selected preference: balanced, speed, or accuracy. It evaluates metadata including reasoningEffort (lower values for speed, higher for accuracy) and vision capability flags to produce an ordered list of recommendations that match the user's optimization priorities.

Where does the hardware profiling data originate in the Magnitude codebase?

Hardware profiling data originates from the ICN hardware scanner, which populates the HardwareRecommendation struct defined in packages/icn-protocol/src/schemas/model-state.ts. This schema validates calculated memory constraints before they reach the catalog aggregation layer in packages/providers/src/catalog-aggregator.ts.

Can I refresh the model catalog if new models become available?

Yes. The ModelCatalog contract exposes a refresh method that invalidates the cached catalog list (governed by ttlMs) and fetches updated model descriptors from provider APIs. You can trigger this via the CLI to ensure recommendations include the latest compatible models released by providers.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}