Complete Guide to the llmfit API Endpoints

The llmfit API exposes nine HTTP JSON endpoints under /api/v1/ that allow you to query system hardware, list and filter model fits, download weights, and generate execution plans, all served by an Axum-powered server running on port 8787 by default.

The llmfit repository by AlexsJones provides a Rust-based toolchain for analyzing how large language models fit on local hardware. The llmfit API endpoints package this analysis into a lightweight HTTP interface, making it accessible to the bundled React dashboard, the CLI, or any external HTTP client.

Core API Architecture

Axum Server Implementation

The API server is built with the Axum web framework and defined in llmfit-tui/src/serve_api.rs. When initialized, the server binds to port 8787 and logs all available endpoints (see lines 180-224 in serve_api.rs). The router mounts all handlers under the /api/v1/ prefix, providing consistent versioning and structured JSON responses for every request.

Core Library Integration

Each endpoint delegates to the llmfit-core crate for business logic. System detection flows through llmfit-core/src/hardware.rs, model metadata resides in llmfit-core/src/models.rs, and fit calculations execute in llmfit-core/src/analysis.rs. This architecture keeps the HTTP layer thin while ensuring the API returns the same hardware-model analysis used by the TUI and command-line interfaces.

Available llmfit API Endpoints

System and Hardware Inspection

GET /api/v1/system returns a JSON payload detailing CPU, RAM, GPU specifications, unified memory availability, and per-GPU bandwidth. This endpoint consumes detection routines from llmfit-core/src/hardware.rs to provide accurate hardware snapshots for downstream planning.

Model Discovery and Filtering

Three endpoints handle model discovery and retrieval:

• GET /api/v1/models lists analyzed model fits. Accepts query parameters including limit, sort, runtime, min_fit, and use_case to filter results.
• GET /api/v1/models/top returns the highest-scoring runnable models, defaulting to the top 5 unless overridden by query parameters.
• GET /api/v1/models/{name} retrieves detailed fit information for a specific model identified by its HuggingFace name, delegating calculations to llmfit-core/src/analysis.rs.

Runtime and Installation Status

• GET /api/v1/runtimes enumerates supported backends (Ollama, Llama.cpp, MLX) and their capabilities.
• GET /api/v1/installed queries the local cache to show which models are already downloaded and available for inference.

Model Management

• POST /api/v1/download initiates a model download. The request body must include the model identifier and can optionally specify a target runtime (e.g., Ollama or Llama.cpp).
• GET /api/v1/download/{id}/status streams download progress for standard operations or polls real-time status when using LM Studio integration.

Execution Planning

POST /api/v1/plan accepts a JSON payload describing hardware constraints (such as ram_gb and vram_gb) and model requirements, then returns a fit-estimation payload including predicted memory usage, throughput, and recommended run mode. This endpoint directly exposes the planning engine from llmfit-core/src/analysis.rs.

Practical Usage Examples

All examples assume the server is running locally on the default port.

Retrieve system specifications:

curl http://localhost:8787/api/v1/system

Get the top 10 marginal-fit models for the "coding" use case:

curl "http://localhost:8787/api/v1/models/top?limit=10&min_fit=marginal&use_case=coding"

Download a model via POST:

curl -X POST -H "Content-Type: application/json" \
     -d '{"name":"mistralai/Mistral-7B-Instruct-v0.2","runtime":"llamacpp"}' \
     http://localhost:8787/api/v1/download

Check download status:

curl http://localhost:8787/api/v1/download/abc123/status

Request a hardware plan estimation:

curl -X POST -H "Content-Type: application/json" \
     -d '{"model":"mistralai/Mistral-7B-Instruct-v0.2","ram_gb":48,"vram_gb":12}' \
     http://localhost:8787/api/v1/plan

Key Source Files

Understanding the request flow requires examining these specific source locations:

• llmfit-tui/src/serve_api.rs: Defines the Axum router and handler implementations for all /api/v1/* routes.
• llmfit-core/src/analysis.rs: Contains the core logic for calculating model fits and generating execution plans used by the /models and /plan endpoints.
• llmfit-core/src/models.rs: Defines the model metadata schema returned by the model listing endpoints.
• llmfit-core/src/hardware.rs: Implements system detection feeding the /system endpoint.
• API.md: Provides human-readable documentation and additional query parameter details.
• scripts/test_api.py: Integration test suite validating endpoint behavior against live server instances.

Summary

• The llmfit API endpoints are served via Axum on port 8787 under the /api/v1/ prefix.
• GET /api/v1/system provides hardware detection for CPUs, RAM, and GPUs.
• Model discovery supports filtering by runtime, fit quality, and use case via /api/v1/models and /models/top.
• POST /api/v1/download and GET /api/v1/download/{id}/status manage model retrieval and progress tracking.
• POST /api/v1/plan generates hardware-specific execution estimates using the core analysis engine.
• All endpoints return JSON structures defined in llmfit-core, ensuring consistency across the TUI, dashboard, and external clients.

Frequently Asked Questions

What port does the llmfit API use by default?

The server binds to port 8787 by default, as defined in the Axum initialization code within llmfit-tui/src/serve_api.rs. You can verify the active port by checking the server logs printed during startup.

How do I filter models by use case or runtime?

Append query parameters to the /api/v1/models endpoint. Valid parameters include use_case (e.g., "coding"), runtime (e.g., "ollama"), min_fit (e.g., "marginal"), and sort. The /api/v1/models/top endpoint provides a convenient shortcut for retrieving the highest-scoring runnable models.

Can I download models via the API?

Yes. Send a POST request to /api/v1/download with a JSON body containing the model name and optional runtime specification. Poll /api/v1/download/{id}/status to track progress, or use the LM Studio integration for real-time status updates.

Where is the API server code located?

The HTTP server implementation resides in llmfit-tui/src/serve_api.rs in the AlexsJones/llmfit repository. This file contains the Axum router setup, route handlers, and the endpoint logging logic referenced in the startup sequence.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}