# How to Serve the llmfit REST API: Complete Setup and Endpoint Guide

> Learn to serve the llmfit REST API using the llmfit-tui binary. This guide covers Axum server setup and JSON endpoints for model discovery, system specs, and deployment planning.

- Repository: [Alex Jones/llmfit](https://github.com/AlexsJones/llmfit)
- Tags: how-to-guide
- Published: 2026-08-19

---

**The llmfit REST API is served by the `llmfit-tui` binary which starts an Axum-based HTTP server via the `run_serve` function in [`llmfit-tui/src/serve_api.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/serve_api.rs), exposing JSON endpoints for model discovery, system specs, and deployment planning on TCP or Unix sockets.**

The **AlexsJones/llmfit** repository provides a comprehensive hardware compatibility analyzer for Large Language Models. Beyond its terminal interface, llmfit embeds a production-ready **REST API** server that exposes the core analysis engine via HTTP, enabling programmatic access to model fitting algorithms, hardware detection, and deployment planning without launching the interactive TUI.

## Starting the llmfit REST API Server

The server is embedded within the `llmfit-tui` crate and activated through command-line flags. You can bind to a TCP address for network access or use a Unix domain socket for local-only communication.

**Start on TCP (default):**

```bash
cargo run --bin llmfit-tui -- --host 0.0.0.0 --port 8080

```

**Start on Unix socket (Linux/macOS):**

```bash
cargo run --bin llmfit-tui -- --unix-socket /tmp/llmfit.sock

```

Once started, the binary prints the listening address and sample URLs, such as `http://0.0.0.0:8080/api/v1/models?limit=20&min_fit=marginal&sort=score`.

## Server Architecture and Startup Flow

When `run_serve` is invoked in [`llmfit-tui/src/serve_api.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/serve_api.rs), it executes a five-phase initialization sequence:

1.  **Hardware Detection** – Calls `super::detect_specs` to generate a **SystemSpecs** struct containing CPU, RAM, and GPU capabilities.
2.  **Catalog Loading** – Instantiates a **ModelDatabase** and clones the complete model list into memory.
3.  **State Construction** – Builds an **AppState** containing node information, hardware specs, the model list, optional context limits, and a thread-safe download tracker.
4.  **Router Assembly** – Invokes `build_router(state)` to wire Axum routes for all public endpoints.
5.  **Runtime Binding** – Launches an async **Tokio** runtime and binds either a TCP listener or Unix domain socket.

All handlers return either `Json` responses or an `ApiError` that implements `IntoResponse`, ensuring errors are always serialized as `{"error":"..."}` with appropriate HTTP status codes.

## REST API Endpoint Reference

The `build_router` function in [`llmfit-tui/src/serve_api.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/serve_api.rs) exposes the following endpoints, organized by functional area:

### Health and System Information

-   **`GET /health`** – Simple liveness check returning operational status and node metadata.
-   **`GET /api/v1/system`** – Returns detailed hardware detection results (CPU, RAM, VRAM). Supports **hardware overrides** via query parameters (`ram_gb`, `vram_gb`, `cpu_cores`) merged by the `effective_specs` function for on-the-fly "what-if" analysis.

**Example with hardware overrides:**

```bash
curl "http://localhost:8080/api/v1/system?ram_gb=64&vram_gb=24&cpu_cores=16" | jq

```

### Model Discovery and Ranking

-   **`GET /api/v1/models`** – Returns a paginated list of model fits with extensive filtering capabilities (provider, minimum fit level, use case).
-   **`GET /api/v1/models/top`** – Shortcut endpoint for retrieving the top N models by a specific sort criterion.
-   **`GET /api/v1/models/{name}`** – Retrieves detailed fit information for a specific model by exact name match.

**Filter for Ollama-compatible coding models:**

```bash
curl "http://localhost:8080/api/v1/models?provider=ollama&min_fit=good&use_case=coding" | jq

```

### Runtime and Installation Status

-   **`GET /api/v1/runtimes`** – Detects which provider runtimes (Ollama, Mlx, LlamaCpp, etc.) are installed on the host system.
-   **`GET /api/v1/installed`** – Lists specific models already installed per detected runtime.

### Download and Deployment Operations

-   **`POST /api/v1/download`** – Initiates a model pull operation. **Restricted to localhost connections only** for security.
-   **`GET /api/v1/download/{id}/status`** – Polls the progress of an active download.
-   **`POST /api/v1/plan`** – Generates a deployment plan estimating context limits, quantization levels, and expected tokens-per-second for specified hardware constraints.

**Start a download (localhost only):**

```bash
curl -X POST http://localhost:8080/api/v1/download \
    -H "Content-Type: application/json" \
    -d '{"model":"llama-2-7b","runtime":"ollama"}' | jq

```

**Estimate a deployment plan:**

```bash
curl -X POST http://localhost:8080/api/v1/plan \
    -H "Content-Type: application/json" \
    -d '{
          "model":"llama-2-7b",
          "context":2048,
          "quant":"q4_0",
          "target_tps":30,
          "ram_gb":48,
          "vram_gb":12,
          "cpu_cores":8
        }' | jq

```

## Security Model and Localhost Restrictions

Because download endpoints manipulate the local filesystem, they implement strict **localhost validation**. The handlers extract `ConnectInfo<SocketAddr>` and verify `addr.ip().is_loopback()`; requests from remote addresses return `403 FORBIDDEN`. This security boundary is enforced in [`serve_api.rs`](https://github.com/AlexsJones/llmfit/blob/main/serve_api.rs) alongside the route definitions.

## Hardware Override for "What-If" Analysis

The API supports temporary hardware specification overrides without modifying system detection. When query parameters like `ram_gb=128` or `vram_gb=48` are present, the `effective_specs` function (lines 331-365 in [`serve_api.rs`](https://github.com/AlexsJones/llmfit/blob/main/serve_api.rs)) merges these values into the base **SystemSpecs**, allowing you to simulate deployment scenarios on different hardware configurations via API calls alone.

## Embedded React Dashboard

All static web assets for the React-based dashboard are compiled into the binary via `include!(concat!(env!("OUT_DIR"), "/web_assets.rs"))`. The fallback route `/*` serves [`index.html`](https://github.com/AlexsJones/llmfit/blob/main/index.html), while `serve_web_path` handles content-type detection and caching headers for assets under `/assets/*`. Simply navigate to the root URL to access the graphical interface.

## Summary

-   The **llmfit REST API** is served by the `llmfit-tui` binary using Axum and Tokio, defined primarily in [`llmfit-tui/src/serve_api.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/serve_api.rs).
-   Start the server with `--host` and `--port` for TCP or `--unix-socket` for local socket access.
-   Endpoints cover health checks, system specs, model discovery, runtime detection, downloads, and deployment planning.
-   **Download endpoints** are strictly limited to localhost connections for security.
-   **Hardware overrides** enable simulated "what-if" analysis by passing query parameters to relevant endpoints.
-   A full **React dashboard** is embedded in the binary and served via fallback routes.

## Frequently Asked Questions

### How do I start the llmfit REST API on a custom port?

Pass the `--port` flag when running the `llmfit-tui` binary. For example, `cargo run --bin llmfit-tui -- --host 0.0.0.0 --port 9000` binds the server to port 9000 on all interfaces. You may also specify `--unix-socket /path/to/socket` for Unix domain socket access instead of TCP.

### Can I access the llmfit API from remote machines?

Yes, the TCP listener respects the `--host` binding (e.g., `0.0.0.0` for all interfaces). However, specific endpoints like `POST /api/v1/download` enforce localhost-only access by checking the source IP address; these will return a 403 error if accessed remotely regardless of the host binding.

### What framework powers the llmfit REST API?

The server is built on **Axum**, part of the Tokio ecosystem, as implemented in [`llmfit-tui/src/serve_api.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/serve_api.rs). It uses standard Tower middleware for routing and `axum::extract::ConnectInfo` for client IP detection. All JSON serialization relies on shared conversion utilities in [`llmfit-tui/src/serve_shared.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/serve_shared.rs).

### How do I perform "what-if" hardware analysis via the API?

Append hardware override parameters to qualifying endpoints like `/api/v1/system` or `/api/v1/plan`. Valid parameters include `ram_gb`, `vram_gb`, and `cpu_cores`. The `effective_specs` function merges these overrides with detected system specs, returning results as if the hardware matched your specified values without requiring physical hardware changes.