# How to Connect FreeLLMAPI to Local LLMs Running Ollama or LM Studio

> Easily connect FreeLLMAPI to local LLMs with Ollama or LM Studio. Our built-in emulation layer unifies access to OpenAI-compatible servers for seamless integration. Get started today.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-08-26

---

**Yes, FreeLLMAPI includes a built‑in Ollama emulation layer that proxies any OpenAI‑compatible local server—including Ollama, LM Studio, llama.cpp, and vLLM—through a unified API surface.**

FreeLLMAPI bridges the gap between local inference engines and standard client libraries by presenting your self‑hosted models as first‑class endpoints. When enabled, the gateway detects local instances, normalizes model identifiers, and translates between OpenAI‑style and Ollama‑native wire formats, allowing you to use the same SDKs and CLI tools you already know.

## Understanding the Ollama Emulation Architecture

The emulation system is opt‑in and disabled by default. Once activated via the **Key → Agent** UI or the `--ollama-emulation` CLI flag, FreeLLMAPI establishes a transparent bridge between client applications and your local inference server.

At its core, the architecture performs four critical functions:

- **Local endpoint detection** – Identifies traffic destined for `localhost` or private subnets and exempts it from cloud‑style quota enforcement.
- **Model name normalization** – Strips Ollama‑specific tags (e.g., `:latest`) and maps proprietary identifier fields to canonical model names.
- **Request protocol translation** – Rewrites OpenAI‑compatible JSON payloads to match Ollama’s expected schema and vice‑versa.
- **Authentication middleware** – Optionally secures local endpoints with per‑token URL patterns while preserving the standard REST interface.

## How FreeLLMAPI Detects and Routes to Local Endpoints

### URL Detection and Trust Rules

In [`server/src/lib/url-guard.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/url-guard.ts), FreeLLMAPI implements a hostname whitelist that recognizes when a request targets a local Ollama or LM Studio instance. When the guard detects destinations such as `localhost:11434` or other loopback addresses, it marks the connection as **trusted** and bypasses rate‑limit quotas that would normally apply to upstream cloud providers.

This means you can run high‑frequency inference jobs against your local GPU without triggering the gateway’s standard throttling mechanisms.

### Model Identifier Normalization

The `normalizeOllamaModel()` function in [`server/src/routes/ollama.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/ollama.ts) sanitizes incoming model strings. For example, a client request specifying `llama2:latest` is stripped of its tag and mapped to the base identifier expected by the local runtime. This normalization ensures compatibility across client tools that may use different naming conventions for the same underlying weights.

## Configuring OpenAI-Compatible Routing

The [`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts) file contains the translation logic that powers the integration. When FreeLLMAPI receives a request at the `/v1/*` prefix, it:

1. Rewrites the `baseURL` to point at your local server (e.g., `http://localhost:11434/v1`).
2. Translates response fields, extracting `message.reasoning` from Ollama outputs when present.
3. Forwards streaming chunks with the correct SSE headers for OpenAI SDK compatibility.

This provider also exposes the full Ollama wire format under `/api/*`, implementing endpoints for tags, chat, generate, show, version, embed, and legacy embeddings as documented in [`server/src/docs/openapi.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/docs/openapi.ts).

## Enabling Ollama Emulation Mode

You can activate the emulation layer through the command line or the web dashboard.

**Option 1: Open-loopback mode (no authentication)**

This configuration allows any client on your local machine to connect without presenting an API key:

```bash
freellmapi config set ollamaEmulation open-loopback

```

**Option 2: Key-required mode (authenticated)**

For shared development environments, generate a token and require it in the URL path:

```bash

# Create a dedicated token

freellmapi token create --name ollama-token

# Use the token in your requests

curl http://localhost:3001/v1/t/<TOKEN>/chat \
  -H "Content-Type: application/json" \
  -d '{"model":"llama2","messages":[{"role":"user","content":"Hello"}]}'

```

## Connecting Client Applications

### Using the Ollama Native API

Send requests directly to the Ollama‑compatible endpoints exposed by FreeLLMAPI:

```bash
curl -X POST http://localhost:3001/api/chat \
  -H "Content-Type: application/json" \
  -d '{
        "model": "llama2:latest",
        "messages": [{"role":"user","content":"Explain quantum computing"}]
      }'

```

### Using the OpenAI SDK with LM Studio

Any OpenAI‑compatible SDK can target your local LLM through FreeLLMAPI by changing the `baseURL`:

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:3001/v1",
  apiKey: "dummy",  // Ignored in open-loopback mode
});

const response = await client.chat.completions.create({
  model: "llama2:latest",
  messages: [{ role: "user", content: "Explain quantum tunneling." }],
});

console.log(response.choices[0].message.content);

```

This approach works identically for **LM Studio**, which exposes an OpenAI‑compatible server on its own port; FreeLLMAPI simply proxies the traffic while adding its own authentication and logging layers.

## Supported Local LLM Runtimes

FreeLLMAPI’s emulation layer is runtime‑agnostic and officially supports:

- **Ollama** – Primary target with full tag and embedding support.
- **LM Studio** – OpenAI‑compatible local server mode.
- **llama.cpp** – When running with `-lv` or similar OpenAI‑compatible server flags.
- **vLLM** – Via its built‑in OpenAI‑compatible API implementation.

Any client that understands the Ollama protocol—including the **Ollama CLI**, **Zed editor**, and **JetBrains AI Assistant**—can connect without code changes once the emulation is enabled.

## Summary

- FreeLLMAPI connects to local LLMs through an **opt‑in Ollama emulation layer** that must be enabled via CLI or dashboard.
- The system uses [`server/src/lib/url-guard.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/url-guard.ts) to **detect local endpoints** and exempt them from quotas.
- Model names are normalized via `normalizeOllamaModel()` in [`server/src/routes/ollama.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/ollama.ts) to handle Ollama‑specific tags.
- The [`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts) file handles protocol translation between OpenAI‑style and Ollama‑native formats.
- You can run in **open-loopback mode** for private use or **key‑required mode** to secure local endpoints with URL‑based tokens.
- Supported runtimes include Ollama, LM Studio, llama.cpp, and vLLM.

## Frequently Asked Questions

### Does FreeLLMAPI work specifically with LM Studio?

Yes. LM Studio exposes an OpenAI‑compatible local server that FreeLLMAPI can proxy through the same `openai-compat` provider used for Ollama. Simply point FreeLLMAPI at LM Studio’s local port (typically `http://localhost:1234/v1`), and the gateway will handle request forwarding and response normalization.

### Do I need an API key to connect to local models?

Not in **open-loopback** mode. This configuration trusts all localhost traffic and ignores the `Authorization` header. If you enable **key‑required** mode, you must generate a token via `freellmapi token create` and include it in the URL path (`/v1/t/{token}/...`), but the key is only validated by FreeLLMAPI, not passed to the local LLM runtime.

### Can I use the standard Ollama CLI with FreeLLMAPI?

Yes. Configure the Ollama CLI to point at FreeLLMAPI’s base URL:

```bash
export OLLAMA_HOST=http://localhost:3001
ollama run llama2

```

The CLI will communicate using the native Ollama protocol, and FreeLLMAPI will route requests to your actual local instance while applying any configured middleware.

### Will local requests count against my rate limits?

No. The [`url-guard.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/url-guard.ts) module explicitly identifies local hostnames and loopback addresses, marking them as trusted. This exempts the traffic from the quota enforcement and billing logic that applies to cloud‑based providers, allowing unlimited local inference.