# Best Practices for Using FreeLLM API: Security, Routing & Performance Guide > Discover best practices for FreeLLM API: secure provider keys with AES-256-GCM, use model auto, and enable sticky sessions for peak performance and reliability. Read our guide now. - Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi) - Tags: best-practices - Published: 2026-07-02 --- **Use `model: "auto"`, enable sticky sessions, and store provider keys encrypted with AES‑256‑GCM to maximize reliability and security.** FreeLLM API is an open‑source, OpenAI‑compatible router that aggregates 18 free‑tier LLM providers and 161 models behind a single `/v1` endpoint. According to the tashfeenahmed/freellmapi source code, following architectural best practices ensures your deployment remains secure, performant, and resilient to provider outages. --- ## Secure API Key Management ### Encrypt Provider Keys at Rest The router encrypts every upstream provider key using **AES‑256‑GCM** before writing to SQLite. This guarantees that a compromised host cannot read raw API keys. In [`server/src/lib/crypto.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/crypto.ts), the encryption envelope includes a nonce and authentication tag, ensuring confidentiality and integrity. ### Isolate Apps with a Unified API Key Clients authenticate with a `freellmapi‑…` bearer token, not the upstream provider keys. This **unified API key** isolates downstream services from your secrets. Configure it in your environment file: ```bash # .env - generated on first run, never commit to version control FREELLMAPI_KEY=freellmapi-your-unified-key ``` Reference the `.env.example` file for the full variable list. --- ## Optimize Model Selection and Routing ### Prefer `model: "auto"` for Intelligent Routing Setting `model: "auto"` delegates model selection to the router. The logic in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) evaluates health status, rate limits, and your configured fallback chain to pick the best available model automatically. ```python from openai import OpenAI client = OpenAI( base_url="http://localhost:3001/v1", api_key="freellmapi-your-unified-key", ) resp = client.chat.completions.create( model="auto", # Router selects best available model messages=[{"role": "user", "content": "Summarise quantum computing."}], ) print(resp.choices[0].message.content) print("Routed via:", resp.headers.get("x-routed-via")) ``` ### Configure Your Fallback Chain The router respects a user‑defined fallback order. Place your most capable (and least rate‑limited) models at the top using the dashboard **Keys → Fallback Chain** page. This directly influences the selection logic in [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts) when automatic fall‑over occurs on 429/5xx responses. --- ## Handle Session Continuity and Context ### Enable Sticky Sessions The router remembers the selected model for **30 minutes per session**, preventing mid‑conversation model switches that can cause hallucination spikes. This behavior is documented in the README features section. ### Turn On Context Handoff for Model Switches When a model switch is unavoidable, set `FREELLMAPI_CONTEXT_HANDOFF=on_model_switch` in your environment. This injects a concise system message informing the new model that it is taking over an ongoing task, preserving conversational continuity. --- ## Monitor Health and Rate Limits ### Leverage Built‑In Health Probes The [`server/src/services/health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/health.ts) module runs periodic probes that classify each key as `healthy`, `rate_limited`, `invalid`, or `error`. The router uses this status to avoid routing to degraded providers. ### Respect Rate Limits with the Ledger In‑memory RPM/RPD/TPM/TPD counters in [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts) enforce caps and cooldowns per key. The router automatically cools down failing keys and retries others, but you should monitor the analytics dashboard to identify patterns. ### Rotate Keys Regularly Replace invalid or rate‑limited keys via the dashboard **Keys** page. The router handles hot‑swapping without downtime. --- ## Keep the Model Catalog Current The router pulls a signed catalog from `freellmapi.co` twice daily, as implemented in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts). For production deployments: - Run the latest version to receive catalog updates - Enable the **Premium (live catalog)** feed to access new models the day they ship This ensures you have access to the full 161‑model pool and any newly added providers. --- ## Deployment and Network Security ### Run in a Trusted Environment Bind the service appropriately for your network topology: ```bash # For local-only access HOST_BIND=127.0.0.1 # For LAN access (protect with firewall rules) HOST_BIND=0.0.0.0 ``` Protect remote access to the unified key with TLS or SSH tunneling. The Docker setup in the README provides containerized isolation. --- ## Advanced Usage Patterns ### Streaming Responses Stream tokens efficiently without managing model selection manually: ```python stream = client.chat.completions.create( model="auto", messages=[{"role": "user", "content": "Stream me a haiku about SQLite."}], stream=True, ) for chunk in stream: print(chunk.choices[0].delta.content or "", end="", flush=True) ``` ### Tool Calling Workflows The router supports OpenAI‑compatible tool calling across providers: ```python tools = [{ "type": "function", "function": { "name": "get_weather", "description": "Get current weather for a city.", "parameters": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], }, }, }] # Step 1: Model requests tool call first = client.chat.completions.create( model="auto", messages=[{"role": "user", "content": "What's the weather in Karachi?"}], tools=tools, tool_choice="required", ) call = first.choices[0].message.tool_calls[0] # Step 2: Execute tool and continue conversation final = client.chat.completions.create( model="auto", messages=[ {"role": "user", "content": "What's the weather in Karachi?"}, first.choices[0].message, {"role": "tool", "tool_call_id": call.id, "content": '{"temp_c": 32, "cond": "sunny"}'}, ], tools=tools, ) print(final.choices[0].message.content) ``` ### Vision and Embeddings Vision requests follow the same `model: "auto"` pattern: ```python resp = client.chat.completions.create( model="auto", messages=[{ "role": "user", "content": [ {"type": "text", "text": "What's in this image?"}, {"type": "image_url", "image_url": {"url": "data:image/png;base64,"}}, ], }], ) ``` For embeddings, use the auto family selector: ```python emb = client.embeddings.create( model="auto", # Defaults to gemini-embedding-001 family input=["text to embed", "more text"], ) print(f"{len(emb.data)} vectors of {len(emb.data[0].embedding)} dimensions") ``` --- ## Extending with Provider Adapters When adding new providers, follow the template in [`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts). Each adapter must implement `chatCompletion()` and `streamChatCompletion()` methods to integrate with the router's provider-agnostic interface. --- ## Summary - **Encrypt keys** using AES‑256‑GCM via [`server/src/lib/crypto.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/crypto.ts) and isolate apps with a unified bearer token - **Use `model: "auto"`** to let [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) select optimal models based on health and rate limits - **Configure fallback chains** in the dashboard, placing reliable models first - **Enable sticky sessions** and **context handoff** (`FREELLMAPI_CONTEXT_HANDOFF=on_model_switch`) for conversation continuity - **Monitor health** via [`server/src/services/health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/health.ts) and **rate limits** via [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts) - **Rotate keys** regularly and **keep catalogs fresh** by updating to the latest version - **Secure deployment** with proper `HOST_BIND` settings and TLS for remote access --- ## Frequently Asked Questions ### How does FreeLLM API secure my provider API keys? FreeLLM API encrypts all provider keys using AES‑256‑GCM before storing them in SQLite, as implemented in [`server/src/lib/crypto.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/crypto.ts). This envelope encryption ensures that even if the host is compromised, the raw keys remain inaccessible without the decryption key. ### What happens when a provider returns a 429 or 5xx error? The router in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) automatically falls back to the next healthy model in your configured chain. The health service tracks key status in real-time, and the rate-limit ledger manages cooldowns to prevent hammering failing endpoints. ### Can I force a specific model instead of using "auto"? Yes, you can pass any specific model name from the 161-model catalog. However, using `model: "auto"` is recommended because it leverages health checks, rate-limit awareness, and your fallback preferences to maximize availability and quality. ### How do I add a new provider to the router? Create a new adapter in `server/src/providers/` following the pattern in [`openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/openai-compat.ts). Each adapter must implement `chatCompletion()` and `streamChatCompletion()` methods that conform to the OpenAI-compatible interface expected by the router.