# How to Use FreeLLMAPI with LangChain and LlamaIndex: Complete Integration Guide

> Integrate FreeLLMAPI with LangChain and LlamaIndex seamlessly. This guide shows how to use the OpenAI-compatible API with simple configuration changes for self-hosted LLMs.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-06-24

---

**Yes, FreeLLMAPI integrates seamlessly with LangChain and LlamaIndex by exposing an OpenAI-compatible REST API that requires only a base URL configuration change to redirect requests from OpenAI's servers to your self-hosted instance.**

The `tashfeenahmed/freellmapi` repository provides a self-hosted HTTP interface that mirrors the OpenAI chat-completion and embeddings schema. Because the request/response format follows the de-facto OpenAI specification, you can use FreeLLMAPI with LangChain or LlamaIndex as a drop-in replacement for OpenAI's official API, routing all LLM calls through your own infrastructure.

## Why FreeLLMAPI Works with LangChain and LlamaIndex

FreeLLMAPI implements the standard OpenAI REST API contract, making it compatible with any client library expecting that interface. In [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts), the Express router registers HTTP endpoints including `/v1/chat/completions` and `/v1/embeddings`, accepting requests that match OpenAI's payload structure.

Both LangChain and LlamaIndex expose configuration parameters that override the default OpenAI base URL (`https://api.openai.com`). By pointing these parameters to your FreeLLMAPI instance (e.g., `http://localhost:3000/v1`), you create a transparent proxy that routes all LLM and embedding requests through your self-hosted backend. No additional code changes are required beyond these initialization settings.

## Configuring LangChain to Use FreeLLMAPI

### Required Parameters

To redirect LangChain's OpenAI client to FreeLLMAPI, you must provide two key arguments:

- **`openai_api_base`**: Set to your FreeLLMAPI endpoint URL (e.g., `http://localhost:3000/v1`)
- **`openai_api_key`**: Any non-empty string works, as [`server/src/services/auth.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/auth.ts) validates keys against an internal quota system rather than OpenAI's authentication service

### Implementation Example

The following snippet demonstrates a complete LangChain integration using the `OpenAI` LLM class:

```python
from langchain.llms import OpenAI
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate

# Configure the client to use FreeLLMAPI

llm = OpenAI(
    temperature=0.7,
    openai_api_base="http://localhost:3000/v1",  # FreeLLMAPI base URL

    openai_api_key="dummy-key"                    # Any non-empty string works

)

# Create and run a chain

prompt = PromptTemplate.from_template("Write a haiku about {topic}.")
chain = LLMChain(llm=llm, prompt=prompt)

print(chain.run({"topic": "autumn"}))

```

## Integrating LlamaIndex with FreeLLMAPI

### Connection Setup

LlamaIndex uses similar configuration options to LangChain. When initializing the OpenAI-compatible LLM, specify:

- **`api_base`**: Your FreeLLMAPI server address with the `/v1` path
- **`api_key`**: A placeholder value (validated locally in [`server/src/services/auth.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/auth.ts))

### Building Indexes with FreeLLMAPI

You can use FreeLLMAPI as the underlying LLM for document indexing and querying:

```python
from llama_index.llms import OpenAI
from llama_index import SimpleDirectoryReader, GPTVectorStoreIndex

# Point to FreeLLMAPI endpoint

llm = OpenAI(
    temperature=0.5,
    model="gpt-3.5-turbo",
    api_base="http://localhost:3000/v1",   # FreeLLMAPI endpoint

    api_key="any-key"
)

# Build index using FreeLLMAPI for LLM operations

documents = SimpleDirectoryReader("data").load_data()
index = GPTVectorStoreIndex.from_documents(documents, llm=llm)

query_engine = index.as_query_engine()
print(query_engine.query("What are the main features of FreeLLMAPI?"))

```

## Working with Embeddings

FreeLLMAPI provides an `/v1/embeddings` endpoint implemented in [`server/src/services/embeddings.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/embeddings.ts), supporting vector generation for both LangChain and LlamaIndex. Configure the embedding classes the same way you configure the LLM classes:

```python
from langchain.embeddings import OpenAIEmbeddings

emb = OpenAIEmbeddings(
    model="text-embedding-ada-002",
    openai_api_base="http://localhost:3000/v1",
    openai_api_key="dummy"
)

vector = emb.embed_query("FreeLLMAPI architecture")

```

This compatibility extends to vector stores, retrievers, and agent tools in both frameworks that rely on OpenAI embedding models.

## Key FreeLLMAPI Components for Integration

Understanding the server architecture helps troubleshoot integration issues:

- **[`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts)**: Registers the HTTP routes (`/v1/chat/completions`, `/v1/embeddings`) that LangChain and LlamaIndex call
- **[`server/src/services/auth.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/auth.ts)**: Validates API keys and enforces per-key usage quotas, accepting any non-empty string for local development
- **[`server/src/services/embeddings.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/embeddings.ts)**: Handles embedding requests with the same request/response contract as OpenAI
- **[`server/src/services/model-listing.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/model-listing.ts)**: Returns available models, which both frameworks may query to validate model names
- **[`server/src/services/scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/scoring.ts)**: Routes chat completion requests to underlying LLM providers (OpenAI, Anthropic, etc.) based on the requested model

## Summary

- FreeLLMAPI exposes OpenAI-compatible endpoints via [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts), making it interoperable with LangChain and LlamaIndex
- Configure LangChain using `openai_api_base` and `openai_api_key` parameters to point to your FreeLLMAPI instance
- Configure LlamaIndex using `api_base` and `api_key` parameters with the same endpoint URL
- Embeddings are available through the `/v1/embeddings` endpoint defined in [`server/src/services/embeddings.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/embeddings.ts)
- No code changes are required in your LangChain or LlamaIndex logic beyond the client initialization configuration

## Frequently Asked Questions

### Is FreeLLMAPI fully compatible with LangChain's OpenAI integration?

Yes. Because FreeLLMAPI implements the standard OpenAI REST API schema in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts), LangChain's OpenAI classes work without modification when you override the base URL to point to your FreeLLMAPI server.

### What authentication is required when connecting LlamaIndex to FreeLLMAPI?

You must provide any non-empty string as the API key. According to [`server/src/services/auth.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/auth.ts), FreeLLMAPI validates keys against its internal quota system rather than OpenAI's authentication, allowing dummy keys like `"any-key"` or `"dummy"` for local development.

### Can I use FreeLLMAPI embeddings with existing LangChain vector stores?

Yes. The `/v1/embeddings` endpoint implemented in [`server/src/services/embeddings.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/embeddings.ts) follows the OpenAI specification exactly. You can use `OpenAIEmbeddings` with a custom `openai_api_base` URL to generate embeddings for any LangChain vector store or LlamaIndex index.

### Do I need to modify my existing LangChain or LlamaIndex code to switch to FreeLLMAPI?

No. Both frameworks support configuration parameters that override the default OpenAI base URL. You only need to change initialization parameters (`openai_api_base` for LangChain, `api_base` for LlamaIndex) without modifying your chain definitions, prompt templates, or query logic.