# How to Configure LLM Routing with Proxy and BYO Modes in TencentDB Agent Memory

> Learn to configure LLM routing with proxy and BYO modes in TencentDB Agent Memory. Optimize your LLM calls by understanding default proxy and custom BYO settings for efficient agent requests.

- Repository: [Tencent Cloud/TencentDB-Agent-Memory](https://github.com/TencentCloud/TencentDB-Agent-Memory)
- Tags: how-to-guide
- Published: 2026-08-26

---

**TencentDB Agent Memory supports two mutually exclusive LLM routing modes: `proxy` (default), which routes all agent requests through the Memory Proxy service at port 8096, and `custom` (BYO), which bypasses the proxy for direct LLM calls using the memory gateway configuration.**

When you configure LLM routing with proxy and BYO modes, you determine whether AI agent requests flow through an intermediary proxy layer or communicate directly with your chosen LLM provider. This architecture, implemented in the `TencentCloud/TencentDB-Agent-Memory` repository, uses the `LLM_MODE` environment variable to control request flow between the Panel UI, Agent SDKs, and upstream language models.

## Understanding the Two LLM Routing Modes

TencentDB Agent Memory separates routing logic from memory services through distinct operational modes defined in the deployment configuration.

**Proxy Mode (Default)** routes all LLM calls from agents through the **Memory Proxy** (`tdai-proxy`). According to [`deploy/panel-knowledge-combined/start-combined.sh`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/deploy/panel-knowledge-combined/start-combined.sh) (lines 82-96), when `LLM_MODE=proxy`, the system writes a `proxy_endpoint` field into [`metadata-instances.json`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/metadata-instances.json), causing client SDKs to target `http://<host>:8096`. The proxy then forwards requests to an upstream LLM defined by `PROXY_UPSTREAM_URL`, `PROXY_UPSTREAM_API_KEY`, and `PROXY_UPSTREAM_MODEL`, while injecting context assets and enforcing authentication logic implemented in [`MemoryProxy/src/handler.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryProxy/src/handler.ts).

**BYO (Custom) Mode** bypasses the proxy entirely. When `LLM_MODE=custom`, the Panel omits the `proxy_endpoint` field (falling back to `gateway_endpoint` per lines 26-40 of [`start-combined.sh`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/start-combined.sh)), and agents call the LLM directly using the memory group configuration (`MEMORY_UPSTREAM_URL`, `MEMORY_UPSTREAM_API_KEY`, `MEMORY_UPSTREAM_MODEL`). This mode eliminates proxy-level features like session initialization and cost guarding in favor of lower latency direct connections.

## Configuration File Structure

Before implementing either mode, understand the key source files controlling the routing behavior:

- **[`deploy/panel-knowledge-combined/start-combined.sh`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/deploy/panel-knowledge-combined/start-combined.sh)** – Sets the `LLM_MODE` default value and injects routing endpoints into the panel configuration JSON.
- **`deploy/global-images/.env.example`** – Defines the complete schema for environment variables across both routing modes.
- **[`MemoryProxy/src/handler.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryProxy/src/handler.ts)** – Contains the core request-forwarding logic that processes proxy-mode LLM calls.
- **[`deploy/global-images/start-proxy.sh`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/deploy/global-images/start-proxy.sh)** – Generates the proxy's [`config.yaml`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/config.yaml) from environment variables (lines 46-78).

## Step-by-Step Configuration Guide

### Initialize Environment Variables

Begin by copying the example environment file in the global-images directory:

```bash
cd deploy/global-images
cp .env.example .env

```

Edit `.env` to set your base LLM credentials. These variables serve as fallbacks or primary configuration depending on your selected mode.

### Configure Proxy Mode (Default)

To enable the default routing through Memory Proxy:

1. Set the mode identifier:
   ```ini
   LLM_MODE=proxy
   ```

2. Define the client-facing proxy address:
   ```ini
   REMOTE_INSTANCE_PROXY_URL=http://localhost:8096
   ```

3. Configure the upstream LLM that the proxy will call:
   ```ini
   PROXY_UPSTREAM_URL=https://api.openai.com/v1
   PROXY_UPSTREAM_API_KEY=sk-your-openai-key
   PROXY_UPSTREAM_MODEL=gpt-4o
   ```

According to the README (lines 90-95), the Panel UI will display this `REMOTE_INSTANCE_PROXY_URL` value in the "客户端接入地址" (Client Access Address) card, ensuring agents connect through the managed proxy endpoint.

### Configure BYO (Custom) Mode

To bypass the proxy and enable direct LLM communication:

1. Set the mode to custom:
   ```ini
   LLM_MODE=custom
   ```

2. Remove or comment out `REMOTE_INSTANCE_PROXY_URL`. The Panel will automatically fall back to the `gateway_endpoint`.

3. Configure the memory group variables for direct access:
   ```ini
   MEMORY_UPSTREAM_URL=https://api.deepseek.com/v1
   MEMORY_UPSTREAM_API_KEY=sk-your-deepseek-key
   MEMORY_UPSTREAM_MODEL=deepseek-coder-v2
   ```

In this configuration, the [`deploy/global-images/start-all.sh`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/deploy/global-images/start-all.sh) script (lines 52-66) orchestrates container startup without initializing the proxy service dependency, and agents connect directly to the memory-core gateway at port 8125.

### Launch the Complete Stack

After configuring `.env`, execute the orchestration script:

```bash
./start-all.sh

```

This script validates the `LLM_MODE` variable and spins up the appropriate services (memory-core, memory-hub, and optionally tdai-proxy).

## SDK Implementation Examples

### Using Proxy Mode in TypeScript

When operating in proxy mode, point the SDK to the proxy port (8096). The proxy handles authentication with the upstream provider:

```typescript
import { MemoryClient } from '@tencentdb-agent-memory/memory-core';

const client = new MemoryClient({
  baseURL: 'http://localhost:8096/v3',
  apiKey: '' // Authentication handled by proxy configuration
});

await client.chatMemory.create({
  messages: [{ role: 'user', content: 'Hello' }]
});

```

### Using BYO Mode in TypeScript

For custom mode, target the memory-hub gateway directly and provide your LLM API key in the client constructor:

```typescript
import { MemoryClient } from '@tencentdb-agent-memory/memory-core';

const client = new MemoryClient({
  baseURL: 'http://localhost:8125/v3',
  apiKey: process.env.MEMORY_UPSTREAM_API_KEY
});

await client.chatMemory.create({
  messages: [{ role: 'user', content: 'Hello' }]
});

```

## Architecture and Source Code References

The routing decision logic resides in multiple coordinated components. In [`MemoryProxy/src/handler.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryProxy/src/handler.ts), the proxy evaluates incoming requests against its configuration before forwarding them to the URL specified in `PROXY_UPSTREAM_URL`.

The environment resolution happens at runtime in [`deploy/panel-knowledge-combined/start-combined.sh`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/deploy/panel-knowledge-combined/start-combined.sh). Lines 82-96 explicitly export `LLM_MODE` with a default value of `proxy`, while lines 26-40 conditionally inject the `proxy_endpoint` field into the panel's metadata JSON only when `REMOTE_INSTANCE_PROXY_URL` is non-empty.

For API specifications, consult [`MemoryProxy/v3-api-memoryproxy-doc.md`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryProxy/v3-api-memoryproxy-doc.md) when implementing proxy-mode clients, and [`MemoryCore/v3-api-memorycore-doc.md`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/v3-api-memorycore-doc.md) for BYO-mode direct gateway integrations. These documents define the `/v3` namespace endpoints that both routing modes ultimately expose to agent applications.

## Summary

- **Proxy Mode** (`LLM_MODE=proxy`) is the default configuration that routes agent requests through `tdai-proxy` at port 8096, enabling centralized authentication, context injection, and cost control.
- **BYO Mode** (`LLM_MODE=custom`) bypasses the proxy for direct LLM calls using `MEMORY_UPSTREAM_*` variables, reducing latency but losing proxy-layer features.
- The `REMOTE_INSTANCE_PROXY_URL` environment variable determines whether the Panel UI exposes a proxy endpoint or falls back to the direct gateway address.
- Configuration changes take effect by restarting the stack via [`deploy/global-images/start-all.sh`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/deploy/global-images/start-all.sh), which enforces the correct service topology based on `LLM_MODE`.

## Frequently Asked Questions

### What is the default LLM routing mode in TencentDB Agent Memory?

**Proxy mode is the default.** According to lines 82-96 of [`deploy/panel-knowledge-combined/start-combined.sh`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/deploy/panel-knowledge-combined/start-combined.sh), the `LLM_MODE` variable defaults to `proxy` if not explicitly set, causing all agent traffic to route through the Memory Proxy service at port 8096.

### Can I use different LLM providers for the proxy and the memory core?

**Yes.** The `PROXY_UPSTREAM_*` and `MEMORY_UPSTREAM_*` variable groups are independent. You can configure the proxy to call OpenAI (`PROXY_UPSTREAM_URL=https://api.openai.com/v1`) while the memory core retains different credentials for internal operations, though typically these reference the same provider unless implementing complex failover scenarios.

### How do I verify which routing mode is currently active?

**Check the Panel UI or environment variables.** Navigate to `http://localhost:8125` and inspect the team's client access address card. If it displays the value of `REMOTE_INSTANCE_PROXY_URL`, proxy mode is active. If it shows a gateway address, BYO mode is enabled. Alternatively, inspect the running containers: active proxy mode requires the `tdai-proxy` container to be healthy on port 8096.

### Does BYO mode support proxy-specific features like session initialization?

**No.** BYO mode bypasses [`MemoryProxy/src/handler.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryProxy/src/handler.ts) entirely, which means features like automatic session initialization, authentication enforcement, and context asset injection managed by the proxy layer are not applied. When using `LLM_MODE=custom`, you must implement these concerns in your client application or upstream LLM gateway.