How to Configure Standalone LLM Overrides in TDAI Core

You can configure standalone LLM overrides in TDAI Core by editing the tdai-gateway.standalone.yaml file, injecting secrets via environment variables, or passing command-line flags that take precedence over file settings.

The TencentDB-Agent-Memory repository provides TDAI Core with flexible configuration mechanisms for standalone deployments. When running outside of managed clusters, the gateway reads its LLM parameters from a central YAML configuration file, but also supports secure secret injection and runtime overrides through environment variables and CLI arguments.

Understanding the Standalone Configuration Architecture

In standalone mode, the gateway initializes its LLM client using settings defined under the top-level llm key in MemoryCore/tdai-gateway.standalone.yaml. According to the source code in MemoryCore/src/config.ts, the system merges configuration from three sources in the following priority order: CLI arguments override environment variables, which override YAML file values. The final configuration is constructed by the buildConfig function before the gateway starts accepting requests.

Three Methods to Override LLM Settings

YAML File Configuration (Persistent)

The primary configuration resides in MemoryCore/tdai-gateway.standalone.yaml. This file defines default values for baseUrl, apiKey, model, maxTokens, and timeoutMs. Values set here persist across restarts and are ideal for stable deployment environments.

Environment Variable Injection (Secure)

Sensitive values like API keys should use placeholder syntax in the YAML file, such as ${TDAI_LLM_API_KEY}. At startup, the env-config utility processes these placeholders through the getEnv function in MemoryCore/src/utils/env.ts, substituting them with actual values from the shell environment. This approach prevents secrets from being committed to version control.

CLI Argument Overrides (Dynamic)

For temporary changes or CI/CD pipelines, pass command-line flags directly to the gateway entry point in MemoryCore/src/cli/index.ts. Supported flags include --llm-baseUrl, --llm-apiKey, --llm-model, --llm-maxTokens, and --llm-timeoutMs. These arguments are merged last by buildConfig, giving them the highest priority.

Step-by-Step Implementation Guide

Step 1: Modify the Standalone YAML Configuration

Open MemoryCore/tdai-gateway.standalone.yaml and locate the llm block. Define your default parameters, using placeholders for sensitive data:

# MemoryCore/tdai-gateway.standalone.yaml

llm:
  baseUrl: "https://api.openai.com/v1"
  apiKey: "${TDAI_LLM_API_KEY}"      # Injected from environment at runtime

  model: "gpt-4o"
  maxTokens: 4096
  timeoutMs: 120000

Only specify the fields you wish to change; omitted keys retain their system defaults.

Step 2: Export Required Environment Variables

Before launching the gateway, export any variables referenced in your YAML placeholders. The getEnv utility in MemoryCore/src/utils/env.ts reads these at startup:

export TDAI_LLM_API_KEY="sk-XXXXXXXXXXXXXXXX"

Step 3: Apply Runtime CLI Overrides (Optional)

When launching the gateway, append any --llm-* flag to override specific settings without editing files. This is useful for A/B testing models or adjusting timeouts for specific jobs:

node dist/index.js \
  --configFile MemoryCore/tdai-gateway.standalone.yaml \
  --llm-model "gpt-4-turbo" \
  --llm-maxTokens 8192 \
  --llm-timeoutMs 60000

The buildConfig logic in MemoryCore/src/config.ts processes these flags after loading the YAML and environment values, ensuring CLI arguments take precedence.

Step 4: Restart the Gateway Process

After modifying configuration files or exporting new environment variables, restart the gateway to apply changes:

npm run start:standalone

# Or using the compiled binary

node MemoryCore/bin/tdai-gateway.mjs \
  --configFile MemoryCore/tdai-gateway.standalone.yaml

The gateway will log the effective LLM configuration during startup, allowing you to verify that overrides were applied correctly.

Critical Source Files and Functions

• MemoryCore/tdai-gateway.standalone.yaml – Default standalone configuration containing the llm schema.
• MemoryCore/src/utils/env.ts – Implements the getEnv function for secure environment variable resolution.
• MemoryCore/src/config.ts – Contains the buildConfig function that merges YAML, environment, and CLI sources.
• MemoryCore/src/cli/index.ts – Entry point that parses --llm-* command-line flags and passes them to the configuration builder.

Summary

• Persistent settings belong in tdai-gateway.standalone.yaml under the llm key.
• Secrets should use ${VAR} placeholders resolved by getEnv in src/utils/env.ts to avoid hardcoding credentials.
• Temporary overrides leverage CLI flags processed by src/cli/index.ts and merged via buildConfig in src/config.ts.
• Always restart the gateway after configuration changes to ensure the new LLM parameters take effect.

Frequently Asked Questions

How does TDAI Core prioritize conflicting configuration values?

TDAI Core uses a cascading priority system implemented in MemoryCore/src/config.ts. The buildConfig function applies values in this order: CLI arguments override environment variables, which override YAML file settings. This ensures that a --llm-model flag passed on the command line will supersede the model defined in tdai-gateway.standalone.yaml, even if both are present.

Can I use environment variables without placeholders in the YAML file?

While the recommended approach uses ${VAR} placeholders, you can technically reference environment variables directly if the application logic supports it. However, the env-config utility specifically looks for placeholder patterns in the YAML and substitutes them using the getEnv function from src/utils/env.ts. Using explicit placeholders ensures compatibility with the standard configuration loader and improves security auditing.

What CLI flags are available for LLM overrides?

The gateway accepts five specific flags for LLM configuration: --llm-baseUrl, --llm-apiKey, --llm-model, --llm-maxTokens, and --llm-timeoutMs. These flags are defined in MemoryCore/src/cli/index.ts and map directly to the corresponding fields in the llm configuration object. Each flag accepts a string or numeric value depending on the parameter type.

Is it possible to configure LLM settings without restarting the gateway?

No, TDAI Core reads configuration once at startup during the initialization phase in src/config.ts. Changes to tdai-gateway.standalone.yaml, environment variables, or CLI arguments require a process restart to take effect. For dynamic model switching without restarts, you would need to implement a custom configuration reload mechanism outside the standard standalone deployment mode.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}