What LLM Providers Are Used in the Context Ablation Experiment? A Complete Provider Guide

The context ablation experiment in the bojieli/ai-agent-book repository supports nine distinct LLM providers: dashscope, qwen, bailian, kimi, moonshot, doubao, siliconflow, deepseek, and openrouter.

The context ablation experiment defined in chapter1/context/run_experiment_1_1.py evaluates how different context windows affect agent performance across multiple model providers. This flexibility allows researchers to benchmark results against various cloud APIs and open-source endpoints using a unified interface. The provider configuration is handled through a central KEY_ENV dictionary that maps each service to its required authentication credentials.

Supported LLM Providers in the Context Ablation Experiment

The experiment supports nine providers through the KEY_ENV dictionary defined at lines 42–51 of chapter1/context/run_experiment_1_1.py. Each entry specifies the environment variable(s) required for authentication.

DashScope Ecosystem (Alibaba Cloud)

The dashscope, qwen, and bailian providers all route through Alibaba Cloud's DashScope platform and share the same authentication mechanism:

• dashscope → DASHSCOPE_API_KEY
• qwen → DASHSCOPE_API_KEY (alias for Qwen models)
• bailian → DASHSCOPE_API_KEY (alias for Bailian platform)

These aliases allow you to specify models using either the platform name (--provider dashscope) or the model family name (--provider qwen).

Moonshot and Kimi Providers

ByteDance's Moonshot API is accessible through two synonymous provider names:

• kimi → MOONSHOT_API_KEY or KIMI_API_KEY
• moonshot → MOONSHOT_API_KEY or KIMI_API_KEY

The experiment uses kimi as the default provider when no --provider argument is specified. Both environment variables are checked to accommodate different user configurations.

Additional Cloud Providers

The experiment integrates with four other major API providers:

• doubao → ARK_API_KEY (ByteDance Ark platform)
• siliconflow → SILICONFLOW_API_KEY (SiliconFlow inference API)
• deepseek → DEEPSEEK_API_KEY (DeepSeek AI API)
• openrouter → OPENROUTER_API_KEY (OpenRouter unified endpoint)

How Provider Selection Works in the Code

In chapter1/context/run_experiment_1_1.py, the KEY_ENV dictionary (lines 42–51) serves as the source of truth for credential resolution. When you invoke the script with --provider <name>, the experiment:

1. Validates the provider name against KEY_ENV.keys()
2. Retrieves the corresponding environment variable name(s)
3. Loads the API key from your shell environment
4. Initializes the appropriate client via agentbook/providers/registry.py

The default value is hardcoded as kimi, meaning python run_experiment_1_1.py executes equivalently to python run_experiment_1_1.py --provider kimi.

Running the Experiment with Different Providers

You can execute the context ablation study against any supported backend by setting the appropriate environment variable and passing the --provider flag.

Environment Setup

Configure your API keys before running the experiment:

export DASHSCOPE_API_KEY="your_dashscope_key"
export MOONSHOT_API_KEY="your_moonshot_key"
export OPENROUTER_API_KEY="your_openrouter_key"
export DEEPSEEK_API_KEY="your_deepseek_key"

Execution Examples

Run with the default kimi provider:

python chapter1/context/run_experiment_1_1.py

Specify openrouter to test against GPT-4o or other models:

python chapter1/context/run_experiment_1_1.py --provider openrouter --model gpt-4o

Use qwen (via DashScope) for Alibaba's Qwen 2.5 models:

python chapter1/context/run_experiment_1_1.py --provider qwen --model qwen-2.5

The script automatically resolves the correct API endpoint and authentication headers based on the provider selected, delegating the actual HTTP implementation to the corresponding module in agentbook/providers/ (such as agentbook/providers/openrouter.py for OpenRouter-specific logic).

Summary

• The context ablation experiment supports nine LLM providers: dashscope, qwen, bailian, kimi, moonshot, doubao, siliconflow, deepseek, and openrouter.
• Provider credentials are mapped in the KEY_ENV dictionary at lines 42–51 of chapter1/context/run_experiment_1_1.py.
• Kimi is the default provider, though any supported provider can be selected via the --provider command-line flag.
• Each provider requires a specific environment variable (e.g., DEEPSEEK_API_KEY, SILICONFLOW_API_KEY) to authenticate requests.
• The provider registry in agentbook/providers/registry.py and individual provider implementations (like agentbook/providers/openrouter.py) handle the underlying API communication.

Frequently Asked Questions

How do I switch from the default Kimi provider to DeepSeek in the context ablation experiment?

Set the DEEPSEEK_API_KEY environment variable in your shell, then run the experiment with the --provider flag: python chapter1/context/run_experiment_1_1.py --provider deepseek. The script will automatically locate your API key using the KEY_ENV mapping and route requests to DeepSeek's API endpoint.

Why do qwen, bailian, and dashscope providers all use the same environment variable?

These are aliases for the same underlying Alibaba Cloud DashScope platform. The KEY_ENV dictionary maps all three provider names to DASHSCOPE_API_KEY (lines 42–51 of run_experiment_1_1.py), allowing you to use semantic naming (--provider qwen) while maintaining a single credential source for the entire DashScope ecosystem.

What happens if I don't set the required environment variable for my chosen provider?

The experiment will fail to initialize the LLM client and raise a KeyError or authentication error when attempting to access the mapped environment variable from KEY_ENV. Ensure you export the correct variable (e.g., export MOONSHOT_API_KEY=...) before executing the script, as the code does not provide fallback mechanisms for missing credentials.

Can I use multiple providers in a single experiment run?

No, the run_experiment_1_1.py script accepts only one --provider argument per execution. To compare context ablation results across multiple providers, you must run the experiment separately for each provider and aggregate the results manually, or modify the experiment loop to iterate over a list of providers in the source code.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}