How to Use the Multi-Model Query Feature in GPT Academic: Query Multiple LLMs Simultaneously

The multi-model query feature in GPT Academic dispatches a single user prompt to multiple LLM back-ends simultaneously by splitting the MULTI_QUERY_LLM_MODELS configuration string on & and spawning parallel threads for each model.

The multi-model query feature allows you to compare responses from different language models—such as GPT-4, Claude, and local models—in a single request. This capability is implemented as a plugin in the binary-husky/gpt_academic repository and leverages a multi-threaded dispatcher to parallelize API calls.

How the Multi-Model Query Feature Works

The architecture consists of three layers: UI routing, configuration management, and parallel execution.

UI Routing and Plugin Selection

When you select "多模型对话" (Multi-Model Dialogue) from the multiplex dropdown in the Gradio interface, the UI maps this selection to the plugin function via get_multiplex_button_functions() in crazy_functional.py:

def get_multiplex_button_functions():
    return {
        "常规对话": "",
        "查互联网后回答": "查互联网后回答",
        "多模型对话": "询问多个GPT模型",  # Maps to Multi_LLM_Query plugin

        "智能召回 RAG": "Rag智能召回",
        "多媒体查询": "多媒体智能体",
    }

This mapping tells the submit handler in main.py to route the request to the Multi_LLM_Query.py plugin instead of the standard chat pipeline.

Configuration and Model Selection

The plugin reads the global configuration variable MULTI_QUERY_LLM_MODELS from config.py. The default value is:

MULTI_QUERY_LLM_MODELS = "gpt-3.5-turbo&chatglm3"

You can override this via:

• Environment variable: export MULTI_QUERY_LLM_MODELS="gpt-4&claude-3-opus"
• Private config: Create config_private.py and set the variable there (takes precedence over config.py)

Multi-Threaded Dispatch

The plugin 同时问询 (simultaneous query) in Multi_LLM_Query.py injects the model list into llm_kwargs and calls request_gpt_model_in_new_thread_with_ui_alive:

def 同时问询(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
    # Inject the multi-model configuration

    llm_kwargs['llm_model'] = get_conf('MULTI_QUERY_LLM_MODELS')
    
    # Dispatch to parallel execution handler

    yield from request_gpt_model_in_new_thread_with_ui_alive(
        inputs=txt,
        llm_kwargs=llm_kwargs,
        chatbot=chatbot,
        history=history,
        sys_prompt=system_prompt
    )

The dispatcher in crazy_utils.py (called by request_gpt_model_in_new_thread_with_ui_alive) splits the llm_model string on & and launches a separate process/thread for each token. Each thread invokes the appropriate bridge module (e.g., request_llms/bridge_chatgpt.py for OpenAI models, request_llms/bridge_chatglm.py for ChatGLM).

Configuring Which Models Are Queried

To customize the multi-model query feature, modify the MULTI_QUERY_LLM_MODELS string. Use & as the delimiter between model identifiers.

Example configurations:

# config.py or config_private.py

MULTI_QUERY_LLM_MODELS = "gpt-4-turbo&claude-3-opus-20240229&glm-4"

Supported model identifiers depend on your configured bridges in request_llms/. Common identifiers include:

• gpt-3.5-turbo, gpt-4, gpt-4-turbo (OpenAI)
• chatglm3, glm-4 (ChatGLM)
• claude-3-opus, claude-3-sonnet (Anthropic, if configured)

Using the Multi-Model Query Feature

Via the Web UI

1. Launch the application: python main.py
2. Enter your prompt in the 输入区 (input area)
3. Click the multiplex dropdown (located to the right of the submit button) and select "多模型对话" (Multi-Model Dialogue)
4. Click 提交 (Submit)
5. The response area will display concatenated results prefixed with model names:

[gpt-3.5-turbo] The answer from GPT-3.5...
[chatglm3] The answer from ChatGLM3...

Programmatically from Python

You can invoke the multi-model query feature from custom scripts using the 同时问询 function:

from toolbox import get_conf, update_ui
from crazy_functions.Multi_LLM_Query import 同时问询

# Prepare arguments

txt = "Explain the architectural differences between transformers and RNNs"
llm_kwargs = {"temperature": 0.7}
plugin_kwargs = {}
chatbot = []      # UI state placeholder

history = []      # Conversation history

system_prompt = get_conf('INIT_SYS_PROMPT')
user_request = {} # Metadata placeholder

# Execute (generator yields UI updates)

gen = 同时问询(txt, llm_kwargs, plugin_kwargs,
               chatbot, history, system_prompt, user_request)

for _ in gen:
    pass  # Consume generator for side effects

# Extract final answer

final_answer = chatbot[-1][1]
print(final_answer)

Runtime Model Override

To query a specific set of models without changing config.py, use the 同时问询_指定模型 (simultaneous query with specified models) variant:

from crazy_functions.Multi_LLM_Query import 同时问询_指定模型

txt = "What are the latest advancements in quantum computing?"
llm_kwargs = {"temperature": 0.5}
plugin_kwargs = {
    "advanced_arg": "gpt-4&claude-3-opus-20240229"  # Override model list

}
chatbot = []
history = []
system_prompt = "You are a helpful assistant."

gen = 同时问询_指定模型(txt, llm_kwargs, plugin_kwargs,
                        chatbot, history, system_prompt, {})

for _ in gen:
    pass

print(chatbot[-1][1])

The advanced_arg parameter overrides the default MULTI_QUERY_LLM_MODELS configuration for that specific request.

Key Source Files and Implementation Details

File	Purpose
crazy_functions/Multi_LLM_Query.py	Implements 同时问询 (default multi-model query) and 同时问询_指定模型 (runtime-specified models). Contains the generator logic that injects model lists into llm_kwargs.
crazy_functional.py	Registers plugins and defines the multiplex button mapping via get_multiplex_button_functions(). Maps "多模型对话" to the Multi_LLM_Query plugin.
config.py	Stores the global MULTI_QUERY_LLM_MODELS configuration (default: "gpt-3.5-turbo&chatglm3").
main.py	Defines the Gradio UI layout, including the multiplex dropdown that triggers the multi-model query feature.
toolbox.py	Provides utility functions like get_conf() for configuration retrieval and update_ui() for chatbot state management.
crazy_utils.py	Contains request_gpt_model_in_new_thread_with_ui_alive, the dispatcher that splits the &-delimited model string and spawns parallel threads.
request_llms/bridge_*.py	Model-specific API bridges (e.g., bridge_chatgpt.py, bridge_chatglm.py) invoked by the dispatcher for each parallel request.

Summary

• The multi-model query feature routes a single prompt to multiple LLMs by splitting the MULTI_QUERY_LLM_MODELS string on & and dispatching parallel threads.
• Configuration is controlled via the MULTI_QUERY_LLM_MODELS variable in config.py or environment variables, defaulting to "gpt-3.5-turbo&chatglm3".
• UI activation requires selecting "多模型对话" from the multiplex dropdown in the Gradio interface defined in main.py.
• Programmatic access is available through the 同时问询 function in crazy_functions/Multi_LLM_Query.py, which yields UI updates and appends concatenated responses to the chatbot history.
• Runtime overrides are supported via the 同时问询_指定模型 variant using the advanced_arg parameter to specify custom model combinations without modifying configuration files.

Frequently Asked Questions

How do I add more models to the multi-model query feature?

Modify the MULTI_QUERY_LLM_MODELS configuration in config.py or set the environment variable. Separate model identifiers with &. For example: "gpt-4&claude-3-opus-20240229&glm-4". Ensure you have configured the corresponding API keys and bridge modules in request_llms/ for each model you add.

Can I use the multi-model query feature without the web interface?

Yes. Import the 同时问询 function from crazy_functions.Multi_LLM_Query and invoke it programmatically. The function is a generator that handles parallel execution via request_gpt_model_in_new_thread_with_ui_alive. You must provide the standard argument tuple including txt, llm_kwargs, chatbot, history, and system_prompt as shown in the programmatic examples above.

What is the difference between 同时问询 and 同时问询_指定模型?

同时问询 (simultaneous query) uses the global MULTI_QUERY_LLM_MODELS configuration defined in config.py to determine which models to query. 同时问询_指定模型 (simultaneous query with specified models) accepts a custom model list via the advanced_arg parameter in plugin_kwargs, allowing runtime specification of models without changing configuration files. Both functions reside in crazy_functions/Multi_LLM_Query.py.

How does the system handle API errors when querying multiple models?

The request_gpt_model_in_new_thread_with_ui_alive dispatcher in crazy_utils.py spawns independent threads or processes for each model. If one model's API fails or times out, the error is caught within that specific thread and typically returned as an error message for that model section, while other parallel requests continue unaffected. The final concatenated result will include the successful responses alongside error notifications for failed models, ensuring partial failures do not block the entire multi-model query.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}