Environment Variables Harvey-Labs Supports and How .env Auto-Loading Works

Harvey-labs uses python-dotenv to automatically load environment variables from a .env file at package import time, making configuration available via os.getenv() throughout the codebase without requiring manual loading calls.

Harvey Labs, an open-source framework for LLM evaluation and benchmarking, relies on environment variables to authenticate with external AI providers and configure runtime behavior. The repository implements automatic .env file loading through python-dotenv, ensuring that variables defined in your project root are immediately available to all adapters and scripts. Understanding which environment variables harvey-labs supports and how the auto-loading mechanism works is essential for configuring API keys and system settings securely.

Core Environment Variables Supported by Harvey-Labs

The codebase recognizes specific variables for LLM provider authentication and pipeline configuration. These are read throughout the harness using os.getenv() with sensible defaults applied when values are absent.

LLM Provider API Keys

The following variables authenticate requests to external model providers. While optional from the code perspective, they are required for production runs that contact the respective services:

• OPENAI_API_KEY – Authentication token for the OpenAI API, consumed in harness/adapters/openai.py
• OPENAI_ORG_ID – Optional organization identifier for OpenAI billing and resource management
• MISTRAL_API_KEY – Token for the Mistral API, used in harness/adapters/mistral.py
• GOOGLE_API_KEY – API key for Google’s Generative AI endpoints, read in harness/adapters/google.py
• FIREWORKS_API_KEY – Authentication token for the Fireworks AI platform
• BASE_TEN_API_KEY – Token used by the Baseten adapter in harness/adapters/baseten.py

System Configuration Variables

General runtime behavior is controlled through the HARVEY_ prefixed variables:

• HARVEY_LOG_LEVEL – Controls logging verbosity (e.g., DEBUG, INFO, WARNING)
• HARVEY_DATA_ROOT – Directory path where data files, caches, and model checkpoints are stored
• HARVEY_MAX_WORKERS – Maximum number of parallel worker processes used by the pipeline
• HARVEY_TIMEOUT_SECONDS – Global request timeout for external API calls

How .env Auto-Loading Works in Harvey-Labs

The automatic loading mechanism is implemented in the package initialization file, ensuring environment variables are populated before any adapter or script executes.

The Bootstrap Mechanism in harvey_labs/__init__.py

When any module imports harvey_labs (or the top-level harness package), the following bootstrap routine executes immediately:

# harvey_labs/__init__.py

import os
import dotenv

# Load .env if it exists; silently ignore if absent

dotenv.load_dotenv()  # Injects KEY=VALUE pairs into os.environ

The dotenv.load_dotenv() call searches for a file named .env in the current working directory (typically the repository root) and injects every key-value pair into the process environment (os.environ). Because this executes at import time, subsequent code can access variables directly without additional loading steps.

Adapter-Level Variable Consumption

After the bootstrap runs, individual adapters retrieve credentials using os.getenv(). For example, in harness/adapters/openai.py, the code accesses the OpenAI configuration as follows:

# harness/adapters/openai.py

import os
import openai

# Variables are already loaded by harvey_labs/__init__.py

openai.api_key = os.getenv("OPENAI_API_KEY")

# Optionally configure organization

org = os.getenv("OPENAI_ORG_ID")
if org:
    openai.organization = org

This pattern repeats across harness/adapters/mistral.py, harness/adapters/google.py, and other provider adapters, each calling os.getenv() to retrieve their respective API keys.

Practical Configuration Examples

Creating and Locating Your .env File

Place a .env file at the repository root with the required key-value pairs:

# .env file (located at repository root)

OPENAI_API_KEY=sk-********************************
OPENAI_ORG_ID=org-abc123
HARVEY_LOG_LEVEL=DEBUG
HARVEY_MAX_WORKERS=8
HARVEY_DATA_ROOT=/path/to/data

When you run entry points like scripts/run_model_sweep.py, the variables are automatically available:

python -m scripts.run_model_sweep --config configs/experiment.yaml

Accessing Variables in Custom Scripts

If you write custom scripts outside the package import hierarchy, you can explicitly load the .env file:

# my_script.py

import os
from dotenv import load_dotenv

# Explicitly load .env – optional if harvey_labs is already imported

load_dotenv()

api_key = os.getenv("OPENAI_API_KEY")
print(f"Configured OpenAI key starts with: {api_key[:4]}...")

Summary

• Harvey-labs supports authentication variables for OpenAI, Mistral, Google, Fireworks, and Baseten, plus system variables prefixed with HARVEY_.
• Auto-loading occurs in harvey_labs/__init__.py via dotenv.load_dotenv(), which runs automatically when the package is imported.
• Variable access happens through os.getenv() in adapter files like harness/adapters/openai.py, with sensible defaults when variables are missing.
• All supported environment variables are technically optional, though API keys are required for actual provider calls.

Frequently Asked Questions

What happens if the .env file is missing?

The bootstrap code in harvey_labs/__init__.py calls dotenv.load_dotenv() without raising an error if the file is absent. The application continues using default values or None for os.getenv() calls, though API calls will fail if required keys are not provided through shell exports.

Can I override .env variables using shell exports?

Yes. Environment variables defined in your shell session take precedence over values in the .env file. The python-dotenv loader never overwrites existing environment variables, so exporting OPENAI_API_KEY before running a script overrides the .env configuration.

Is OPENAI_ORG_ID required to use the OpenAI adapter?

No. The OPENAI_ORG_ID variable is optional. The adapter in harness/adapters/openai.py checks for its presence with if org: before setting openai.organization, allowing the code to function with only OPENAI_API_KEY defined.

How do I change the logging level in harvey-labs?

Set the HARVEY_LOG_LEVEL environment variable in your .env file or shell export to values like DEBUG, INFO, or WARNING. The codebase reads this variable during initialization to configure the logging infrastructure accordingly.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}