SQLAlchemy Connection Pooling Settings for High-Concurrency Workloads in LMForge

The LMForge platform configures SQLAlchemy with a default QueuePool size of 30 connections and a 3600-second recycle timeout, exposed through environment variables defined in api/config/default_config.py and loaded via api/config/config.py.

The LMForge end-to-end LLMOps platform for multi-model agents relies on robust database connectivity to handle concurrent requests from multiple AI agents and user sessions. Understanding the SQLAlchemy connection pooling configuration is essential for maintaining performance under high-concurrency workloads. The platform centralizes these settings in its configuration module, allowing operators to tune pool behavior without modifying application code.

Default Connection Pool Configuration

The LMForge platform uses SQLAlchemy’s QueuePool implementation by default, which maintains a pool of persistent database connections to minimize connection overhead. These defaults are hardcoded in api/config/default_config.py to provide immediate out-of-the-box performance for production deployments.

The standard configuration includes:

• SQLALCHEMY_POOL_SIZE: Set to 30 persistent connections
• SQLALCHEMY_POOL_RECYCLE: Set to 3600 seconds (1 hour)

These values establish a baseline suitable for moderate to high-throughput scenarios, balancing resource utilization against connection acquisition latency.

Environment Variable Overrides

Rather than requiring code changes to adjust pool behavior, LMForge reads connection parameters from environment variables through the api/config/config.py module. This module constructs the SQLALCHEMY_ENGINE_OPTIONS dictionary that gets passed directly to SQLAlchemy’s engine creator:

# api/config/config.py (excerpt)

self.SQLALCHEMY_ENGINE_OPTIONS = {
    "pool_size": int(_get_env("SQLALCHEMY_POOL_SIZE")),          # ← pool size

    "pool_recycle": int(_get_env("SQLALCHEMY_POOL_RECYCLE")),    # ← recycle timeout

}

This approach enables dynamic tuning across different deployment environments—development, staging, and production—without rebuilding container images or redeploying application code.

Tuning for High-Concurrency Workloads

When scaling LMForge to handle hundreds or thousands of simultaneous agent interactions, adjusting the connection pool parameters becomes critical for preventing database bottlenecks.

Adjusting Pool Size

The pool_size parameter controls the maximum number of connections maintained in the pool. For high-concurrency workloads:

• Increase the value when observing connection wait times or pool exhaustion errors
• Match the value to your database server’s maximum connection limit divided by the number of LMForge application instances
• Monitor active connections using database administration tools to identify saturation points

Increasing this value from the default 30 to 50, 80, or higher allows more worker processes and threads to hold simultaneous database sessions, reducing latency during traffic spikes.

Connection Recycling Strategy

The pool_recycle parameter prevents stale connections by automatically closing and replacing connections after a specified idle duration. The default 3600-second (1 hour) setting aligns with typical PostgreSQL idle_timeout configurations.

For environments with aggressive firewall rules or database-side connection limits:

• Reduce the recycle timeout to 1800 seconds (30 minutes) or 900 seconds (15 minutes) to force periodic connection renewal
• Consider network infrastructure timeouts when setting this value to prevent mid-transaction disconnections

This proactive recycling prevents "OperationalError" exceptions caused by databases or middleware dropping idle connections after long periods of inactivity.

Code Examples

Docker Compose Configuration

Override defaults in container orchestration manifests to match your infrastructure capacity:

services:
  api:
    image: lmforge/api:latest
    environment:
      - SQLALCHEMY_DATABASE_URI=postgresql://user:pass@db:5432/lmforge
      - SQLALCHEMY_POOL_SIZE=50          # increase for higher concurrency

      - SQLALCHEMY_POOL_RECYCLE=1800     # recycle after 30 min if preferred

Programmatic Configuration

For testing scenarios or custom deployment scripts, modify the configuration object directly before engine instantiation:

from api.config.config import Config

# Load default config

cfg = Config()

# Override pool settings on the fly

cfg.SQLALCHEMY_ENGINE_OPTIONS.update({
    "pool_size": 80,          # very high concurrency

    "pool_recycle": 7200,     # 2 hours

})

# Create the engine (example with SQLAlchemy Core)

from sqlalchemy import create_engine
engine = create_engine(
    cfg.SQLALCHEMY_DATABASE_URI,
    **cfg.SQLALCHEMY_ENGINE_OPTIONS,
    echo=cfg.SQLALCHEMY_ECHO,
)

Summary

• Default QueuePool settings in LMForge specify 30 connections with 3600-second recycling, defined in api/config/default_config.py
• Environment-driven configuration via SQLALCHEMY_POOL_SIZE and SQLALCHEMY_POOL_RECYCLE variables enables deployment-specific tuning without code changes
• High-concurrency optimization requires increasing pool_size to match expected simultaneous request volume while ensuring database server capacity limits are respected
• Connection recycling at appropriate intervals prevents stale connection errors in long-running LLMOps platform instances

Frequently Asked Questions

How does increasing the SQLALCHEMY_POOL_SIZE improve high-concurrency performance?

A larger pool size allows more worker threads and processes to maintain active database connections simultaneously. When the LMForge platform handles concurrent requests from multiple AI agents, each request typically requires a database session. Without sufficient pooled connections, requests queue up waiting for available connections, increasing latency. Increasing the pool size from the default 30 to 50 or 80 reduces this contention, though the value must remain below your database server's maximum connection limit.

What is the purpose of the SQLALCHEMY_POOL_RECYCLE setting?

The pool recycle setting automatically closes and replaces connections after a specified number of seconds of idle time. This prevents errors caused by databases, firewalls, or middleware dropping idle connections after timeout periods. The default 3600-second (1 hour) value in LMForge aligns with common PostgreSQL configurations, but you should reduce this value if your infrastructure enforces shorter idle timeouts or if you encounter "connection already closed" errors during long-running operations.

Can I override connection pool settings without modifying the source code?

Yes. The LMForge platform loads these values from environment variables in api/config/config.py, allowing you to adjust SQLALCHEMY_POOL_SIZE and SQLALCHEMY_POOL_RECYCLE through Docker Compose files, Kubernetes manifests, or shell environment exports. This externalized configuration approach supports the twelve-factor app methodology and enables different tuning for development, staging, and production environments using the same container image.

Where are the default connection pool values defined in the repository?

The default values are defined in api/config/default_config.py, which serves as the authoritative source for all configuration defaults including the 30-connection pool size and 3600-second recycle timeout. These values are then read and processed in api/config/config.py, which constructs the final SQLALCHEMY_ENGINE_OPTIONS dictionary passed to SQLAlchemy's engine creation function.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}