# How to Adjust Max Concurrent Requests in oMLX's Scheduler: 3 Configuration Methods

> Learn to adjust max concurrent requests in oMLX's scheduler. Discover three methods: CLI flag, environment variable, and settings file to effectively configure your oMLX instance.

- Repository: [Jun Kim/omlx](https://github.com/jundot/omlx)
- Tags: how-to-guide
- Published: 2026-05-11

---

**oMLX controls simultaneous request processing through the `max_concurrent_requests` setting stored in `SchedulerSettings` at [`omlx/settings.py`](https://github.com/jundot/omlx/blob/main/omlx/settings.py), which you can override via CLI flag, environment variable, or settings file to directly configure scheduler capacity in [`omlx/scheduler.py`](https://github.com/jundot/omlx/blob/main/omlx/scheduler.py).**

The request scheduler in the jundot/omlx repository manages inference workloads by limiting how many sequences can be processed simultaneously. Adjusting the max concurrent requests in oMLX's scheduler is essential for optimizing throughput on different hardware configurations. This setting defaults to 8 and determines both `max_num_seqs` and `completion_batch_size` in the underlying scheduler configuration.

## Understanding the Scheduler Architecture

### SchedulerSettings Definition

In [`omlx/settings.py`](https://github.com/jundot/omlx/blob/main/omlx/settings.py), the `SchedulerSettings` dataclass defines the configuration boundary for the scheduler. The `max_concurrent_requests` field (line 219) defaults to 8 and serves as the upstream source for all concurrency limits according to the source code.

### SchedulerConfig Application

When the engine initializes, `settings.to_scheduler_config()` converts the value into a `SchedulerConfig` instance defined in [`omlx/scheduler.py`](https://github.com/jundot/omlx/blob/main/omlx/scheduler.py) (lines 362-370). Here, the value populates both `max_num_seqs` and `completion_batch_size` (line 1199), making the scheduler's capacity directly proportional to your configured limit.

## Configuration Methods

You can adjust max concurrent requests in oMLX's scheduler through three hierarchical overrides following the standard precedence: **CLI arguments** > **environment variables** > **settings file** > **defaults**.

### Method 1: Command-Line Flag

The `--max-concurrent-requests` flag provides immediate, session-specific overrides without persisting changes to disk.

```bash
omlx serve --max-concurrent-requests 64

```

### Method 2: Environment Variable

Set `OMLX_MAX_CONCURRENT_REQUESTS` to persist the setting across command invocations without modifying configuration files.

```bash
export OMLX_MAX_CONCURRENT_REQUESTS=128
omlx serve

```

### Method 3: Settings File

For persistent configuration across restarts, edit the `scheduler.max_concurrent_requests` key in your JSON or YAML settings file:

```json
{
  "scheduler": {
    "max_concurrent_requests": 256
  }
}

```

## Configuration Propagation and Validation

The `init_settings()` function processes these overrides in sequence: first parsing CLI arguments, then checking for the environment variable, and finally falling back to the settings file. If you provide an invalid value (≤ 0), validation raises an error and aborts startup before the scheduler initializes.

Internally, the conversion happens in `to_scheduler_config()`:

```python
SchedulerConfig(
    max_num_seqs=settings.scheduler.max_concurrent_requests,
    completion_batch_size=settings.scheduler.max_concurrent_requests,
    # ...

)

```

## Programmatic Access

For testing or custom scripts, you can initialize settings programmatically using `argparse.Namespace` as implemented in the source:

```python
from argparse import Namespace
from omlx.settings import init_settings, get_settings

# Override via fake CLI namespace

init_settings(cli_args=Namespace(max_concurrent_requests=64))

settings = get_settings()
print("Concurrent requests:", settings.scheduler.max_concurrent_requests)  # → 64

```

This approach is validated in [`tests/test_settings.py`](https://github.com/jundot/omlx/blob/main/tests/test_settings.py) and [`tests/test_cli.py`](https://github.com/jundot/omlx/blob/main/tests/test_cli.py), which verify default values, CLI parsing, environment overrides, and input validation.

## Summary

- The max concurrent requests setting resides in `SchedulerSettings` ([`omlx/settings.py`](https://github.com/jundot/omlx/blob/main/omlx/settings.py)) and defaults to **8**
- It propagates to `SchedulerConfig` in [`omlx/scheduler.py`](https://github.com/jundot/omlx/blob/main/omlx/scheduler.py) to control `max_num_seqs` and `completion_batch_size`
- Override via `--max-concurrent-requests`, `OMLX_MAX_CONCURRENT_REQUESTS`, or the `scheduler.max_concurrent_requests` key in your settings file
- Values ≤ 0 trigger validation errors during `init_settings()` and prevent startup
- Programmatic configuration is available via `init_settings()` with `argparse.Namespace`

## Frequently Asked Questions

### What is the default max concurrent requests value in oMLX?

The default value is **8** concurrent requests, defined in the `SchedulerSettings` dataclass at line 219 of [`omlx/settings.py`](https://github.com/jundot/omlx/blob/main/omlx/settings.py). This conservative default works for most consumer GPUs but may need increasing for server-class hardware with larger VRAM capacity.

### Where does the max concurrent requests setting get applied in the scheduler?

The setting transfers from `SchedulerSettings` to `SchedulerConfig` in [`omlx/scheduler.py`](https://github.com/jundot/omlx/blob/main/omlx/scheduler.py) via the `to_scheduler_config()` method. It populates both `max_num_seqs` and `completion_batch_size` at line 1199, directly determining how many sequences the scheduler can process simultaneously during inference.

### Can I set max concurrent requests through multiple methods simultaneously?

Yes, oMLX follows a strict configuration hierarchy. CLI flags take precedence over environment variables, which override settings files, which finally fall back to the default value of 8. You can define a base value in your settings JSON and temporarily override it with environment variables or command-line flags for specific sessions.

### What happens if I set max concurrent requests to zero or a negative number?

The `init_settings()` function validates the input and raises a configuration error before the engine starts, as validated in [`tests/test_settings.py`](https://github.com/jundot/omlx/blob/main/tests/test_settings.py). The scheduler requires at least one concurrent request to function, so invalid values (≤ 0) abort startup immediately with a validation error.