# How LMForge Handles Data Persistence and Backups with Docker Volumes

> Discover how LMForge ensures data persistence and backups using Docker volumes. Learn how host directories are mounted to safeguard your LLM agent data across restarts and simplify backups.

- Repository: [Haohao/lmforge-end-to-end-llmops-platform-for-multi-model-agents](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents)
- Tags: how-to-guide
- Published: 2026-03-03

---

**LMForge mounts host directories as Docker volumes for every stateful component, ensuring data survives container restarts while enabling simple filesystem-based backups.**

LMForge is an end-to-end LLMOps platform for multi-model agents that relies entirely on Docker volumes for data persistence. By mapping host-side directories into containers, the platform guarantees that PostgreSQL databases, Redis caches, Weaviate vector stores, and user-uploaded datasets remain intact even when containers are destroyed or upgraded.

## Docker Volume Architecture in LMForge

The persistence strategy is defined in [`docker/docker-compose.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/docker/docker-compose.yaml), where every service mounts specific host paths to container paths. This approach separates the application lifecycle from data lifecycle, allowing operators to back up, migrate, or restore individual components without touching container images.

### Component Volume Mapping

LMForge allocates dedicated directories under `./volumes/` for each persistent service:

| Component | Host Directory | Container Path | Stored Data |
|-----------|----------------|----------------|-------------|
| **API** | `./volumes/app/storage` | `/app/api/storage` | Dataset files, uploaded assets, temporary processing files |
| **PostgreSQL** | `./volumes/db/data` | `/var/lib/postgresql/data/pgdata` | Relational tables, schema migrations, transaction logs |
| **Redis** | `./volumes/redis/data` | `/data` | Cache snapshots and AOF persistence files |
| **Weaviate** | `./volumes/weaviate` | `/var/lib/weaviate` | Vector indexes and metadata stores |

These mounts ensure that when you run `docker compose down`, the data remains in `./volumes/` on the host filesystem, ready for the next `docker compose up`.

## Log Rotation and Retention Policies

Beyond database persistence, LMForge implements automatic log retention to prevent disk exhaustion. The API service writes logs to a dedicated storage directory that is created at runtime if missing.

### Automatic Log Backup Configuration

In [`api/internal/extension/logging_extension.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/internal/extension/logging_extension.py), the application initializes a `ConcurrentTimedRotatingFileHandler` that rotates logs at midnight and maintains a 30-day history:

```python
import os
import logging
from concurrent_log_handler import ConcurrentTimedRotatingFileHandler

def setup_logging():
    log_folder = os.path.join(os.getcwd(), "storage", "log")
    if not os.path.exists(log_folder):
        os.makedirs(log_folder)  # Runtime directory creation

    
    log_file = os.path.join(log_folder, "app.log")
    handler = ConcurrentTimedRotatingFileHandler(
        log_file,
        when='midnight',
        interval=1,
        backupCount=30,  # Retain 30 days of logs

        encoding='utf-8'
    )
    
    formatter = logging.Formatter(
        "[%(asctime)s.%(msecs)03d] %(filename)s -> %(funcName)s line:%(lineno)d [%(levelname)s]: %(message)s"
    )
    handler.setFormatter(formatter)
    
    logger = logging.getLogger()
    logger.setLevel(logging.DEBUG)
    logger.addHandler(handler)

```

The `backupCount=30` parameter implements a retention policy without requiring external cron jobs or cleanup scripts. When the 31st daily log file is created, the oldest is automatically deleted.

## Backup Strategies for LMForge Docker Volumes

Because LMForge stores all stateful data in host-mounted directories under `./volumes/`, you can implement backups using standard filesystem tools without requiring database-specific dump commands.

### Filesystem Snapshots

The simplest approach archives the entire volumes directory while containers are running (hot backup), since Docker volume mounts are atomic at the filesystem level:

```bash
#!/usr/bin/env bash
set -euo pipefail

BACKUP_ROOT="${PWD}/backup"
mkdir -p "$BACKUP_ROOT"

# Create timestamped archives of each volume

tar czf "$BACKUP_ROOT/pg_$(date +%F).tar.gz" -C ./volumes/db data
tar czf "$BACKUP_ROOT/redis_$(date +%F).tar.gz" -C ./volumes/redis data
tar czf "$BACKUP_ROOT/storage_$(date +%F).tar.gz" -C ./volumes/app storage
tar czf "$BACKUP_ROOT/weaviate_$(date +%F).tar.gz" -C ./volumes/weaviate .

```

### Docker Native Commands

For point-in-time consistency, you can use `docker cp` to extract data from running containers without direct host path access:

```bash

# Copy PostgreSQL data directory from container to host backup location

docker cp llmops-db:/var/lib/postgresql/data/pgdata ./backup/pgdata-$(date +%F)

# Copy Redis persistence files

docker cp llmops-redis:/data ./backup/redis-$(date +%F)

```

### Volume-Level Backup Tools

Production deployments often use specialized tools that handle Docker volumes natively:

- **Restic** or **Borg**: Point at `./volumes/` for deduplicated, encrypted backups
- **Volume plugins**: Use `vieux/sshfs` or `local-persist` driver for network-attached storage
- **ZFS/Btrfs snapshots**: If the host filesystem supports copy-on-write, snapshot the `./volumes` dataset instantly

## Key Configuration Files

Understanding these files helps you customize persistence behavior:

| File | Purpose |
|------|---------|
| **[`docker/docker-compose.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/docker/docker-compose.yaml)** | Declares all volume mounts for PostgreSQL, Redis, Weaviate, and API storage |
| **[`api/internal/extension/logging_extension.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/internal/extension/logging_extension.py)** | Configures `ConcurrentTimedRotatingFileHandler` with 30-day retention |
| **`api/.dockerignore`** | Excludes `storage/log` and temporary files from the Docker image build, ensuring only runtime data persists in volumes |
| **[`docker/postgres/init.sql`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/docker/postgres/init.sql)** | Runs only on fresh volumes; skipped if `./volumes/db/data` already contains data, protecting existing databases during redeploys |

## Summary

- **LMForge uses Docker volumes** mapped to `./volumes/` subdirectories to persist PostgreSQL, Redis, Weaviate, and user-uploaded data across container restarts.
- **Volume definitions** live in [`docker/docker-compose.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/docker/docker-compose.yaml), binding host paths like `./volumes/db/data` to container paths like `/var/lib/postgresql/data/pgdata`.
- **Log retention** is handled automatically by `ConcurrentTimedRotatingFileHandler` in [`api/internal/extension/logging_extension.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/internal/extension/logging_extension.py), keeping 30 days of rotated logs without manual cleanup.
- **Backups are filesystem-native**—you can archive `./volumes/` using `tar`, `docker cp`, or tools like Restic, since all state exists on the host filesystem.

## Frequently Asked Questions

### Where does LMForge store PostgreSQL data?

LMForge stores PostgreSQL data in `./volumes/db/data` on the host, which is mounted to `/var/lib/postgresql/data/pgdata` inside the container. This mapping is defined in [`docker/docker-compose.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/docker/docker-compose.yaml) and ensures that database files survive container recreation.

### How long does LMForge retain application logs?

The platform retains application logs for **30 days**. The `ConcurrentTimedRotatingFileHandler` configured in [`api/internal/extension/logging_extension.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/internal/extension/logging_extension.py) rotates logs daily at midnight and automatically deletes files older than the 30-day `backupCount` limit.

### Can I back up LMForge volumes while containers are running?

Yes. Because LMForge uses bind mounts to host directories rather than Docker's internal volume driver, you can safely create hot backups using `tar`, `rsync`, or snapshot tools while containers remain running. The filesystem handles consistency for the mounted directories.

### What happens to data during a container restart?

Data persists completely. Since all stateful information resides in host-mounted volumes under `./volumes/`, restarting or recreating containers with `docker compose down && docker compose up` does not affect the stored data. PostgreSQL, Redis, Weaviate, and uploaded files remain intact.