# What Is the Role of Celery and Redis in Securo? Architecture and Implementation Guide

> Discover how Celery and Redis power Securo's distributed task queue for scalable, asynchronous processing. Learn their roles in background job execution, message brokering, and result storage.

- Repository: [securo-finance/securo](https://github.com/securo-finance/securo)
- Tags: architecture
- Published: 2026-08-28

---

**Celery and Redis serve as Securo's distributed task queue system, with Celery executing background jobs and Redis acting as both the message broker and result backend to enable asynchronous, scalable processing of time‑consuming operations.**

Securo, an open‑source financial platform, relies on **Celery** and **Redis** to keep its API responsive while handling heavy‑duty work behind the scenes. Whether syncing bank connections, refreshing foreign‑exchange rates, or applying asset growth rules, these operations run outside the request/response cycle thanks to this asynchronous architecture. This guide examines how these components are wired together in the `securo-finance/securo` codebase, with concrete examples from the source files.

## How Celery and Redis Fit Into Securo's Architecture

### Component Responsibilities

| Component | Purpose in Securo | Key Source File |
|-----------|-------------------|-----------------|
| **Celery** | Executes background tasks, provides periodic scheduling via Celery Beat, and offers a simple enqueue API (`celery_app.send_task`) | [`backend/app/worker.py`](https://github.com/securo-finance/securo/blob/main/backend/app/worker.py) |
| **Redis** | Acts as **message broker** (task transport) and **result backend** (outcome storage); centralizes state for worker coordination | [`backend/app/core/config.py`](https://github.com/securo-finance/securo/blob/main/backend/app/core/config.py) (lines 107‑109) |
| **Docker/Kubernetes** | Orchestrates `celery‑worker` and `celery‑beat` containers as separate services | `docker‑compose.yml` (lines 149‑156) |

When an API endpoint receives a request requiring intensive processing—such as uploading a knowledge document or syncing a banking connection—it **dispatches a Celery task** rather than blocking the response. Celery serializes the request as JSON, pushes it to Redis, and a worker process dequeues and executes it. Results are written back to Redis for callers to query.

## Celery Configuration in Securo

The central Celery application is defined in [`backend/app/worker.py`](https://github.com/securo-finance/securo/blob/main/backend/app/worker.py), where the broker and backend are configured to use the same Redis instance.

```python

# backend/app/worker.py (simplified structure)

from celery import Celery
from app.core.config import settings

celery_app = Celery(
    "securo",
    broker=settings.redis_url,      # Redis as message broker

    backend=settings.redis_url,     # Redis as result backend

)

celery_app.conf.beat_schedule = {
    "sync-all-connections-hourly": {
        "task": "app.tasks.sync_tasks.sync_all_connections",
        "schedule": 60 * 60,  # every hour

    },
    # ... additional scheduled tasks

}

```

The `redis_url` is sourced from [`backend/app/core/config.py`](https://github.com/securo-finance/securo/blob/main/backend/app/core/config.py) at lines 107‑109, ensuring consistent configuration across broker, backend, and any direct Redis operations.

## Redis as Broker and Backend

Redis fulfills **dual roles** in Securo's Celery stack:

- **Message broker**: Transports task messages from producers (API endpoints, other services) to available workers. This decouples task submission from execution.
- **Result backend**: Persists task return values, exceptions, and metadata. Callers can query task status and retrieve outcomes asynchronously.

Using Redis for both purposes simplifies deployment—only one service to manage—while providing sufficient throughput and persistence for Securo's workload patterns.

## Defining and Enqueuing Tasks

### Enqueuing from API Endpoints

API endpoints trigger asynchronous work without waiting for completion:

```python

# backend/app/main.py (lines 89-91, simplified)

from app.worker import celery_app

def trigger_sync():
    # Returns immediately; task executes in background

    celery_app.send_task("app.tasks.sync_tasks.sync_all_connections")

```

The `send_task` method accepts the task's fully qualified name, serializes arguments, and places the message on the Redis queue.

### Task Implementation

Actual task logic resides in dedicated modules under `backend/app/tasks/`:

```python

# backend/app/tasks/sync_tasks.py (lines 82-84)

from app.worker import celery_app

@celery_app.task(name="app.tasks.sync_tasks.sync_all_connections")
def sync_all_connections():
    # Heavy-weight logic contacting external banking APIs

    ...

```

The explicit `name` parameter ensures stable task identifiers across refactors, critical for reliable enqueuing and Celery Beat scheduling.

## Periodic Task Scheduling with Celery Beat

Celery Beat, a built‑in scheduler, uses the same Redis store to coordinate recurring jobs across multiple replicas:

```python

# backend/app/worker.py (lines 21-27)

celery_app.conf.beat_schedule = {
    "sync-all-connections-hourly": {
        "task": "app.tasks.sync_tasks.sync_all_connections",
        "schedule": 60 * 60,  # 3600 seconds = hourly

    },
    # ... other scheduled tasks (FX rates, asset growth rules, etc.)

}

```

A single `celery‑beat` container runs this scheduler; if deployed with multiple replicas, external locking (e.g., `redis‑lock` or database‑backed `django‑celery‑beat`) prevents duplicate executions.

## Container Orchestration

Securo's `docker‑compose.yml` defines separate services for workers and the scheduler, each connecting to the shared Redis instance:

```yaml

# docker-compose.yml (lines 149-156)

celery-worker:
  image: securo:latest
  command: celery -A app.worker worker --loglevel=info --concurrency=2
  depends_on:
    - redis

celery-beat:
  image: securo:latest
  command: celery -A app.worker beat --loglevel=info
  depends_on:
    - redis

```

The `concurrency=2` parameter controls worker parallelism; this scales horizontally by adding more `celery‑worker` replicas, all consuming from the same Redis queue.

## Architectural Benefits

- **Decoupled execution**: API returns immediately; heavy work proceeds asynchronously, improving perceived latency.
- **Horizontal scalability**: Add worker containers to increase throughput without code changes.
- **Fault tolerance**: Tasks persist in Redis; crashed workers don't lose pending jobs.
- **Idempotent design**: Scheduled tasks like `apply_asset_growth_rules` are written to run safely even if executed multiple times.

## Key Files Reference

| File | Role |
|------|------|
| [`backend/app/worker.py`](https://github.com/securo-finance/securo/blob/main/backend/app/worker.py) | Celery app instantiation, broker/backend config, beat schedule |
| [`backend/app/core/config.py`](https://github.com/securo-finance/securo/blob/main/backend/app/core/config.py) | `redis_url` setting (lines 107‑109) |
| `backend/app/tasks/*.py` | Task implementations (sync, FX, asset growth, etc.) |
| [`docker-compose.yml`](https://github.com/securo-finance/securo/blob/main/docker-compose.yml) | Service definitions for `celery-worker`, `celery-beat`, `redis` |
| `charts/securo/templates/` | Kubernetes equivalents (Helm charts) |

## Summary

- **Celery** provides Securo's asynchronous task execution and periodic scheduling infrastructure.
- **Redis** functions as both **message broker** and **result backend**, unifying state management for the task queue.
- Tasks are defined in `backend/app/tasks/` and enqueued via `celery_app.send_task()` from API endpoints.
- [`docker-compose.yml`](https://github.com/securo-finance/securo/blob/main/docker-compose.yml) orchestrates separate `celery-worker` and `celery-beat` containers for scalable, resilient background processing.

## Frequently Asked Questions

### What is the role of Redis in Securo's Celery setup?

Redis serves **two purposes**: as the **message broker** that routes tasks from producers to workers, and as the **result backend** that stores task outcomes for later retrieval. Both roles use the same `redis_url` configured in [`backend/app/core/config.py`](https://github.com/securo-finance/securo/blob/main/backend/app/core/config.py) (lines 107‑109), simplifying deployment while providing the persistence and throughput Securo requires.

### How does Securo schedule recurring background tasks?

Securo uses **Celery Beat**, configured in [`backend/app/worker.py`](https://github.com/securo-finance/securo/blob/main/backend/app/worker.py) (lines 21‑27) via the `beat_schedule` dictionary. The beat process runs as a separate container (`celery-beat` in [`docker-compose.yml`](https://github.com/securo-finance/securo/blob/main/docker-compose.yml)) and pushes scheduled task messages to Redis at defined intervals, where workers pick them up for execution.

### Can Securo scale Celery workers horizontally?

Yes. The [`docker-compose.yml`](https://github.com/securo-finance/securo/blob/main/docker-compose.yml) defines stateless `celery-worker` containers that consume from the shared Redis queue. Increasing replica count distributes load automatically, with no code changes required. The `--concurrency` parameter controls threads per worker; combine with horizontal scaling for fine‑grained throughput control.

### Where are Celery tasks defined in the Securo repository?

Task implementations reside in `backend/app/tasks/` (e.g., [`sync_tasks.py`](https://github.com/securo-finance/securo/blob/main/sync_tasks.py)). Each task uses the `@celery_app.task` decorator with an explicit `name` parameter for stable referencing. The central Celery application is defined in [`backend/app/worker.py`](https://github.com/securo-finance/securo/blob/main/backend/app/worker.py), which configures the broker, backend, and periodic schedule.