How Plane Uses Celery Tasks to Handle Asynchronous Operations: Notifications, Exports, and Email

Plane offloads all time-consuming I/O operations to Celery workers, using a beat scheduler for periodic jobs and Redis locks to prevent duplicate email sends.

Plane, the open-source project management platform, leverages Celery to handle asynchronous operations like notifications, exports, and email sending outside the Django request/response cycle. By queuing tasks in the plane.bgtasks package and processing them via dedicated workers, the application maintains responsive API endpoints while background jobs handle data serialization, file uploads, and message delivery.

Celery Configuration and Task Discovery

The asynchronous architecture centers on plane.celery, which instantiates a Celery app and loads configuration from plane.settings.common. The setup uses app.autodiscover_tasks() to automatically locate every task decorated with @shared_task within the plane.bgtasks package. This modular approach ensures that any new background task placed in the bgtasks directory is immediately available to workers without manual registration.

The beat scheduler—powered by django-celery-beat—defines periodic jobs in the beat_schedule configuration. Key recurring tasks include a five-minute email queue flush and daily maintenance routines, ensuring timely delivery and system cleanup without blocking user requests.

Handling Notifications Asynchronously

When issue-related events occur—such as comments created or state changes—the API view enqueues the notifications task using notifications.delay(...). This task, defined in apps/api/plane/bgtasks/notification_task.py, performs two distinct operations:

• In-app notifications: Builds Notification objects for every subscriber who should receive an in-app badge.
• Email logging: Creates EmailNotificationLog rows for users whose preferences allow email alerts.

These log rows are not sent immediately. Instead, they persist in the database as a queue, decoupling the notification creation from the actual email transmission to allow for batch processing and retry logic.

Processing Email Queues with Periodic Tasks

Email delivery runs on a separate schedule managed by the beat job check-every-five-minutes-to-send-email-notifications. Every five minutes, this job executes plane.bgtasks.email_notification_task.stack_email_notification, which groups pending EmailNotificationLog entries by receiver.

For each receiver batch, the system fires a send_email_notification task via send_email_notification.delay(...). This task implements several safety mechanisms:

• Redis locking: Obtains a lock using acquire_lock before processing to prevent duplicate sends when schedules overlap.
• Template rendering: Uses Django’s EmailMultiAlternatives to construct and deliver the message.
• Status updating: Marks log rows as sent after successful delivery, then releases the lock via release_lock.

This pattern ensures exactly-once delivery semantics even when multiple workers process the same queue.

Background Export Generation

Export operations follow a similar asynchronous pattern. When a user triggers an export from the UI, the view immediately calls issue_export_task.delay() with parameters including provider type, workspace ID, project IDs, and a token ID.

The task in apps/api/plane/bgtasks/export_task.py executes the following workflow:

1. Queries the required issues and serializes them through DataExporter (supporting CSV, JSON, or XLSX formats).
2. Writes each file into a ZIP archive using create_zip_file.
3. Uploads the archive to S3 or MinIO via upload_to_s3.
4. Updates the ExporterHistory record with a pre-signed URL for download.

Because the export task runs completely off-thread, the HTTP request returns immediately with a token that the frontend polls to track completion status.

Safety Mechanisms and Task Orchestration

Plane implements several safeguards to ensure reliable background processing:

Redis Locks for Idempotency: Long-running or idempotent jobs like email sending use Redis-based locking (acquire_lock/release_lock) to prevent duplicate execution when beat schedules overlap or workers retry failed tasks.

Periodic Maintenance: The beat_schedule in apps/api/plane/celery.py defines additional periodic tasks for hard deletes, cleanup operations, and metric pushing, ensuring the system remains performant without manual intervention.

Shared Task Decorator: All tasks use the @shared_task decorator, making them discoverable by the autodiscovery mechanism and ensuring compatibility with the Celery app configuration.

Summary

• Plane uses Celery workers and a beat scheduler to handle asynchronous operations outside the Django request cycle.
• Notifications are created immediately via notifications.delay() but emails are queued in EmailNotificationLog rows for batch processing.
• The check-every-five-minutes-to-send-email-notifications beat job aggregates and sends emails using Redis locks to prevent duplicates.
• Exports run via issue_export_task, which serializes data, creates ZIP archives, uploads to S3, and updates ExporterHistory records asynchronously.
• All tasks reside in plane.bgtasks, use @shared_task, and are auto-discovered by the Celery app configuration.

Frequently Asked Questions

How does Plane prevent duplicate email notifications?

Plane uses Redis-based locking mechanisms within the send_email_notification task. Before processing a batch, the task calls acquire_lock to obtain a lock, processes the emails, and then calls release_lock. This prevents multiple workers from sending the same notification when the five-minute beat schedule overlaps or during task retries.

What triggers the export task in Plane?

When a user initiates an export from the UI, the API view calls issue_export_task.delay() with parameters including the provider type (CSV, JSON, or XLSX), workspace ID, project IDs, and a unique token ID. This immediately queues the background task, allowing the HTTP response to return while the worker handles data serialization and file upload.

How often does Plane process queued emails?

Plane processes the email queue every five minutes via the check-every-five-minutes-to-send-email-notifications beat schedule. This periodic task runs stack_email_notification, which aggregates pending EmailNotificationLog entries and dispatches individual send_email_notification tasks for each receiver batch.

What is the purpose of the plane.bgtasks package?

The plane.bgtasks package serves as the dedicated location for all Celery background tasks, including notification handling, email delivery, and export generation. Using app.autodiscover_tasks() in plane.celery, the system automatically discovers any task decorated with @shared_task within this package, ensuring modular and scalable task registration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}