How Background Jobs Are Scheduled and Executed in the LifeTrace Jobs Module

LifeTrace orchestrates background processing through a dual-layer architecture where SchedulerManager wraps APScheduler with a persistent SQLite store and thread-pool executor, while JobManager dynamically registers interval jobs and immediately pauses disabled ones to allow runtime toggling without recreation.

The lifetrace.jobs package in the freeu-group/lifetrace repository provides the backbone for all asynchronous background work, from screen recording to OCR processing. Understanding how background jobs are scheduled and executed in the jobs module reveals a design that prioritizes persistence, graceful shutdowns, and dynamic configuration. The implementation splits responsibilities between low-level scheduling infrastructure in lifetrace/jobs/scheduler.py and high-level orchestration logic in lifetrace/jobs/job_manager.py.

Architecture Overview

The system decouples scheduling mechanics from business logic through two tightly-coupled components that handle every aspect of background execution.

SchedulerManager

The SchedulerManager class in lifetrace/jobs/scheduler.py serves as a thin wrapper around APScheduler. It instantiates a persistent SQLite job-store located via get_scheduler_database_path() from lifetrace/util/path_utils.py, and configures a ThreadPoolExecutor limited by settings["scheduler.max_workers"]. The manager exposes convenience methods for the rest of the codebase:

• add_interval_job() – registers recurring tasks with second/minute/hour intervals
• add_date_job() – schedules one-off tasks at specific run_date values
• remove_job(), pause_job(), resume_job() – manage runtime state
• modify_job_interval() – updates execution frequency without recreating the job
• start() and shutdown() – control the APScheduler daemon lifecycle

The global scheduler singleton is accessed via the cached factory get_scheduler_manager(), ensuring all modules share the same underlying executor and job store.

JobManager

The JobManager in lifetrace/jobs/job_manager.py acts as the high-level entry point that decides which logical background activities should exist based on configuration and module activation state. For every job type—recorder, OCR, activity aggregation, data cleaning, deadline reminders, and proactive OCR—it performs three critical steps:

1. Lazy imports the concrete execution function (e.g., _execute_capture_task maps to lifetrace.jobs.recorder.execute_capture_task)
2. Registers an interval job with the scheduler regardless of the feature flag status, ensuring job metadata persists in the SQLite store
3. Immediately pauses the job if settings.get("jobs.<name>.enabled") evaluates to False

This "schedule-then-pause" pattern allows administrators to toggle features at runtime without recreating job definitions or losing execution history.

Job Lifecycle and Execution Flow

When the LifeTrace application initializes, the JobManager.start_all() method drives the startup sequence through a predictable orchestration phase.

Startup Sequence

The initialization flow proceeds as follows:

JobManager.start_all()
 ├─ checks module activation via _is_module_active(...)
 ├─ starts SchedulerManager (via get_scheduler_manager())
 ├─ for each job type:
 │   ├─ _start_<name>_job()
 │   │   ├─ optionally instantiates service singleton (e.g., get_recorder_instance())
 │   │   ├─ scheduler.add_interval_job(func=_execute_<name>_task, ...)
 │   │   └─ if disabled → scheduler.pause_job("<name>_job")
 └─ logs the final registration summary

Each _start_<name>_job helper corresponds to a specific background activity defined in files like lifetrace/jobs/recorder.py for screen capture or lifetrace/jobs/ocr.py for text recognition.

Graceful Shutdown

When the application receives a termination signal, JobManager.stop_all() invokes SchedulerManager.shutdown(wait=True). The wait=True parameter blocks the calling thread until all currently executing jobs finish, preventing data corruption in mid-flight operations like database writes or file system modifications.

Persistence and Concurrency Configuration

The jobs module implements robust persistence and thread-safety through concrete configuration tied to physical storage.

SQLite Job Store: APScheduler writes job metadata to a SQLite file returned by get_scheduler_database_path(), ensuring scheduled tasks survive application restarts. The database maintains the state of paused jobs, next run times, and execution counts.

Thread Pool Limits: Concurrency is controlled via settings["scheduler.max_workers"], which initializes the ThreadPoolExecutor. This prevents resource exhaustion when multiple intensive tasks like OCR processing and audio analysis run simultaneously.

Dynamic Reconfiguration: Because jobs are always registered but conditionally paused, changing jobs.<name>.enabled in lifetrace/util/settings.py takes effect immediately via resume_job() or pause_job() without requiring a process restart.

Managing Jobs at Runtime

The scheduler exposes APIs for manipulating existing jobs without modifying source code or restarting the daemon.

Adding Custom Recurring Jobs

You can inject new background work from any module by obtaining the scheduler singleton and registering an interval:

# my_custom_job.py

from lifetrace.jobs.scheduler import get_scheduler_manager
from lifetrace.util.logging_config import get_logger

log = get_logger()

def my_task(param1: str):
    log.info(f"Running my custom task with {param1}")

def schedule_my_task():
    scheduler = get_scheduler_manager()
    scheduler.add_interval_job(
        func=my_task,
        job_id="my_custom_job",
        name="My Custom Job",
        seconds=300,               # run every 5 minutes

        replace_existing=True,
        kwargs={"param1": "demo"},
    )
    # Optional: pause if you want it disabled by default

    # scheduler.pause_job("my_custom_job")

Pausing and Resuming Dynamically

Toggle job execution at runtime based on user preferences or system load:

from lifetrace.jobs.scheduler import get_scheduler_manager

sched = get_scheduler_manager()
sched.pause_job("ocr_job")          # temporarily stop OCR scans

sched.resume_job("ocr_job")         # continue them later

Modifying Intervals Without Recreation

Adjust execution frequency for existing jobs without losing execution history or job IDs:

from lifetrace.jobs.scheduler import get_scheduler_manager

sched = get_scheduler_manager()
sched.modify_job_interval(
    job_id="recorder_job",
    seconds=30,                     # change recorder interval to 30s

)

Extending the Background Job System

Adding new background work requires three steps to maintain consistency with the existing architecture:

1. Define the task function in a new file (e.g., lifetrace/jobs/my_feature.py) with an entry point like execute_my_feature_task()
2. Create a startup helper in JobManager named _start_my_feature_job() that lazily imports the task, calls scheduler.add_interval_job(), and conditionally pauses based on settings.get("jobs.my_feature.enabled")
3. Register the startup call inside JobManager.start_all() alongside existing jobs like the recorder and OCR tasks

This pattern ensures new jobs inherit automatic persistence, graceful shutdown behavior, and dynamic enable/disable capabilities without modifying the core scheduler logic.

Summary

• Two-layer architecture: SchedulerManager handles APScheduler mechanics while JobManager controls which logical jobs exist based on configuration.
• Persistent storage: Jobs are stored in a SQLite database via get_scheduler_database_path(), preserving state across restarts.
• Dynamic toggling: Jobs are always scheduled but immediately paused when disabled, allowing runtime re-enabling without recreation.
• Graceful shutdown: shutdown(wait=True) ensures running tasks complete before the process exits.
• Runtime modification: Intervals and pause states can be changed on the fly via modify_job_interval(), pause_job(), and resume_job().

Frequently Asked Questions

How does LifeTrace persist scheduled jobs across application restarts?

According to the source code in lifetrace/jobs/scheduler.py, the SchedulerManager configures APScheduler with a SQLite job-store located at the path returned by get_scheduler_database_path() in lifetrace/util/path_utils.py. This stores job metadata, next run times, and pause states to disk, ensuring that scheduled tasks survive process restarts without requiring rescheduling during startup.

Can I change job intervals without restarting the application?

Yes. The SchedulerManager exposes modify_job_interval(job_id, ...) which updates the trigger configuration of an existing job in the persistent store. Because the job definition is updated directly in the SQLite-backed APScheduler instance, the new interval takes effect immediately for the next execution cycle without needing to recreate the job or restart the daemon.

What happens to running jobs when the application shuts down?

When JobManager.stop_all() is invoked during shutdown, it calls SchedulerManager.shutdown(wait=True). The wait=True parameter blocks the shutdown sequence until all currently executing jobs finish their work. This prevents partial writes or corrupted state in tasks like the recorder (which writes video files) or OCR processing (which updates database records).

How do I disable a background job without removing it entirely?

The JobManager implements a "schedule-then-pause" pattern: it registers every job with the scheduler during startup regardless of configuration, then immediately calls scheduler.pause_job("<job_id>") if settings.get("jobs.<name>.enabled") is False. This keeps the job definition in the persistent store but prevents execution, allowing you to resume it later via scheduler.resume_job() without recreating the schedule or losing historical run data.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}