How Security Health Check Jobs Work in Maybe Finance: Implementation Guide

The Maybe finance app runs daily security health check jobs through SecurityHealthCheckJob to verify that market securities can still be priced by external providers, automatically marking securities as offline after five consecutive failures and purging stale price data.

The Maybe open-source finance platform relies on security health check jobs to maintain data integrity for market securities. These scheduled background jobs verify that every ticker symbol remains accessible to price providers, handling thousands of securities in prioritized batches while preventing API overload. The implementation spans a dedicated job class and a comprehensive health checker model that manages state transitions and error handling.

Job Entry Point: SecurityHealthCheckJob

The orchestration begins in app/jobs/security_health_check_job.rb, which serves as the scheduled entry point for the daily validation cycle.

class SecurityHealthCheckJob < ApplicationJob
  queue_as :scheduled

  def perform
    return if Rails.env.development?   # skip in dev

    Security::HealthChecker.check_all
  end
end

The job queues itself as :scheduled and immediately delegates to Security::HealthChecker.check_all. An early return prevents execution in development environments, ensuring local development does not trigger unnecessary external API calls. According to the Maybe codebase, this job triggers daily via the scheduler defined in config/schedule.yml.

Core Implementation in Security::HealthChecker

The bulk of the logic resides in app/models/security/health_checker.rb, which implements a robust batch processing system with failure tracking and automatic remediation.

Prioritized Batch Processing with check_all

The check_all method processes securities in two distinct priority tiers to ensure new securities receive immediate attention while maintaining existing ones:

def check_all
  # 1️⃣  Never‑checked securities → unlimited, highest priority

  never_checked_scope.find_each { |security| new(security).run_check }

  # 2️⃣  “Due” securities → limited by DAILY_BATCH_SIZE

  due_for_check_scope.limit(DAILY_BATCH_SIZE).each { |security| new(security).run_check }
end

Never-checked securities (where last_health_check_at is nil) receive unlimited processing capacity during each run, guaranteeing that newly added tickers undergo immediate validation. Due securities—those exceeding the HEALTH_CHECK_INTERVAL of seven days—process in limited batches controlled by DAILY_BATCH_SIZE (set to 1000), protecting external price provider APIs from request flooding.

Query Scopes for Targeted Checking

The class defines private scope methods to efficiently query the database:

def never_checked_scope
  Security.where(last_health_check_at: nil)
end

def due_for_check_scope
  Security.where(last_health_check_at: ..HEALTH_CHECK_INTERVAL.ago)
          .order(last_health_check_at: :asc)
end

The range syntax ..HEALTH_CHECK_INTERVAL.ago creates an inclusive range matching any timestamp older than seven days, while the ascending order ensures oldest checks receive priority.

Single Security Validation via run_check

Each security undergoes individual validation through the instance method run_check, which coordinates logging, price fetching, state updates, and error tracking:

def run_check
  Rails.logger.info("Running health check for #{security.ticker}")

  if latest_provider_price
    handle_success
  else
    handle_failure
  end
rescue => e
  Sentry.capture_exception(e) { |scope| scope.set_tags(security_id: @security.id) }
ensure
  security.update!(last_health_check_at: Time.current)
end

The method attempts to fetch the current price through latest_provider_price. Successful fetches trigger handle_success, while failures increment error counters via handle_failure. The ensure block guarantees last_health_check_at updates regardless of outcome, preventing stuck records. Unexpected exceptions route to Sentry with the security ID tagged for debugging.

Price Fetching Logic

The latest_provider_price method interfaces with the configured price provider (such as Synth API):

def latest_provider_price
  return nil unless provider.present?

  response = provider.fetch_security_price(
    symbol: security.ticker,
    exchange_operating_mic: security.exchange_operating_mic,
    date: Date.current
  )
  return nil unless response.success?
  response.data.price
end

If the provider is absent or returns a non-successful response, the method returns nil, signaling a failure to the calling logic.

Success and Failure State Management

Successful validations reset failure counters and ensure the security remains online:

def handle_success
  security.update!(
    offline: false,
    failed_fetch_count: 0,
    failed_fetch_at: nil
  )
end

Failed validations increment the failed_fetch_count and store the timestamp:

def handle_failure
  new_failure_count = security.failed_fetch_count.to_i + 1
  new_failure_at   = Time.current

  if new_failure_count > MAX_CONSECUTIVE_FAILURES
    convert_to_offline_security!
  else
    security.update!(
      failed_fetch_count: new_failure_count,
      failed_fetch_at:    new_failure_at
    )
  end
end

The constant MAX_CONSECUTIVE_FAILURES equals 5. After exceeding this threshold, the security moves offline via convert_to_offline_security!, which marks the record as offline: true, resets failure counters, and deletes all associated price records:

def convert_to_offline_security!
  security.update!(offline: true, failed_fetch_count: 0, failed_fetch_at: nil)
  Price.where(security_id: security.id).delete_all   # purge stale price rows

end

This data cleanup prevents the application from displaying or calculating with untrusted historical prices for delisted or invalid securities.

Scheduling Configuration

The daily execution relies on the scheduler configuration in config/schedule.yml, which queues SecurityHealthCheckJob once per day in production environments. The job respects the environment guard in the job class itself, ensuring development and test environments skip the expensive health check cycle.

Testing the Health Check System

Comprehensive unit tests in test/models/security/health_checker_test.rb verify the batch logic, scope accuracy, and state transitions. The test suite uses Mocha expectations to assert that check_all invokes run_check the correct number of times for each scope, and confirms that securities transition to offline status exactly after five consecutive failures.

Practical Code Examples

Running the Health Check Manually

Trigger the full daily cycle immediately from the Rails console:

# Load the class if needed in console

require_relative 'app/models/security/health_checker'

# Execute the complete check for all applicable securities

Security::HealthChecker.check_all

Checking a Single Security

Validate an individual ticker without processing the entire batch:

security = Security.find_by(ticker: 'AAPL')
checker = Security::HealthChecker.new(security)
checker.run_check

# Inspect results

security.reload
puts "Offline: #{security.offline}, Failures: #{security.failed_fetch_count}"

Customizing the Price Provider

Swap the underlying data source by configuring a custom provider in an initializer:

# config/initializers/security_provider.rb

module Security
  def self.provider
    @provider ||= CustomPriceProvider.new(api_key: ENV['CUSTOM_API_KEY'])
  end
end

The Security::HealthChecker automatically uses this provider on the next scheduled run.

Summary

• Entry point: SecurityHealthCheckJob in app/jobs/security_health_check_job.rb queues daily and delegates to the health checker.
• Batch processing: Security::HealthChecker.check_all prioritizes never-checked securities, then processes up to 1000 due securities per day.
• Failure tolerance: Securities tolerate 5 consecutive fetch failures before the system marks them offline via convert_to_offline_security!.
• Data integrity: Offline conversion purges all Price records for the security to prevent stale data usage.
• Scheduling: The job runs via config/schedule.yml in production only, with environment guards preventing dev execution.

Frequently Asked Questions

How often do security health check jobs run in Maybe?

The jobs execute once daily in production environments according to the schedule defined in config/schedule.yml. The SecurityHealthCheckJob class includes an early return that skips execution entirely when Rails.env.development? evaluates to true, ensuring local development environments do not waste API quota or processing time on health checks.

What happens when a security fails the health check repeatedly?

After 5 consecutive failures (controlled by the MAX_CONSECUTIVE_FAILURES constant in app/models/security/health_checker.rb), the system invokes convert_to_offline_security!. This method marks the security as offline: true, resets the failure counters, and executes Price.where(security_id: security.id).delete_all to remove all historical price data for that ticker, ensuring the application does not reference untrusted stale prices.

Why does the health checker process securities in two separate batches?

The check_all method uses two scopes—never_checked_scope and due_for_check_scope—to prioritize new securities while protecting external API rate limits. Never-checked securities process without limits to ensure immediate validation of newly added tickers, while "due" securities (those unchecked for over seven days) process in limited batches of 1000 per day to prevent overwhelming the price provider and to distribute load evenly across the week.

Can I manually trigger a health check for a specific security?

Yes. Instantiate Security::HealthChecker with a specific security record and call run_check directly:

security = Security.find_by(ticker: 'TSLA')
Security::HealthChecker.new(security).run_check

This executes the full validation cycle—including price fetching, state updates, and error logging—for that single security without affecting the scheduled batch processing of other records.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}