How the Portal Health Check in ai-job-search Detects Broken Job Scrapers

The portal health check detects broken job scrapers by analyzing returned results for degradation signals—such as null company names or HTML entities in titles—then running bounded sentinel probes to confirm failures before reporting them.

The ai-job-search repository by MadsLorentzen automates job discovery across multiple Danish and international portals. Its portal health check mechanism, implemented as Step 4.75 in the job-scraper skill, continuously validates that CLI-based scrapers remain functional after site updates or redesigns.

How the Three-Stage Detection Pipeline Works

The health check follows a rigorous three-stage pipeline defined in .claude/skills/job-scraper/SKILL.md. This design ensures that portals are only declared broken after concrete evidence is observed, never due to temporary rate limits or missing historical data.

Stage 1: Analyzing the Degraded Scan

After a standard /scrape run completes Step 1b (the CLI search), the health check examines actual results from each enabled portal. It flags a portal as degraded if any of the following conditions apply to all returned results:

• The company field is null or empty.
• The title field is empty or contains raw HTML entities such as &.
• The job URL does not belong to the portal's own domain.

Additionally, if a portal returns zero results during the current run, the check consults the historical data in job_scraper/seen_jobs.json. If the portal previously produced jobs—identified by the stored portal field or by matching the URL domain—the sudden absence of results marks the portal as suspect.

Stage 2: Executing the Sentinel Probe

For every portal flagged as suspect, the health check initiates a bounded escalation probe:

1. It executes the portal's documented search command using the example query from its own SKILL.md, limiting output to three items with JSON formatting.
2. If this probe returns no results, the check retries once using a generic common word.
3. If the second attempt also yields nothing, the portal is definitively marked broken.
4. A 429 Too Many Requests response or any rate-limit page is treated as inconclusive, sparing the portal from being labeled broken during temporary blocks.

This probe mechanism guarantees the check remains bounded—at most one probe, one retry, and one optional detail fetch per portal.

Stage 3: Reporting Health Status

Healthy portals remain silent in the output. Any portal classified as degraded, broken, or inconclusive generates a health: line in the final Step 5 summary output. For example:

health: jobbank – degraded (company null on all results)
health: jobnet  – broken (sentinel probe returned no jobs)

Users are then prompted to optionally disable faulty portals by setting enabled: false in their configuration until the scraper is updated.

Running the Portal Health Check

You can invoke the health check independently or as part of the standard workflow. These commands are specified in the skill documentation:

# Run a normal job scrape (includes health check at the end)

/scrape

# Run the health check alone (probe mode, no new searches)

/scrape health           # checks all enabled portals

/scrape health jobnet    # probes only the “jobnet” portal

Key Files Supporting the Health Check

The detection logic relies on several specific files within the repository:

• .claude/skills/job-scraper/SKILL.md: Contains the full specification of the job-scraper workflow, including the complete Step 4.75 health-check logic.
• job_scraper/seen_jobs.json: Stores previously seen job entries; used to detect "sudden silence" when a portal stops returning results.
• search-queries.md: Provides example queries referenced by the sentinel probe when testing portal responsiveness.
• .agents/skills/*/SKILL.md: Per-portal skill files that define the exact CLI invocation and example queries reused during health-check probing.

Summary

• The portal health check operates as Step 4.75 in the job-scraper skill, running automatically after standard scrapes or via dedicated commands.
• Degradation detection identifies portals returning null company names, malformed titles, or off-domain URLs, while historical comparison in job_scraper/seen_jobs.json spots sudden silence.
• Sentinel probes confirm breakage by retrying searches with documented example queries and generic fallbacks, treating rate-limit responses as inconclusive rather than broken.
• The system is intentionally bounded and honest, requiring concrete evidence before declaring a portal broken and outputting structured health: reports for actionable remediation.

Frequently Asked Questions

What triggers a portal to be flagged as degraded?

A portal is flagged as degraded when all results from a standard scrape exhibit data quality issues—specifically null or empty company fields, empty titles containing HTML entities like &, or URLs that do not match the portal's domain. This indicates the scraper is still running but likely parsing changed HTML structures incorrectly.

How does the sentinel probe confirm a scraper is broken?

The sentinel probe executes the portal's documented search command using its example query from SKILL.md, limiting results to three items. If no jobs return, it retries with a generic common word. Only after both attempts fail is the portal marked broken, ensuring temporary glitches or empty query results do not trigger false positives.

What happens when a portal returns a 429 error during health checks?

Rate-limit responses such as 429 Too Many Requests are treated as inconclusive. According to the logic in .claude/skills/job-scraper/SKILL.md, the health check distinguishes between technical blocking and actual breakage, refusing to declare a portal broken when it is merely throttling requests.

Where is the health check logic documented?

The complete health check specification resides in .claude/skills/job-scraper/SKILL.md, which details the three-stage pipeline from degraded scan analysis through sentinel probing to final reporting. Individual portal configurations in .agents/skills/*/SKILL.md provide the example queries and CLI commands used during the escalation probe stage.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}