# How the Portal Health Check in ai-job-search Detects Broken Job Scrapers > Learn how the portal health check in ai-job-search detects broken job scrapers by analyzing results for degradation signals and running sentinel probes to confirm failures. - Repository: [Mads Lorentzen/ai-job-search](https://github.com/MadsLorentzen/ai-job-search) - Tags: internals - Published: 2026-07-21 --- **The portal health check detects broken job scrapers by analyzing returned results for degradation signals—such as null company names or HTML entities in titles—then running bounded sentinel probes to confirm failures before reporting them.** The ai-job-search repository by MadsLorentzen automates job discovery across multiple Danish and international portals. Its **portal health check** mechanism, implemented as Step 4.75 in the job-scraper skill, continuously validates that CLI-based scrapers remain functional after site updates or redesigns. ## How the Three-Stage Detection Pipeline Works The health check follows a rigorous three-stage pipeline defined in [`.claude/skills/job-scraper/SKILL.md`](https://github.com/MadsLorentzen/ai-job-search/blob/main/.claude/skills/job-scraper/SKILL.md). This design ensures that portals are only declared broken after concrete evidence is observed, never due to temporary rate limits or missing historical data. ### Stage 1: Analyzing the Degraded Scan After a standard `/scrape` run completes Step 1b (the CLI search), the health check examines actual results from each enabled portal. It flags a portal as **degraded** if **any** of the following conditions apply to **all** returned results: - The `company` field is `null` or empty. - The `title` field is empty or contains raw HTML entities such as `&`. - The job URL does not belong to the portal's own domain. Additionally, if a portal returns **zero** results during the current run, the check consults the historical data in [`job_scraper/seen_jobs.json`](https://github.com/MadsLorentzen/ai-job-search/blob/main/job_scraper/seen_jobs.json). If the portal previously produced jobs—identified by the stored `portal` field or by matching the URL domain—the sudden absence of results marks the portal as **suspect**. ### Stage 2: Executing the Sentinel Probe For every portal flagged as suspect, the health check initiates a bounded escalation probe: 1. It executes the portal's documented **search** command using the **example query** from its own [`SKILL.md`](https://github.com/MadsLorentzen/ai-job-search/blob/main/SKILL.md), limiting output to three items with JSON formatting. 2. If this probe returns no results, the check retries once using a generic common word. 3. If the second attempt also yields nothing, the portal is definitively marked **broken**. 4. A `429 Too Many Requests` response or any rate-limit page is treated as **inconclusive**, sparing the portal from being labeled broken during temporary blocks. This probe mechanism guarantees the check remains **bounded**—at most one probe, one retry, and one optional detail fetch per portal. ### Stage 3: Reporting Health Status Healthy portals remain silent in the output. Any portal classified as **degraded**, **broken**, or **inconclusive** generates a `health:` line in the final Step 5 summary output. For example: ``` health: jobbank – degraded (company null on all results) health: jobnet – broken (sentinel probe returned no jobs) ``` Users are then prompted to optionally disable faulty portals by setting `enabled: false` in their configuration until the scraper is updated. ## Running the Portal Health Check You can invoke the health check independently or as part of the standard workflow. These commands are specified in the skill documentation: ```bash # Run a normal job scrape (includes health check at the end) /scrape # Run the health check alone (probe mode, no new searches) /scrape health # checks all enabled portals /scrape health jobnet # probes only the “jobnet” portal ``` ## Key Files Supporting the Health Check The detection logic relies on several specific files within the repository: - **[`.claude/skills/job-scraper/SKILL.md`](https://github.com/MadsLorentzen/ai-job-search/blob/main/.claude/skills/job-scraper/SKILL.md)**: Contains the full specification of the job-scraper workflow, including the complete Step 4.75 health-check logic. - **[`job_scraper/seen_jobs.json`](https://github.com/MadsLorentzen/ai-job-search/blob/main/job_scraper/seen_jobs.json)**: Stores previously seen job entries; used to detect "sudden silence" when a portal stops returning results. - **[`search-queries.md`](https://github.com/MadsLorentzen/ai-job-search/blob/main/search-queries.md)**: Provides example queries referenced by the sentinel probe when testing portal responsiveness. - **`.agents/skills/*/SKILL.md`**: Per-portal skill files that define the exact CLI invocation and example queries reused during health-check probing. ## Summary - The portal health check operates as **Step 4.75** in the job-scraper skill, running automatically after standard scrapes or via dedicated commands. - **Degradation detection** identifies portals returning null company names, malformed titles, or off-domain URLs, while historical comparison in [`job_scraper/seen_jobs.json`](https://github.com/MadsLorentzen/ai-job-search/blob/main/job_scraper/seen_jobs.json) spots sudden silence. - **Sentinel probes** confirm breakage by retrying searches with documented example queries and generic fallbacks, treating rate-limit responses as inconclusive rather than broken. - The system is intentionally **bounded** and **honest**, requiring concrete evidence before declaring a portal broken and outputting structured `health:` reports for actionable remediation. ## Frequently Asked Questions ### What triggers a portal to be flagged as degraded? A portal is flagged as degraded when **all** results from a standard scrape exhibit data quality issues—specifically null or empty `company` fields, empty titles containing HTML entities like `&`, or URLs that do not match the portal's domain. This indicates the scraper is still running but likely parsing changed HTML structures incorrectly. ### How does the sentinel probe confirm a scraper is broken? The sentinel probe executes the portal's documented search command using its **example query** from [`SKILL.md`](https://github.com/MadsLorentzen/ai-job-search/blob/main/SKILL.md), limiting results to three items. If no jobs return, it retries with a generic common word. Only after both attempts fail is the portal marked **broken**, ensuring temporary glitches or empty query results do not trigger false positives. ### What happens when a portal returns a 429 error during health checks? Rate-limit responses such as `429 Too Many Requests` are treated as **inconclusive**. According to the logic in [`.claude/skills/job-scraper/SKILL.md`](https://github.com/MadsLorentzen/ai-job-search/blob/main/.claude/skills/job-scraper/SKILL.md), the health check distinguishes between technical blocking and actual breakage, refusing to declare a portal broken when it is merely throttling requests. ### Where is the health check logic documented? The complete health check specification resides in **[`.claude/skills/job-scraper/SKILL.md`](https://github.com/MadsLorentzen/ai-job-search/blob/main/.claude/skills/job-scraper/SKILL.md)**, which details the three-stage pipeline from degraded scan analysis through sentinel probing to final reporting. Individual portal configurations in `.agents/skills/*/SKILL.md` provide the example queries and CLI commands used during the escalation probe stage.