How `liveness-core.mjs` Determines Expired vs Active Job Postings in career-ops

liveness-core.mjs exports a pure classifyLiveness function that consumes HTTP status codes, final URLs, page bodies, and detected apply controls to return a deterministic expired, active, or uncertain verdict, ensuring the santifer/career-ops pipeline filters out dead listings before further processing.

In the santifer/career-ops repository, liveness-core.mjs acts as the central arbiter for job posting validity. All scanners, browser-based helpers, and verification scripts import its single exported function to decide whether a listing is still open. Understanding the role of liveness-core.mjs in determining expired vs active job postings is critical because it prevents stale data from propagating through the tracker and PDF generation stages.

The classifyLiveness API and Return Values

The module exposes one function: classifyLiveness({ status, finalUrl, bodyText, applyControls }). It returns a small, serializable object shaped as { result, code, reason } where result is one of three strings:

• expired — The posting is no longer available.
• active — The posting is still open and accepts applications.
• uncertain — The crawler cannot be sure, often due to a bot challenge.

Because the function is pure and performs no side effects, it can be imported safely into both headless Playwright runs and lightweight CLI scripts.

Hierarchy of Heuristics in liveness-core.mjs

The classifier applies an ordered set of rules. Earlier, stronger signals override later, weaker ones.

Hard HTTP Gone Signals (404/410)

The highest-priority expiration signal comes directly from the HTTP layer. If the server returns a 404 or 410 status code, classifyLiveness immediately marks the posting as expired without inspecting the body.

if (status === 404 || status === 410) {
  return { result: 'expired', code: 'http_gone', ... };
}

This logic is implemented in liveness-core.mjs at lines 74-76.

Bot and Anti-Scraping Detection

Anti-bot walls must not be misclassified as expired jobs. When the response body matches known Cloudflare, hCaptcha, or similar challenge patterns—or when the status is 403 or 503—the function returns uncertain. This preserves listings that are merely blocked rather than removed.

const botChallenge = firstMatch(BOT_CHALLENGE_PATTERNS, bodyText);
if (botChallenge) {
  return { result: 'uncertain', code: 'bot_challenge', ... };
}
if (status === 403 || status === 503) {
  return { result: 'uncertain', code: 'http_blocked', ... };
}

See liveness-core.mjs lines 82-88.

Expired URL Redirects and Hard Banners

Two content-level rules catch soft 200 responses that actually represent dead jobs. First, if the final URL contains an error query such as ?error=true, the posting is marked expired (lines 90-93). Second, the module runs HARD_EXPIRED_PATTERNS against the body text to catch explicit phrases like “job no longer available” (lines 95-98).

const expiredUrl = firstMatch(EXPIRED_URL_PATTERNS, finalUrl);
if (expiredUrl) {
  return { result: 'expired', code: 'expired_url_param', ... };
}

const expiredBody = firstMatch(HARD_EXPIRED_PATTERNS, bodyText);
if (expiredBody) {
  return { result: 'expired', code: 'hard_expired_text', ... };
}

Active Apply Controls

The primary positive signal is the presence of an apply button or link. If the applyControls array contains text matching APPLY_PATTERNS, the posting is classified as active because a genuine application path still exists.

if (hasApplyControl(applyControls)) {
  return { result: 'active', code: 'apply_control_visible', ... };
}

This rule resides at liveness-core.mjs lines 100-102.

Low-Value Content Filters

Before falling back to uncertainty, the module filters out pages that lack meaningful posting data. Generic search results matching LISTING_PAGE_PATTERNS are marked expired (lines 104-107), as are pages whose bodyText is shorter than MIN_CONTENT_CHARS (300 characters), indicating a navigation stub or footer fragment (lines 109-111).

const listingPage = firstMatch(LISTING_PAGE_PATTERNS, bodyText);
if (listingPage) {
  return { result: 'expired', code: 'listing_page', ... };
}

if (bodyText.trim().length < MIN_CONTENT_CHARS) {
  return { result: 'expired', code: 'insufficient_content', ... };
}

Final Uncertain Fallback

When none of the above rules trigger, classifyLiveness returns uncertain with the code no_apply_control. This indicates the page appears legitimate but offers no clear application element.

return { result: 'uncertain', code: 'no_apply_control', reason: 'no clear apply control found' };

The fallback is defined at liveness-core.mjs lines 113-114.

Reusing classifyLiveness Across the career-ops Pipeline

Because the logic is pure, it is reused by multiple entry points in the repository.

Direct CLI Verification in test-all.mjs

The test-all.mjs harness demonstrates major branches by calling the function directly against saved or live HTTP responses.

import { classifyLiveness } from './liveness-core.mjs';

const result = classifyLiveness({
  status: 200,
  finalUrl: 'https://example.com/job/123',
  bodyText: pageHtml,
  applyControls: ['Apply', 'Easy Apply']
});
console.log(result);
// → { result: 'active', code: 'apply_control_visible', reason: 'visible apply control detected' }

(source: test-all.mjs lines 100-102)

Playwright Browser Wrapper in liveness-browser.mjs

For headless verification, liveness-browser.mjs extracts data from a Playwright page object and passes it into the same core function.

import { classifyLiveness } from './liveness-core.mjs';

export async function checkLiveness(page) {
  const status = page.response()?.status() ?? 0;
  const finalUrl = page.url();
  const bodyText = await page.content();
  const applyControls = await page.$$eval('button, a', els => els.map(e => e.textContent));

  return classifyLiveness({ status, finalUrl, bodyText, applyControls });
}

(source: liveness-browser.mjs lines 5-9)

Downstream Pipeline Actions

When the scanner fetches a posting, the returned result drives downstream behavior:

• expired — The job is ignored and recorded in scan-history.tsv.
• active — The job proceeds to evaluation, PDF generation, and tracker entry.
• uncertain — The job is flagged for manual verification.

Summary

• liveness-core.mjs provides a pure, deterministic classifyLiveness function consumed by scanners and verifiers.
• It evaluates HTTP status codes, final URLs, body text, and apply controls in a strict priority order.
• Hard signals like 404/410 and explicit expired banners dominate softer cues such as missing apply buttons.
• Anti-bot challenges are deliberately classified as uncertain to avoid false negatives.
• The module supports both headless browser and script-driven workflows across liveness-browser.mjs and test-all.mjs.

Frequently Asked Questions

What inputs does classifyLiveness require?

It expects an object with four properties: status, finalUrl, bodyText, and applyControls. These values are typically harvested from an HTTP response or a Playwright page evaluation and then fed into the classifier.

Why does liveness-core.mjs return uncertain instead of expired for 403 and 503 errors?

Status codes like 403 and 503, along with Cloudflare or hCaptcha challenges, indicate anti-bot blocking rather than a removed job. Marking them uncertain prevents the pipeline from incorrectly discarding active listings that are merely inaccessible to the crawler.

How does the module distinguish between an expired job and a generic listing page?

It matches the body text against LISTING_PAGE_PATTERNS and checks the content length against MIN_CONTENT_CHARS (300 characters). Both filters catch thin or search-result pages that lack a specific posting detail.

Can classifyLiveness be used outside of the browser automation flow?

Yes. Because the function is pure with no side effects, test-all.mjs and other Node.js scripts can import it directly from liveness-core.mjs to evaluate raw HTTP responses without launching Playwright.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}