# How the Zero-Token Portal Scanner Works with Greenhouse, Ashby, and Lever

> Discover how the zero-token portal scanner efficiently scrapes job listings from Greenhouse, Ashby, and Lever via public APIs, consuming zero LLM tokens for initial crawls.

- Repository: [Santiago Fernández de Valderrama/career-ops](https://github.com/santifer/career-ops)
- Tags: how-to-guide
- Published: 2026-07-03

---

**The Career-Ops zero-token portal scanner retrieves job listings from Greenhouse, Ashby, and Lever using pure HTTP GET requests to public APIs, consuming no LLM tokens during the initial crawl.**

The `santifer/career-ops` repository implements a cost-efficient job board crawler that interfaces directly with Applicant Tracking System (ATS) providers. By leveraging provider-specific public API endpoints rather than browser automation or AI inference for data extraction, the zero-token portal scanner can process thousands of career pages at minimal operational cost. The architecture relies on a modular provider system that normalizes disparate ATS responses into a canonical job format before any optional LLM-based processing occurs.

## Core Architecture and Provider Loading

The orchestration logic lives in **`scan.mjs`**, which dynamically discovers and loads provider modules at runtime. The `loadProviders()` function (lines 61‑86 of `scan.mjs`) reads every `*.mjs` file in the `providers/` directory, skipping files prefixed with an underscore, and imports them via `await import()`. Each module exports an `id`, an optional `detect()` method, and a required `fetch()` method that conforms to the zero-token contract.

## Provider Detection and Resolution

For each company defined in [`portals.yml`](https://github.com/santifer/career-ops/blob/main/portals.yml), the scanner invokes `resolveProvider()` (lines 91‑124 of `scan.mjs`). The resolution follows a strict precedence:

1. **Explicit provider field** – If the entry specifies `provider: greenhouse`, that module is selected immediately.
2. **Local parser** – If a local parsing rule exists for the domain.
3. **Dynamic detection** – Each provider’s `detect(entry)` method inspects the `careers_url` (or an explicit `api:` field) and returns an API endpoint object if the host matches.

The first provider to return a truthy value wins the resolution.

## ATS Provider Implementations

Each ATS module implements the same interface: `detect()` returns metadata including the concrete API URL, and `fetch()` performs the HTTP request and normalizes the response.

### Greenhouse

The Greenhouse provider (`providers/greenhouse.mjs`) extracts the board slug from URLs such as `https://jobs.company.com/job-boards.greenhouse.io/company` using a regex in `detect()` (lines 51‑55). It constructs the endpoint `https://boards-api.greenhouse.io/v1/boards/<slug>/jobs` and returns it to the orchestrator.

The `fetch()` method (lines 60‑73) calls `ctx.fetchJson(apiUrl, {redirect:'error'})` via the shared `_http.mjs` context. It maps the JSON payload to a canonical job object containing `title`, `url`, `company`, `location`, and `postedAt`, filtering out entries lacking an `absolute_url`.

### Ashby

The Ashby provider (`providers/ashby.mjs`) detects URLs containing `jobs.ashbyhq.com` or explicit `api:` fields. It validates the host against an allow-list before returning the public Ashby API endpoint. The `fetch()` implementation retrieves the JSON posting list and normalizes it to the same canonical shape as Greenhouse, ensuring downstream filters receive consistent data structures regardless of the source ATS.

### Lever

The Lever provider (`providers/lever.mjs`) recognizes career pages hosted at `https://jobs.lever.co/company`. Its `detect()` method builds the endpoint `https://api.lever.co/v0/postings/<company>`. The `fetch()` function hits this endpoint, parses the JSON array of postings, and maps each entry to the standard job object format, completing the zero-token retrieval cycle.

## Data Normalization and Filtering

After fetching, raw job lists pass through a series of pure-JavaScript filters defined in `scan.mjs` (lines 26‑98). These include `buildTitleFilter()`, `buildLocationFilter()`, `buildSalaryFilter()`, `buildContentFilter()`, and `buildCooldownFilter()`, along with URL deduplication. All filters execute against the canonical job objects before any LLM-based scoring, guaranteeing the scanner remains token-free during the extraction phase.

## Optional Liveness Verification

The zero-token path excludes browser automation to minimize costs. However, users can invoke **`node scan.mjs --verify`** to enable the liveness check. This triggers `verifyOffers()` (lines 443‑523 of `scan.mjs`), which launches Playwright (Chromium) to confirm that job URLs still host live "Apply" buttons. This step occurs **after** the zero-token fetch and filter pipeline, ensuring Playwright costs are incurred only when explicitly requested for final validation.

## Running the Scanner

Execute a standard zero-token scan that writes new offers to [`data/pipeline.md`](https://github.com/santifer/career-ops/blob/main/data/pipeline.md):

```bash
node scan.mjs

```

Preview changes without writing files:

```bash
node scan.mjs --dry-run

```

Run with optional liveness verification (adds Playwright overhead):

```bash
node scan.mjs --verify

```

Rescue moved URLs (e.g., Greenhouse 404s) during verification:

```bash
node scan.mjs --verify --rediscover-404

```

## Summary

- **Zero-token architecture**: The scanner uses simple HTTP GET requests to public ATS APIs (Greenhouse, Ashby, Lever) rather than LLM inference or browser rendering for data extraction.
- **Modular providers**: `scan.mjs` loads provider modules dynamically from `providers/`, with each module exposing `detect()` and `fetch()` methods.
- **Canonical normalization**: All ATS responses are mapped to a uniform job object format before filtering.
- **Pure-JS filtering**: Title, location, salary, content, and cooldown filters run as fast JavaScript predicates without consuming tokens.
- **Optional verification**: Playwright-based liveness checks occur only when the `--verify` flag is passed, keeping the default path completely token-free.

## Frequently Asked Questions

### What makes the scanner "zero-token"?

The scanner retrieves job listings using direct HTTP requests to public ATS endpoints (e.g., `boards-api.greenhouse.io`, `api.lever.co`). Because it parses JSON responses with native JavaScript rather than calling Claude or other LLMs to extract structured data, it consumes zero AI tokens during the crawl phase.

### How does the scanner detect which ATS a company uses?

The `resolveProvider()` function (lines 91‑124 of `scan.mjs`) checks three sources in order: an explicit `provider` field in the configuration, a local parser rule, or the `detect()` method exported by each provider module. Greenhouse, Ashby, and Lever each implement `detect()` to recognize their respective URL patterns and return the appropriate API endpoint.

### Can I add support for a new ATS provider?

Yes. Create a new file in `providers/` (e.g., `custom-ats.mjs`) that exports an `id`, an optional `detect(entry)` function that returns an API URL if the entry matches, and a `fetch(entry, ctx)` function that returns an array of canonical job objects. The orchestrator will automatically load the module on the next run of `scan.mjs`.

### When does the scanner use Playwright?

Playwright is invoked only when the `--verify` flag is passed. The `verifyOffers()` function (lines 443‑523) then launches a headless browser to check that job URLs are still active. This occurs after the zero-token fetch and filter pipeline, ensuring that token consumption (and browser overhead) happens only during the optional verification step.