How Batch Scoring in `/rank` Evaluates Multiple Job Postings

The /rank command evaluates multiple job postings in parallel by dispatching batches of approximately five jobs to general-purpose agents, which fetch each posting's full text, score it against a weighted four-dimension rubric, and return structured JSON results that are aggregated into a ranked shortlist.

The /rank command in the MadsLorentzen/ai-job-search repository serves as an automated triage system that bridges bulk job scraping and individual application preparation. By implementing batch scoring across parallel agents, it transforms raw scraped data from /job_scraper/seen_jobs.json into a prioritized shortlist using the evaluation framework defined in /.claude/skills/job-application-assistant/04-job-evaluation.md.

The Six-Step Batch Scoring Pipeline

According to the specification in /.claude/commands/rank.md (lines 0-117), the batch scoring process follows a strict pipeline that ensures reproducible evaluation of every newly scraped posting.

Step 0: Parse Input Arguments

The process begins by parsing command arguments to determine the evaluation scope. The system accepts --all to re-rank every unapplied posting, --top N to limit output, or a focus term to filter by keyword. By default, the command targets all entries marked new in /job_scraper/seen_jobs.json.

Step 1: Load State and Filter Jobs

The scoring engine reads two critical data stores:

• /job_scraper/seen_jobs.json: The database of scraped postings containing titles, company names, and URLs
• /job_search_tracker.csv: The exclusion set of already-applied jobs that must be omitted from evaluation

The system filters postings by status (new or --all) and optional focus terms, then loads the candidate profile from 01-candidate-profile.md and the evaluation rubric from 04-job-evaluation.md to prepare scoring context.

Step 2: Parallel Fetch and Score

This is the core batch operation. Jobs are split into batches of approximately five postings per parallel general-purpose agent. Each agent receives inline the job list (title, company, URL) and the complete scoring rubric.

For each posting, the agent:

1. WebFetches the URL to extract full posting text
2. Scores content across four dimensions (0-100 scale) based solely on the posting text
3. Marks dead or redirected URLs as expired
4. Returns a JSON object containing:
   • Dimension scores (technical, experience, behavioral, career)
   • Location verdict (PASS, FLAG, or FAIL)
   • Application deadline
   • Identified strengths and gaps
   • Detected language

The scoring strictly follows the honesty rule: agents extract gaps and strengths directly from posted text without manufacturing details or conducting external company research.

Step 3: Aggregate and Rank

The main process computes a weighted overall score using the formula defined in the evaluation framework:

overall_score = (technical * 0.30) + (experience * 0.25) + 
                (behavioral * 0.15) + (career * 0.30)

Jobs are mapped to verdict bands:

• Strong Fit: ≥ 75
• Good Fit: 60-74
• Moderate Fit: 45-59
• Weak Fit: 30-44
• Poor Fit: < 30

The location veto removes any job with a FAIL verdict from the shortlist regardless of numeric score. Deadline urgency (≤ 7 days) adds a 🔥 marker and serves as a tiebreaker in the final sort order.

Step 4: Update State

Results are persisted in-place to /job_scraper/seen_jobs.json, updating each entry with:

• rank_score: The weighted overall score
• rank_verdict: The fit category
• rank_date: Timestamp of evaluation
• status: Changed to ranked (or expired for dead links)

No modifications are made to /job_search_tracker.csv during this process.

Step 5: Present Shortlist

The command generates a markdown table displaying the top N jobs (controlled by --top), including scores, verdicts, deadline markers, and brief explanatory paragraphs. The output explicitly identifies these as triage scores derived only from posting text, inviting users to select promising candidates for the comprehensive /apply workflow.

The Four-Dimension Scoring Rubric

The evaluation framework in /.claude/skills/job-application-assistant/04-job-evaluation.md defines four quantified dimensions that agents must assess:

Technical Skills Evaluation

Scores range 0-100 based on alignment between required skills and the candidate's primary skill set. Strong matches where most required skills appear in [YOUR_PRIMARY_SKILLS] receive 80-100, while partial matches or heavy reliance on secondary skills score lower.

Experience Level Assessment

Direct experience in the specific role domain yields 80-100 points. Adjacent or transferable experience scores 60-79, while significant gaps in required experience domains result in lower scores.

Behavioral and Career Fit

Behavioral fit (0-100) measures culture alignment and working style compatibility. Career alignment (0-100) evaluates how energizing the role is for long-term trajectory. Together with technical and experience scores, these form the weighted composite.

Weighting Formula and Verdict Thresholds

The rubric applies fixed weights that prioritize technical capability and career trajectory equally:

Dimension	Weight
Technical	30%
Experience	25%
Behavioral	15%
Career	30%

Location assessment operates outside this weighted sum as a binary filter.

Handling Edge Cases and Constraints

Expired Posting Detection

When agents encounter HTTP errors or redirects during WebFetch operations, the job is immediately marked expired in the output JSON. These entries receive no scores and are excluded from ranking calculations but retained in the state file for record-keeping.

Location Veto Logic

Location rules operate as a hard constraint. If the posting requires relocation to a prohibited area or violates commute constraints defined in the candidate profile, the agent returns FAIL, triggering immediate exclusion from the shortlist regardless of perfect scores in other dimensions. FLAG statuses allow borderline cases through with warning indicators.

Urgency Detection and Tie-Breaking

The system extracts application deadlines from posting text. Jobs with deadlines within seven days receive a 🔥 urgency marker. When overall scores tie, these urgent postings rank higher in the final sorted list.

Practical Implementation Examples

Invoking Batch Scoring from CLI

# Rank the 5 newest unscraped jobs (default)

claude /rank

# Filter for data science roles and return top 10

claude /rank "data science" --top 10

# Re-evaluate all unapplied jobs after profile updates

claude /rank --all

Simulating the Scoring Loop

The following Python pseudocode illustrates the parallel dispatch pattern implemented in rank.md (lines 36-58):

import asyncio
import json
from pathlib import Path

async def score_job_batch(jobs, rubric):
    """Process ~5 jobs per parallel agent"""
    results = []
    for job in jobs:
        try:
            text = await webfetch(job['url'])
            results.append({
                "key": job['key'],
                "status": "scored",
                "scores": {
                    "technical": evaluate(text, rubric.technical),
                    "experience": evaluate(text, rubric.experience),
                    "behavioral": evaluate(text, rubric.behavioral),
                    "career": evaluate(text, rubric.career)
                },
                "location": assess_location(text, rubric),
                "deadline": extract_date(text)
            })
        except ConnectionError:
            results.append({"key": job['key'], "status": "expired"})
    return results

# Parallel execution across agent workers

jobs = load_new_jobs('/job_scraper/seen_jobs.json')
batches = chunk(jobs, size=5)
results = await asyncio.gather(*[
    score_job_batch(batch, rubric) for batch in batches
])

Aggregating Weighted Scores

This snippet demonstrates the aggregation logic from rank.md (lines 63-73):

WEIGHTS = {
    "technical": 0.30, 
    "experience": 0.25,
    "behavioral": 0.15, 
    "career": 0.30
}

def calculate_overall(scores):
    return sum(scores[k] * WEIGHTS[k] for k in WEIGHTS)

# Filter expired and apply location veto

valid_jobs = [
    j for j in results 
    if j['status'] != 'expired' and j['location'] != 'FAIL'
]

# Compute weighted scores

for job in valid_jobs:
    job['overall'] = calculate_overall(job['scores'])
    job['verdict'] = get_verdict_band(job['overall'])

# Sort by score desc, urgency desc

valid_jobs.sort(
    key=lambda x: (x['overall'], x.get('deadline_urgent', False)), 
    reverse=True
)

Summary

• Batch scoring in /rank evaluates multiple job postings simultaneously using parallel agents processing approximately five jobs each.
• Four dimensions (Technical 30%, Experience 25%, Behavioral 15%, Career 30%) are weighted to produce an overall 0-100 score mapped to verdict bands.
• Location vetos act as hard filters (FAIL removes jobs regardless of score), while deadline urgency (≤7 days) boosts ranking position.
• State persistence updates /job_scraper/seen_jobs.json with scores and verdicts, leaving /job_search_tracker.csv untouched.
• Honesty constraints require agents to score only from posting text without external research or manufactured details.

Frequently Asked Questions

How does the system handle rate limiting when fetching job postings in parallel?

The batch scoring architecture distributes WebFetch operations across multiple general-purpose agents, with each agent handling approximately five sequential fetches. This naturally throttles the request rate compared to a single process hitting dozens of URLs simultaneously. Dead or redirected URLs are gracefully handled by marking the job expired rather than retrying, preventing infinite loops on bad endpoints.

Why is location assessment treated as a binary veto rather than part of the weighted score?

Location constraints represent absolute deal-breakers—such as mandatory relocation to excluded regions or impossible commute distances—that override any qualitative fit. By implementing location as a separate PASS/FLAG/FAIL verdict rather than a weighted dimension, the system ensures that a job requiring cross-country relocation cannot score highly simply because the technical requirements match perfectly.

What happens to job postings with expired or redirected URLs during batch scoring?

When an agent encounters a non-200 HTTP status or connection error during WebFetch, the job is immediately marked with "status": "expired" in the returned JSON. The main process excludes these entries from ranking calculations and updates /job_scraper/seen_jobs.json to reflect the expired status, preventing future re-evaluation attempts while maintaining the record for audit purposes.

How does batch scoring in /rank differ from the individual evaluation in /apply?

Batch scoring provides a triage evaluation using only the posting text for rapid filtering of many opportunities, while /apply conducts comprehensive research including company background checks, salary benchmarking, and tailored application strategy. The /rank output explicitly notes that scores derive solely from posting content and invites users to run /apply for selected jobs to receive the full deep-dive analysis.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}