How AI Job Search Evaluates Job Fit: A Complete Guide to the Scoring Framework

The AI Job Search framework evaluates job fit using a three-stage scoring system that extracts job data, calculates a weighted percentage rating (0-100), and normalizes results to rank opportunities.

The AI Job Search framework, developed by MadsLorentzen, automates candidate-job matching through a structured evaluation pipeline. At its core lies the Job-Evaluation skill (04-job-evaluation.md), which transforms raw job postings into ranked recommendations based on candidate preferences defined in CLAUDE.md.

---

The Three-Stage Fit Evaluation Process

The framework processes every job posting through a standardized pipeline:

Stage 1: Data Extraction

The job scraper (job_scraper) converts unstructured postings into normalized CSV rows with standardized columns.

Key extracted fields include:

• title — job title and role designation
• location — geographic requirements or remote status
• skills — required and preferred competencies
• salary — compensation range
• seniority — experience level expectations

This normalization ensures consistent evaluation regardless of source format.

---

Stage 2: Fit-Rating Calculation

The fit rating (fit_rating) is computed as a weighted sum comparing extracted fields against the candidate profile.

The underlying formula follows this structure:

fit_rating = Σ (weight_i × match_i)

Where each field receives a configured weight and match score, producing a final percentage between 0–100.

This calculation is validated in tests/test_upskill_skill.py, specifically in test_step2_reads_ranked_jobs_with_moderate_fit_floor. The test suite confirms that derived ratings must be expressed as integer percentages (e.g., 72 rather than 0.72).

---

Stage 3: Normalization and Verdict Assignment

The upskill command transforms raw ratings into actionable rankings through two operations:

1. Score normalization: (100 - fit_rating) / 100
2. Verdict assignment: categorical labels (high, moderate, low)

Edge case handling is strict—rows with blank or non-numeric fit_rating values are skipped with a logged warning. This behavior is verified in test_step3_handles_blank_fit_rating in the same test file.

---

Running the Evaluation Pipeline

Rank Command: Generate Initial Scores

ai-job-search rank \
  --candidate-profile CLAUDE.md \
  --jobs-file scraped_jobs.csv \
  --output ranked_jobs.csv

This command reads the evaluation rules from /.claude/skills/job-application-assistant/04-job-evaluation.md, applies weighted scoring to each job, and outputs a ranked CSV.

---

Upskill Command: Normalize and Classify

ai-job-search upskill \
  --input ranked_jobs.csv \
  --output upskilled_jobs.csv

Applies the normalization formula and attaches textual verdicts. Jobs without valid numeric ratings are excluded from output.

---

Programmatic Inspection (Python)

import csv

with open('ranked_jobs.csv') as f:
    reader = csv.DictReader(f)
    for row in reader:
        print(f"{row['title']}: fit_rating = {row['fit_rating']}%")

Sample output:

Senior Data Engineer: fit_rating = 78%
Junior Analyst: fit_rating = 34%
Staff ML Engineer: fit_rating = 91%

---

Core Configuration Files

File	Purpose
CLAUDE.md	Candidate profile defining skill priorities, salary expectations, location preferences, and seniority targets
/.claude/skills/job-application-assistant/04-job-evaluation.md	Scoring framework specification—weights, thresholds, and field contribution logic
tests/test_upskill_skill.py	Unit tests validating fit-rating extraction, edge case handling, and normalization correctness
.claude/commands/rank.md	Command orchestration for the full evaluation pipeline

---

Summary

• The AI Job Search framework evaluates job fit through extraction, weighted scoring, and normalization
• Fit ratings are percentage values (0–100) derived from candidate-profile matching
• The upskill command transforms ratings into final rankings using (100 - fit_rating) / 100
• Invalid or missing ratings are skipped with warnings, ensuring data quality
• All logic is test-backed in test_upskill_skill.py and configuration-driven via CLAUDE.md and 04-job-evaluation.md

---

Frequently Asked Questions

What file stores my candidate preferences?

Your profile resides in CLAUDE.md at the repository root. This file defines your target skills, salary range, preferred locations, and seniority level, which serve as the baseline for all fit calculations.

How is the fit rating calculated from job data?

The framework applies weighted field matching defined in 04-job-evaluation.md. Each extracted field (title, skills, salary, etc.) receives a weight and match score; the sum produces a 0–100 percentage. This algorithm is validated by test_step2_reads_ranked_jobs_with_moderate_fit_floor.

What happens if a job posting lacks required data?

Jobs with blank or non-numeric fit_rating values are excluded from final rankings and flagged with a warning. The test_step3_handles_blank_fit_rating test confirms this behavior prevents corrupt data from affecting results.

Can I customize the weights used in scoring?

Yes. Modify /.claude/skills/job-application-assistant/04-job-evaluation.md to adjust field weights and thresholds. Changes apply immediately to subsequent rank command executions without code changes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}