How CareerOps Defines Evaluation Logic and Scoring Weights

CareerOps calculates job fit scores by parsing job descriptions into six weighted dimension blocks (A–F) and a legitimacy tier (Block G), applying archetype-specific weights from modes/_profile.md and system defaults from templates/scoring.yml to generate a final 0–5 rating.

CareerOps is an open-source framework that brings systematic rigor to job offer evaluation. At its core, the tool relies on a configurable evaluation logic and scoring weights system that transforms unstructured job descriptions into quantitative fit scores. This architecture allows users to define precisely how dimensions like compensation, technology stack, and culture contribute to the final assessment based on their specific career archetype.

The Block-Based Evaluation Architecture

CareerOps segments every job description into six dimension blocks (A through F) plus a legitimacy tier (Block G). Each block represents a distinct evaluation dimension:

• Block A: Compensation and benefits
• Block B: Location and remote work flexibility
• Block C: Technology stack and engineering practices
• Block D: Leadership and management quality
• Block E: Culture and values alignment
• Block F: Growth trajectory and business impact
• Block G: Posting legitimacy and authenticity signals

During the extraction phase defined in modes/oferta.md, the system parses raw job description text into these structured blocks. Each block receives a confidence score between 0 and 1 representing the clarity and match strength of that dimension.

Where Evaluation Logic and Scoring Weights Are Defined

The scoring system separates user-specific configuration from system defaults across two primary files.

User Archetype Configuration in modes/_profile.md

The modes/_profile.md file serves as the user-layer configuration repository. It contains archetype definitions that specify which dimensions matter most for specific role types. For example, a data engineering archetype might weight Technology at 30% and Compensation at 25%, while minimizing Location weight for remote-first positions.

This file stores the active weight map that overrides system defaults during evaluation runs. When the pipeline executes via node evaluate.mjs or the oferta CLI mode, it first queries this profile to determine the current archetype's priorities.

System Default Schema in templates/scoring.yml

When modes/_profile.md does not specify a weight for a particular block, the system falls back to templates/scoring.yml. This system-layer YAML file defines the default weight schema used by modes/_shared.md and other shared prompts. The default allocation distributes importance across the six blocks as follows:

# Default weights from templates/scoring.yml

A: 0.20   # Compensation

B: 0.15   # Location

C: 0.25   # Technology Stack

D: 0.10   # Leadership / Management

E: 0.15   # Culture / Values

F: 0.15   # Growth / Impact

legitimacyMultiplier:
  high: 1.0
  medium: 0.8
  low: 0.5

The Calculation Pipeline

The evaluation pipeline processes extracted blocks through a weighted aggregation algorithm implemented in the core evaluation runner. The logic follows these steps:

1. Block Extraction: Parse the JD into blocks A–F using extractors defined in modes/*/oferta.md
2. Weight Resolution: Load archetype-specific weights from modes/_profile.md or fall back to templates/scoring.yml
3. Weighted Summation: Multiply each block's confidence score (0–1) by its corresponding weight percentage
4. Legitimacy Adjustment: Apply the Block G multiplier (high=1.0, medium=0.8, low=0.5) to the raw sum
5. Normalization: Convert the final value to a 0–5 scale for the generated report

This calculation is performed programmatically as shown in the following JavaScript implementation pattern:

// Example: Loading weights and calculating final score
import yaml from 'js-yaml';
import { readFileSync } from 'fs';

const profile = yaml.load(readFileSync('modes/_profile.md', 'utf8'));
const systemDefaults = yaml.load(readFileSync('templates/scoring.yml', 'utf8'));
const weights = profile.scoringWeights || systemDefaults;

let rawScore = 0;
for (const block of ['A','B','C','D','E','F']) {
  const confidence = blocks[block].confidence; // 0-1 from extractor
  rawScore += confidence * weights[block];
}

const finalScore = rawScore * legitimacyMultiplier; // 0-5 scale

Command-line execution automatically handles this logic:

# Run evaluation - weights loaded automatically based on active profile

career-ops oferta https://example.com/job-description

# Output includes: "Score: 4.2/5"

Customizing Scoring Weights for Different Archetypes

Users can override default evaluation logic by modifying the scoringWeights section in modes/_profile.md. This customization allows the same job description to yield different scores based on career priorities.

For instance, a candidate prioritizing work-life balance might configure:

# Custom archetype in modes/_profile.md

archetype: "work-life-priority"
scoringWeights:
  A: 0.15  # Lower compensation emphasis

  B: 0.30  # Higher location/flexibility weight

  C: 0.20
  D: 0.15
  E: 0.15
  F: 0.05  # Lower growth urgency

The front-end component web/src/components/score-methodology.tsx visualizes these breakdowns, showing which dimensions drove the score up or down based on the active archetype weights.

Summary

• CareerOps uses a six-block architecture (A–F) plus legitimacy tier (G) to structure job evaluation
• Scoring weights are defined hierarchically: user archetypes in modes/_profile.md override system defaults in templates/scoring.yml
• The calculation pipeline multiplies block confidence by weights, sums the products, and applies the Block G legitimacy multiplier
• Custom archetypes enable personalized evaluation logic without modifying core system files
• Final scores are rendered on a 0–5 scale via CLI reports and the React-based methodology viewer

Frequently Asked Questions

How does CareerOps resolve conflicts between user weights and system defaults?

The evaluation logic prioritizes the scoringWeights map defined in modes/_profile.md. If a specific block weight is missing from the user profile, the system falls back to the corresponding value in templates/scoring.yml. This ensures that incomplete custom configurations never break the scoring pipeline while allowing granular control over specific dimensions.

What data type does the legitimacy multiplier use, and how is it determined?

Block G represents a categorical multiplier rather than a continuous variable. According to templates/scoring.yml, the legitimacy tier maps to discrete float values: high authenticity receives a 1.0 multiplier, medium receives 0.8, and low receives 0.5. This tier is determined during the extraction phase by analyzing posting signals such as company verification status and description quality.

Can I add evaluation blocks beyond the standard A–F schema?

The current architecture in modes/_shared.md references a fixed set of six dimension blocks. Adding new blocks would require extending the schema in templates/scoring.yml, updating the extraction logic in modes/oferta.md, and modifying the calculation loop in the evaluation runner. The system is designed around these six dimensions to ensure consistency across archetypes.

How do I verify which weights were applied to a specific job score?

The evaluation report generated by evaluate.mjs includes a methodology breakdown. Additionally, the web/src/components/score-methodology.tsx component displays the active weight map and block-by-block contribution when viewing results in the web interface. For CLI debugging, inspect the modes/_profile.md file to confirm your archetype's current scoringWeights configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}