# How CareerOps Defines Evaluation Logic and Scoring Weights

> Learn how CareerOps defines evaluation logic and scoring weights. Discover how job descriptions are parsed and weighted to calculate accurate job fit scores.

- Repository: [Santiago Fernández de Valderrama/career-ops](https://github.com/santifer/career-ops)
- Tags: internals
- Published: 2026-08-22

---

**CareerOps calculates job fit scores by parsing job descriptions into six weighted dimension blocks (A–F) and a legitimacy tier (Block G), applying archetype-specific weights from [`modes/_profile.md`](https://github.com/santifer/career-ops/blob/main/modes/_profile.md) and system defaults from [`templates/scoring.yml`](https://github.com/santifer/career-ops/blob/main/templates/scoring.yml) to generate a final 0–5 rating.**

CareerOps is an open-source framework that brings systematic rigor to job offer evaluation. At its core, the tool relies on a configurable evaluation logic and scoring weights system that transforms unstructured job descriptions into quantitative fit scores. This architecture allows users to define precisely how dimensions like compensation, technology stack, and culture contribute to the final assessment based on their specific career archetype.

## The Block-Based Evaluation Architecture

CareerOps segments every job description into **six dimension blocks** (A through F) plus a **legitimacy tier** (Block G). Each block represents a distinct evaluation dimension:

- **Block A**: Compensation and benefits
- **Block B**: Location and remote work flexibility
- **Block C**: Technology stack and engineering practices
- **Block D**: Leadership and management quality
- **Block E**: Culture and values alignment
- **Block F**: Growth trajectory and business impact
- **Block G**: Posting legitimacy and authenticity signals

During the extraction phase defined in [`modes/oferta.md`](https://github.com/santifer/career-ops/blob/main/modes/oferta.md), the system parses raw job description text into these structured blocks. Each block receives a confidence score between 0 and 1 representing the clarity and match strength of that dimension.

## Where Evaluation Logic and Scoring Weights Are Defined

The scoring system separates user-specific configuration from system defaults across two primary files.

### User Archetype Configuration in [`modes/_profile.md`](https://github.com/santifer/career-ops/blob/main/modes/_profile.md)

The [`modes/_profile.md`](https://github.com/santifer/career-ops/blob/main/modes/_profile.md) file serves as the user-layer configuration repository. It contains **archetype definitions** that specify which dimensions matter most for specific role types. For example, a data engineering archetype might weight **Technology** at 30% and **Compensation** at 25%, while minimizing **Location** weight for remote-first positions.

This file stores the active weight map that overrides system defaults during evaluation runs. When the pipeline executes via `node evaluate.mjs` or the `oferta` CLI mode, it first queries this profile to determine the current archetype's priorities.

### System Default Schema in [`templates/scoring.yml`](https://github.com/santifer/career-ops/blob/main/templates/scoring.yml)

When [`modes/_profile.md`](https://github.com/santifer/career-ops/blob/main/modes/_profile.md) does not specify a weight for a particular block, the system falls back to [`templates/scoring.yml`](https://github.com/santifer/career-ops/blob/main/templates/scoring.yml). This system-layer YAML file defines the **default weight schema** used by [`modes/_shared.md`](https://github.com/santifer/career-ops/blob/main/modes/_shared.md) and other shared prompts. The default allocation distributes importance across the six blocks as follows:

```yaml

# Default weights from templates/scoring.yml

A: 0.20   # Compensation

B: 0.15   # Location

C: 0.25   # Technology Stack

D: 0.10   # Leadership / Management

E: 0.15   # Culture / Values

F: 0.15   # Growth / Impact

legitimacyMultiplier:
  high: 1.0
  medium: 0.8
  low: 0.5

```

## The Calculation Pipeline

The evaluation pipeline processes extracted blocks through a weighted aggregation algorithm implemented in the core evaluation runner. The logic follows these steps:

1. **Block Extraction**: Parse the JD into blocks A–F using extractors defined in `modes/*/oferta.md`
2. **Weight Resolution**: Load archetype-specific weights from [`modes/_profile.md`](https://github.com/santifer/career-ops/blob/main/modes/_profile.md) or fall back to [`templates/scoring.yml`](https://github.com/santifer/career-ops/blob/main/templates/scoring.yml)
3. **Weighted Summation**: Multiply each block's confidence score (0–1) by its corresponding weight percentage
4. **Legitimacy Adjustment**: Apply the Block G multiplier (high=1.0, medium=0.8, low=0.5) to the raw sum
5. **Normalization**: Convert the final value to a 0–5 scale for the generated report

This calculation is performed programmatically as shown in the following JavaScript implementation pattern:

```javascript
// Example: Loading weights and calculating final score
import yaml from 'js-yaml';
import { readFileSync } from 'fs';

const profile = yaml.load(readFileSync('modes/_profile.md', 'utf8'));
const systemDefaults = yaml.load(readFileSync('templates/scoring.yml', 'utf8'));
const weights = profile.scoringWeights || systemDefaults;

let rawScore = 0;
for (const block of ['A','B','C','D','E','F']) {
  const confidence = blocks[block].confidence; // 0-1 from extractor
  rawScore += confidence * weights[block];
}

const finalScore = rawScore * legitimacyMultiplier; // 0-5 scale

```

Command-line execution automatically handles this logic:

```bash

# Run evaluation - weights loaded automatically based on active profile

career-ops oferta https://example.com/job-description

# Output includes: "Score: 4.2/5"

```

## Customizing Scoring Weights for Different Archetypes

Users can override default evaluation logic by modifying the `scoringWeights` section in [`modes/_profile.md`](https://github.com/santifer/career-ops/blob/main/modes/_profile.md). This customization allows the same job description to yield different scores based on career priorities.

For instance, a candidate prioritizing work-life balance might configure:

```yaml

# Custom archetype in modes/_profile.md

archetype: "work-life-priority"
scoringWeights:
  A: 0.15  # Lower compensation emphasis

  B: 0.30  # Higher location/flexibility weight

  C: 0.20
  D: 0.15
  E: 0.15
  F: 0.05  # Lower growth urgency

```

The front-end component [`web/src/components/score-methodology.tsx`](https://github.com/santifer/career-ops/blob/main/web/src/components/score-methodology.tsx) visualizes these breakdowns, showing which dimensions drove the score up or down based on the active archetype weights.

## Summary

- CareerOps uses a **six-block architecture** (A–F) plus legitimacy tier (G) to structure job evaluation
- **Scoring weights** are defined hierarchically: user archetypes in [`modes/_profile.md`](https://github.com/santifer/career-ops/blob/main/modes/_profile.md) override system defaults in [`templates/scoring.yml`](https://github.com/santifer/career-ops/blob/main/templates/scoring.yml)
- The **calculation pipeline** multiplies block confidence by weights, sums the products, and applies the Block G legitimacy multiplier
- **Custom archetypes** enable personalized evaluation logic without modifying core system files
- Final scores are rendered on a **0–5 scale** via CLI reports and the React-based methodology viewer

## Frequently Asked Questions

### How does CareerOps resolve conflicts between user weights and system defaults?

The evaluation logic prioritizes the `scoringWeights` map defined in [`modes/_profile.md`](https://github.com/santifer/career-ops/blob/main/modes/_profile.md). If a specific block weight is missing from the user profile, the system falls back to the corresponding value in [`templates/scoring.yml`](https://github.com/santifer/career-ops/blob/main/templates/scoring.yml). This ensures that incomplete custom configurations never break the scoring pipeline while allowing granular control over specific dimensions.

### What data type does the legitimacy multiplier use, and how is it determined?

Block G represents a categorical multiplier rather than a continuous variable. According to [`templates/scoring.yml`](https://github.com/santifer/career-ops/blob/main/templates/scoring.yml), the legitimacy tier maps to discrete float values: **high** authenticity receives a 1.0 multiplier, **medium** receives 0.8, and **low** receives 0.5. This tier is determined during the extraction phase by analyzing posting signals such as company verification status and description quality.

### Can I add evaluation blocks beyond the standard A–F schema?

The current architecture in [`modes/_shared.md`](https://github.com/santifer/career-ops/blob/main/modes/_shared.md) references a fixed set of six dimension blocks. Adding new blocks would require extending the schema in [`templates/scoring.yml`](https://github.com/santifer/career-ops/blob/main/templates/scoring.yml), updating the extraction logic in [`modes/oferta.md`](https://github.com/santifer/career-ops/blob/main/modes/oferta.md), and modifying the calculation loop in the evaluation runner. The system is designed around these six dimensions to ensure consistency across archetypes.

### How do I verify which weights were applied to a specific job score?

The evaluation report generated by `evaluate.mjs` includes a methodology breakdown. Additionally, the [`web/src/components/score-methodology.tsx`](https://github.com/santifer/career-ops/blob/main/web/src/components/score-methodology.tsx) component displays the active weight map and block-by-block contribution when viewing results in the web interface. For CLI debugging, inspect the [`modes/_profile.md`](https://github.com/santifer/career-ops/blob/main/modes/_profile.md) file to confirm your archetype's current `scoringWeights` configuration.