# How merge-tracker.mjs Handles TSV Tracker Additions and Link Normalization in Career Ops

> Discover how merge-tracker.mjs integrates TSV tracker additions and normalizes links for Career Ops. Learn about duplicate detection and atomic merging.

- Repository: [Santiago Fernández de Valderrama/career-ops](https://github.com/santifer/career-ops)
- Tags: internals
- Published: 2026-06-12

---

**The `merge-tracker.mjs` script processes pending TSV files from `batch/tracker-additions/`, normalizes report links to be relative to the tracker directory using `normalizeReportLink()`, detects duplicates via report numbers and fuzzy matching, and atomically merges valid entries into [`applications.md`](https://github.com/santifer/career-ops/blob/main/applications.md) while skipping or updating existing records based on score comparison.**

The `merge-tracker.mjs` utility in the `santifer/career-ops` repository automates the integration of batch-generated TSV tracker additions into the master applications tracker. This defensive pipeline supports multiple column layouts, validates canonical status values against a predefined set, and ensures that Markdown report links remain functional whether the tracker resides at the repository root or under `data/` (addressing issue #760).

## Discovering Pending TSV Files

The script begins by scanning the dedicated additions directory for pending batch files. Using `readdirSync` on `batch/tracker-additions/`, it filters for files ending with the `.tsv` extension:

```javascript
const tsvFiles = readdirSync(ADDITIONS_DIR).filter(f => f.endsWith('.tsv'));

```

If no files are detected, the script exits early with a confirmation message (`✅ No pending additions to merge.`). When files are present, the script acquires an exclusive filesystem lock to prevent race conditions during the merge process.

## Parsing TSV Content with Multiple Format Support

The `parseTsvContent(content, filename)` function (lines 65–86) handles three distinct input formats flexibly, allowing contributors to submit data in whatever format they generated.

### Handling Pipe-Delimited Markdown Tables

When the content starts with a pipe character (`|`), the parser recognizes it as a Markdown table row:

```javascript
if (content.startsWith('|')) {
  // Split on pipe delimiter and trim whitespace
  const columns = content.split('|').map(c => c.trim()).filter(c => c);
}

```

This detection occurs at lines 77–84, enabling direct copying from rendered Markdown tables into TSV files.

### Tab-Separated Value Processing

For standard TSV files, the parser splits on tab characters:

```javascript
const columns = content.split('\t');

```

The function gracefully handles malformed rows by emitting warnings and returning `null` (lines 79–82 and 99–101), preventing a single bad row from crashing the entire batch.

### Column Order Detection and Validation

The script uses heuristics on columns 4 and 5 to determine whether they represent *status-score* or *score-status* ordering (lines 103–126). Once detected, the status value undergoes normalization through `validateStatus()` (lines 26–54), which ensures only canonical values from the predefined set (lines 24–44) are accepted. Scores are extracted and later parsed by `parseScore()` (lines 48–51) for numerical comparison.

## Normalizing Report Links for Cross-Directory Compatibility

A critical function of the merger is ensuring report links work regardless of the tracker's location. Raw TSV entries contain root-relative paths (e.g., [`reports/123-acme-senior-engineer.md`](https://github.com/santifer/career-ops/blob/main/reports/123-acme-senior-engineer.md)), but the tracker may reside in [`data/applications.md`](https://github.com/santifer/career-ops/blob/main/data/applications.md) or [`./applications.md`](https://github.com/santifer/career-ops/blob/main/./applications.md).

The script imports the normalization utility from `tracker-links.mjs`:

```javascript
import { normalizeReportLink as normalizeLink } from './tracker-links.mjs';

const normalizeReportLink = (reportField) => normalizeLink(reportField, TRACKER_DIR, REPORTS_ROOT);

```

During the merge loop, each addition's report field is rewritten before duplicate detection:

```javascript
addition.report = normalizeReportLink(addition.report);

```

This transformation (lines 31–35) converts root-relative paths to absolute paths, then re-relativizes them to `TRACKER_DIR`. The result is a properly relative link like `[123](../reports/123-acme-senior-engineer.md)` when the tracker lives in `data/`, or `[123](reports/123-acme-senior-engineer.md)` when at the repo root.

## Duplicate Detection and Scoring Logic

After normalization, the script implements a three-tier strategy to identify existing entries (lines 36–70).

### Three-Tier Duplicate Finding Strategy

1. **Exact report number match**: Extracts the number from the Markdown link using `extractReportNum()` and searches for existing rows with identical numbers (lines 39–48).

2. **Exact tracker-row number and company match**: Guards against numbering mismatches across different companies by checking both the row number and normalized company name (lines 50–61).

3. **Fuzzy company-role match**: Uses `roleFuzzyMatch()` (lines 17–41) after normalizing company names via `normalizeCompany()` (line 57) to catch entries where the report number may have changed but the company and role combination represents the same application.

### Score-Based Merge Decisions

When duplicates are found, the script compares scores using `parseScore()`:

- **Higher score**: Triggers an update (lines 72–84), rewriting the row while preserving the existing status and PDF flag, and appending a note indicating the re-evaluation.
- **Lower or equal score**: The addition is skipped (lines 85–87) to prevent stale data from overwriting fresher evaluations.

For entirely new entries, the script inserts the row using either the TSV-provided number or the next sequential available number (lines 88–99). All actions are logged with emojis (`➕ Add`, `🔄 Update`, `⏭️ Skip`) for immediate visual feedback.

## Atomic Writes and File Management

When not running in `--dry-run` mode, the script writes the final tracker content using `writeFileAtomic()` (lines 86–96), preventing partial writes that could corrupt the tracker during system interruptions.

Successfully processed TSV files are moved to `batch/tracker-additions/merged/` (lines 19–24), ensuring they are not re-processed in subsequent runs. This housekeeping maintains the integrity of the batch processing queue.

## Summary

- **Discovery**: Scans `batch/tracker-additions/` for `.tsv` files and exits early if none exist.
- **Parsing**: Handles pipe-delimited Markdown, tab-separated values, and variable column ordering while validating canonical status values.
- **Normalization**: Rewrites report links using `normalizeReportLink()` to ensure they remain functional relative to the tracker's location.
- **Deduplication**: Employs three strategies—report number matching, tracker-row matching, and fuzzy company-role matching—to identify existing entries.
- **Merging**: Updates existing entries when the new score is higher, adds new entries when no match exists, and skips when the score is lower or equal.
- **Safety**: Uses atomic file writes and exclusive locks to prevent data corruption during concurrent operations.

## Frequently Asked Questions

### How does merge-tracker.mjs handle different TSV column formats?

The `parseTsvContent()` function automatically detects whether the input uses pipe-delimited Markdown tables (starting with `|`) or standard tab-separated values. It then applies heuristics to columns 4 and 5 to determine if they represent status-score or score-status ordering, ensuring flexibility regardless of how the TSV was generated.

### What happens if a TSV file contains an invalid status value?

The script calls `validateStatus()` (lines 26–54) which normalizes input against the canonical status set defined in the source code. Invalid status values trigger validation errors before the merge proceeds, preventing malformed data from entering the master tracker.

### How does the script prevent duplicate application entries?

The script checks for duplicates using three methods: exact report number matching via `extractReportNum()`, exact tracker-row number combined with normalized company name, and fuzzy matching of company-role pairs using `roleFuzzyMatch()`. When duplicates are found, it compares scores using `parseScore()` and only updates if the new entry has a higher score.

### Why are report links normalized, and how does it work?

Report links are normalized to ensure they work correctly whether the tracker file is at the repository root or under `data/`. The `normalizeReportLink()` function (utilizing `tracker-links.mjs`) converts root-relative paths to paths relative to the tracker's directory, addressing issue #760 and ensuring clickable links in all repository layouts.