Career-Ops Pattern Analysis Script: How It Detects ATS Channel Performance

The analyze-patterns.mjs script in the santifer/career-ops repository parses application trackers and evaluation reports to identify which ATS vendors (Greenhouse, Lever, Ashby, Workday) function as "dead channels" with statistically lower advance rates, enabling data-driven decisions about application routing.

The pattern analysis script serves as the rejection-pattern detector for the career-ops pipeline. Located at analyze-patterns.mjs in the repository root, it aggregates data from data/applications.md and linked evaluation reports to produce actionable intelligence about hiring funnel performance.

What Is the Pattern Analysis Script?

analyze-patterns.mjs functions as the core analytics engine for the career-ops workflow. It reads the applications tracker (data/applications.md), loads corresponding markdown evaluation reports from the reports/ directory, and extracts structured data including scores, archetypes, remote policies, and technical gaps.

The script executes a seven-stage pipeline:

1. Parse tracker – parseTracker() converts markdown rows into JavaScript objects using utilities from tracker-parse.mjs [line 84-95].
2. Load reports – parseReport(reportPath) extracts Machine Summary blocks, scoring tables, and gap tables from individual evaluation reports [line 98-135].
3. Enrich entries – Each row is enhanced with normalized status, classified outcome, remote bucket classification, company size inference, and ATS vendor detection via detectVendor() [line 36-38, 66-75].
4. Aggregate statistics – Builds conversion funnels, archetype breakdowns, blocker analysis, and ATS channel analysis [line 46-86].
5. Detect dead channels – Identifies underperforming ATS vendors using statistical thresholds.
6. Generate recommendations – Produces actionable suggestions based on aggregated data [line 124-176].
7. Output results – Emits a JSON payload containing metadata, funnel, vendorAnalysis, and recommendations [line 122-133].

How ATS Channel Analysis Works in analyze-patterns.mjs

The ATS channel analysis implementation evaluates whether specific Applicant Tracking System vendors create algorithmic bottlenecks, following research from Bommasani et al., "Algorithmic Monocultures in Hiring", FAccT 2026.

Vendor Detection from Report URLs

The detectVendor(url) function (lines 60-75) creates a URL object from the report's stored URL and matches the hostname against VENDOR_HOST_PATTERNS. This pattern map identifies four community ATS platforms:

• Greenhouse (greenhouse.io domains)
• Lever (lever.co domains)
• Ashby (ashby.io domains)
• Workday (workday.com domains)

Each enriched entry receives a vendor property: e.vendor = detectVendor(reportData?.url) [line 29-38].

Aggregating Submission Metrics

After enrichment, the script filters for submitted entries (statuses: applied, responded, interview, offer, rejected, or discarded) and reduces them into a vendorMap [line 71-78]. For each vendor, it tracks:

• total: Total applications submitted through this channel
• advanced: Count reaching responded, interview, or offer status

The script calculates advanceRate = advanced / total * 100 and sharePct (percentage of total submissions) for each vendor [line 84-92].

Statistical Validation and Dead Channel Detection

The analysis implements rigorous statistical guards before making claims:

Minimum Sample Size: The constant MIN_VENDOR_N (default: 8) ensures sufficient data before analysis. Vendors below this threshold receive an "n too small for a claim" designation [line 80-92].

Dead Channel Identification: A vendor qualifies as a dead channel if it meets three criteria simultaneously [line 99-108]:

1. Represents ≥ 25% of total submissions (sharePct >= 25)
2. Meets minimum sample size (sufficientSample: true)
3. Advance rate is ≥ 10 percentage points lower than the leave-one-out rate of all other channels combined

When detected, the script recommends routing those companies through referral or direct contact rather than the standard application portal.

Implementing the Analysis Pipeline

Running the Pattern Analysis Script

Execute the script from the repository root to generate either JSON output or human-readable summaries:

# JSON output (default)

node analyze-patterns.mjs > pattern-report.json

# Human-readable table with ATS channel analysis

node analyze-patterns.mjs --summary

Both commands read data/applications.md and all reports under reports/, then emit the complete result structure including vendorAnalysis.

Accessing ATS Channel Data Programmatically

Import the analysis module to access structured vendor data in JavaScript:

import analyze from './analyze-patterns.mjs';

const result = await analyze();
const { vendorAnalysis } = result;

// Iterate vendor performance
vendorAnalysis.breakdown.forEach(v => {
  console.log(`${v.vendor}: ${v.total} apps, ${v.advanceRate}% advance rate`);
});

// Check for dead channels
const deadChannels = vendorAnalysis.breakdown.filter(v => 
  v.sufficientSample && 
  v.sharePct >= 25 && 
  v.advanceRate < vendorAnalysis.overallAdvanceRate - 10
);

if (deadChannels.length > 0) {
  console.log(`Dead channels detected: ${deadChannels.map(c => c.vendor).join(', ')}`);
}

Understanding the vendorAnalysis Output

The JSON structure provides comprehensive ATS channel metrics:

{
  "vendorAnalysis": {
    "scope": ["greenhouse", "lever", "ashby", "workday"],
    "minSampleForClaim": 8,
    "submitted": 42,
    "identified": 35,
    "coveragePct": 83,
    "overallAdvanceRate": 38,
    "breakdown": [
      {
        "vendor": "greenhouse",
        "total": 18,
        "advanced": 4,
        "advanceRate": 22,
        "sharePct": 43,
        "sufficientSample": true
      },
      {
        "vendor": "lever",
        "total": 9,
        "advanced": 7,
        "advanceRate": 78,
        "sharePct": 21,
        "sufficientSample": true
      }
    ],
    "citation": "Bommasani et al., Algorithmic Monocultures in Hiring, FAccT 2026 (arXiv:2605.27371)"
  }
}

In this example, Greenhouse holds 43% of submissions but only a 22% advance rate—significantly below the 38% overall average. With sufficient sample size and high share, the script flags this as a dead channel requiring alternative routing strategies.

Summary

• analyze-patterns.mjs serves as the central rejection-pattern detector for the career-ops pipeline, parsing data/applications.md and linked evaluation reports.
• ATS channel analysis detects vendor-specific bottlenecks by parsing report URLs to identify Greenhouse, Lever, Ashby, and Workday submissions.
• Dead channel detection requires ≥ 25% submission share, ≥ 8 sample size, and ≥ 10 percentage point lower advance rates than other channels combined.
• The script outputs a vendorAnalysis object containing breakdown statistics, coverage percentages, and actionable recommendations for routing applications away from underperforming ATS platforms.
• All analysis references the algorithmic monoculture research from Bommasani et al. (FAccT 2026) to ensure channel yield evaluation rather than discriminatory filtering.

Frequently Asked Questions

What ATS vendors does the pattern analysis script detect?

The script detects four community ATS vendors: Greenhouse, Lever, Ashby, and Workday. The detectVendor() function in analyze-patterns.mjs matches report URLs against VENDOR_HOST_PATTERNS to identify these platforms based on their canonical hostnames (e.g., greenhouse.io, lever.co).

How does the script determine if an ATS channel is "dead"?

A channel qualifies as dead when it accounts for at least 25% of total submissions, meets the minimum sample size of 8 applications, and shows an advance rate at least 10 percentage points lower than the aggregated rate of all other channels. This calculation uses leave-one-out analysis to compare each vendor against the rest of the pipeline.

What is the minimum sample size required for ATS channel claims?

The constant MIN_VENDOR_N defaults to 8 submissions. Vendors with fewer than 8 applications receive an "n too small for a claim" designation, preventing statistically unreliable conclusions from small sample sets.

Where does the script get the URLs for vendor detection?

The script extracts URLs from the Machine Summary blocks within individual markdown evaluation reports stored in the reports/ directory. During the enrichment phase, parseReport() reads each report file referenced in the tracker, and detectVendor() processes the URL field to determine the ATS provider.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}