# Troubleshooting Common hiring-agent Issues: 5-Stage Pipeline Fixes

> Resolve common hiring-agent issues like PDF extraction errors and LLM misconfiguration. Discover 5-stage pipeline fixes for efficient troubleshooting. Read now for solutions.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: how-to-guide
- Published: 2026-07-12

---

**Most hiring-agent failures stem from PDF extraction errors, LLM misconfiguration, or stale development caches in the five-stage pipeline spanning [`pymupdf_rag.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pymupdf_rag.py) to [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py).**

The interviewstreet/hiring-agent repository transforms resume PDFs into structured evaluations through a multi-stage LLM pipeline. When the pipeline breaks, symptoms often appear far from the root cause in [`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py) or [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py). Understanding the architectural flow—from PyMuPDF extraction to fairness-aware scoring—lets you isolate failures quickly.

## PDF Extraction Failures in pymupdf_rag.py

The pipeline begins in [`pymupdf_rag.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pymupdf_rag.py), where the **PDFExtractor** class converts documents to Markdown-like text. Failures here cascade downstream, causing malformed section parsing or empty LLM prompts.

### Corrupt or Unsupported PDF Formats

If you encounter `RuntimeError: cannot open file` in [`pymupdf_rag.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pymupdf_rag.py), the PDF is likely corrupted or uses an unsupported spec. Verify the file opens in a standard viewer, then convert it to a stable PDF/A format using `pdftk` or similar tools. Ensure the file path is absolute and the process has read permissions.

### Missing Text from Custom Fonts

When headings or tables disappear from the generated Markdown, the PDF likely uses custom fonts or image-based text that PyMuPDF cannot interpret. Enable image extraction and OCR fallback:

```python
from pymupdf_rag import PDFExtractor

pdf_path = "resume.pdf"
try:
    extractor = PDFExtractor(pdf_path, extract_images=True)
    markdown = extractor.to_markdown()
    print(f"Extracted {len(markdown)} characters")
except Exception as e:
    raise RuntimeError(f"Failed to open PDF: {e}")

```

Adjust the `to_markdown` helper in [`pymupdf_rag.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pymupdf_rag.py) to treat non-standard heading markers as level-2 headings if the document uses unusual typography.

## LLM Provider Configuration Errors

The **LLM provider layer** in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) and [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) bridges the pipeline to Gemini, Ollama, or other backends. Misconfiguration here produces `AuthenticationError` or empty JSON responses that break the [`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py) section parser.

### Authentication Failures with Gemini or Ollama

Check that `.env` contains valid variables before activating the virtual environment:

```bash

# .env

LLM_PROVIDER=gemini
DEFAULT_MODEL=gemini-1.5-flash
GEMINI_API_KEY=your_key_here

```

Reload the environment with `source .venv/bin/activate` and verify connectivity:

```python
from llm_utils import get_provider
provider = get_provider()
print(f"Using provider: {provider.__class__.__name__}")

```

### Malformed JSON from Template Mismatches

When [`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py) returns unexpected JSON structures, the Jinja templates in `prompts/templates/` likely diverge from the LLM's expected schema. Reinstall dependencies to ensure Jinja2 compatibility, then diff your local templates against the repository's `README` examples.

## GitHub Enrichment Failures in github.py

The **GitHubClient** class in [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py) fetches profile data and selects the top 7 repositories via LLM-driven analysis. This stage fails silently or with `KeyError` when API limits or schema changes occur.

### Rate Limiting and Missing Tokens

If no repositories appear despite a valid username in the resume, check for a missing `GITHUB_TOKEN` or rate-limit exhaustion. Create a personal access token with `repo` scope, export it in `.env`, and wait for GitHub's hourly rate limit reset if you've exceeded 60 unauthenticated requests.

### API Response Structure Changes

A `KeyError: 'login'` indicates GitHub modified their API response format. Update [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py) to use the new field name or fallback to `html_url`:

```python
from github import GitHubClient

client = GitHubClient()
repos = client.get_user_repos("octocat")
print(f"Fetched {len(repos)} repositories")

```

Run the repository-fetching unit tests in `tests/` to confirm the response shape matches the expected Pydantic models.

## Evaluation and Scoring Anomalies in evaluator.py

The **Evaluator** class in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) applies fairness-aware scoring rules using LLM-generated evidence. Template corruption or logic drift causes missing categories or incorrect calculations.

### Missing Scoring Categories

If "open_source" or "production" categories disappear from the output, the Jinja template (`resume_evaluation_criteria.jinja`) was edited incorrectly. Restore the original template from version control and run a dry evaluation:

```python
from evaluator import Evaluator

evaluator = Evaluator()
result = evaluator.evaluate(json_resume, dry_run=True)
print(result.scores)

```

### Incorrect Bonus Calculations

When bonus points or deductions seem mathematically wrong, compare the current [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) against the `main` branch. The fairness constraints for `bonus_points` and `deductions` may have been altered, requiring you to re-apply the original weighting logic.

## Development Mode Caching Issues

When `DEVELOPMENT_MODE=True` in [`config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/config.py), the pipeline caches intermediate JSON and appends results to `resume_evaluations.csv` via [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py). This creates two distinct failure modes.

### Stale Cache Artifacts

If results appear outdated despite PDF changes, delete cached files to force reprocessing:

```python
import glob, os

for path in glob.glob("cache/resumecache_*.json"):
    os.remove(path)
print("Cache cleared")

```

Set `DEVELOPMENT_MODE=False` in [`config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/config.py) for production runs that bypass the cache entirely.

### CSV Export Permission Errors

A `PermissionError` when writing to `resume_evaluations.csv` indicates file-system restrictions. Change permissions with `chmod 664 resume_evaluations.csv` or execute the script with a user that has write access to the working directory.

## Summary

- **PDF extraction** fails in [`pymupdf_rag.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pymupdf_rag.py) due to corrupted files or non-standard fonts—verify files and enable OCR when needed.
- **LLM configuration** requires correct `.env` variables (`LLM_PROVIDER`, `GEMINI_API_KEY`) and matching Jinja templates in `prompts/templates/`.
- **GitHub enrichment** needs a valid `GITHUB_TOKEN` to avoid rate limits and may require field name updates if the API changes.
- **Evaluation anomalies** in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) stem from corrupted templates or modified fairness logic—use `dry_run=True` to debug.
- **Development mode** caches results in `cache/` and writes to CSV; clear caches and check permissions when results seem stale.

## Frequently Asked Questions

### Why does hiring-agent return empty JSON for some resume sections?

Empty JSON usually indicates the LLM provider timed out or the prompt template in `prompts/templates/` references a deprecated schema. Verify your `LLM_PROVIDER` and `DEFAULT_MODEL` environment variables, then compare your local templates against the repository's `README` examples to ensure field alignment.

### How do I fix GitHub API rate limits when processing multiple candidates?

Create a personal access token with `repo` scope and set it as `GITHUB_TOKEN` in your `.env` file. Without authentication, GitHub limits you to 60 requests per hour; with a token, the limit increases to 5,000 requests per hour. If you hit the limit, wait for the hourly reset or implement exponential backoff in [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py).

### What causes incorrect scoring calculations in the final evaluation?

Incorrect calculations typically result from manual edits to [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) that altered the `bonus_points` or `deductions` logic, or from a corrupted `resume_evaluation_criteria.jinja` template. Run `evaluator.py --dry-run` (or call `evaluate()` with `dry_run=True`) to inspect intermediate scores, then restore the original template from the `main` branch if categories are missing.

### How do I clear the development cache to force a fresh resume analysis?

Delete the JSON files in the `cache/` directory or set `DEVELOPMENT_MODE=False` in [`config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/config.py). For a one-time reset, run `rm cache/*.json` before executing [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py). This ensures PyMuPDF re-extracts the PDF and the LLM reprocesses all sections rather than loading stale intermediate results.