# How to Handle LLM Parsing Failures in Project Selection: Fallback Mechanisms in the Hiring Agent

> Learn about LLM parsing fallback mechanisms in hiring-agent. Discover how the system ensures pipeline continuity by returning the first 7 projects when LLM parsing fails.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: how-to-guide
- Published: 2026-07-19

---

**When the LLM fails to parse project selection responses, the hiring-agent implements a two-layered fallback strategy that returns the first 7 projects from the original list to ensure pipeline continuity.**

The `interviewstreet/hiring-agent` repository relies on large language models to rank and select top GitHub projects for technical interviews. When **LLM parsing failures** occur during project selection, the system cannot afford to crash or return empty results, so it employs defensive programming patterns in [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py) to guarantee a deterministic output of exactly 7 unique projects even when the model returns malformed JSON or encounters network errors.

## Two-Layered Fallback Strategy for LLM Parsing Errors

The codebase implements a defensive hierarchy that catches failures at different levels of the LLM invocation stack. Both layers converge on the same safe default: returning the first 7 projects from the original input list.

### JSON Parsing Fallback (github.py lines 31–35)

After receiving the LLM response, the code attempts to clean the text and parse it as structured JSON using `json.loads()`. If the model outputs malformed JSON—such as missing brackets, trailing commas, or explanatory text outside the JSON structure—the `json.JSONDecodeError` exception triggers an immediate fallback.

In this exception block, the system logs the parsing error and returns `projects[:7]`, preserving the downstream contract while discarding the unparsable LLM output. This ensures that even when the **LLM fails to parse project selection responses** into valid JSON, the hiring workflow continues uninterrupted.

### General Exception Fallback (github.py lines 38–41)

Beyond JSON parsing errors, a broader `except` clause catches any other failure modes including network timeouts, authentication errors with the LLM provider, or unexpected response formats. This outer exception handler serves as the ultimate safety net.

When triggered, it logs the specific error details and similarly returns the first 7 projects from the original list. This dual-layer approach ensures that both data-format failures and infrastructure failures result in graceful degradation rather than pipeline termination.

## Implementation Details in github.py

The core selection logic resides in [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py), where the `select_top_projects()` function orchestrates the LLM call and fallback handling. The function first prepares the chat parameters using templates from [`prompts/template_manager.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompts/template_manager.py), then invokes the provider through utilities defined in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py).

Here is the simplified flow demonstrating the exception handling:

```python
def select_top_projects(projects):
    try:
        # Prepare prompt and call LLM via provider

        response = provider.chat(**chat_params)
        response_text = response["message"]["content"]
        
        # Attempt to extract and parse JSON from LLM output

        response_text = extract_json_from_response(response_text.strip())
        selected = json.loads(response_text)  # May raise JSONDecodeError

        # ... deduplicate and validate exactly 7 unique items ...

        
    except json.JSONDecodeError as e:
        # JSON could not be parsed → fallback to first 7 original projects

        print(f"ERROR: {e}")
        return projects[:7]
        
    except Exception as e:
        # Network errors, timeouts, or other failures → same fallback

        print(f"Error using LLM: {e}")
        return projects[:7]

```

This implementation guarantees that the function always returns a list object, preventing `None` or empty results from propagating to downstream evaluation stages.

## The Seven-Project Contract

The fallback mechanism hardcodes a return of exactly **7 unique projects** because the downstream evaluation pipeline expects this specific cardinality. By slicing `projects[:7]`, the system maintains interface consistency regardless of LLM availability or output quality.

When the LLM succeeds, the code deduplicates its selections and validates the count; when it fails, the deterministic slice ensures the contract holds. This design choice reflects a **resilient architecture** that prioritizes workflow completion over perfect LLM execution.

## Supporting Files and Infrastructure

While [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py) contains the fallback logic, related components handle the LLM interaction setup:

- **[`prompts/template_manager.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompts/template_manager.py)** – Loads the `github_project_selection.jinja` template that structures the prompt asking the LLM to rank candidate projects. Poorly formatted LLM outputs often stem from template ambiguity, making this file relevant to parsing failure prevention.

- **[`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py)** – Provides `initialize_llm_provider()` and model-parameter utilities that the selection step relies on. Configuration errors in this module could trigger the general exception fallback in [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py).

## Summary

- **Two-layered defense**: The system catches `JSONDecodeError` specifically, then uses a general `except` clause for all other failures.
- **Deterministic fallback**: Both error paths return `projects[:7]` from the original list, ensuring exactly 7 projects are always available.
- **Source location**: All fallback logic is implemented in [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py) between lines 31 and 41.
- **Contract preservation**: The 7-project guarantee prevents downstream pipeline failures regardless of LLM reliability.

## Frequently Asked Questions

### What happens when the LLM returns malformed JSON during project selection?

The code catches the `JSONDecodeError` in the exception block at lines 31–35 of [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py), logs the error, and immediately returns the first 7 projects from the original input list. This prevents the malformed output from crashing the selection pipeline.

### How does the hiring-agent handle network timeouts or LLM provider failures?

Network timeouts and provider errors are caught by the general `except` clause at lines 38–41 of [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py). Like the JSON parsing fallback, this handler logs the error and returns the first 7 projects, ensuring the hiring workflow continues even when the LLM service is unavailable.

### Why does the fallback return exactly 7 projects?

The downstream evaluation pipeline requires exactly 7 unique projects to maintain interface contracts. Returning `projects[:7]` provides a deterministic, safe default that satisfies this requirement without requiring additional validation logic or empty-state handling in subsequent processing steps.

### Which file manages the prompts that might cause parsing errors?

The [`prompts/template_manager.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompts/template_manager.py) file loads the `github_project_selection.jinja` template used to request ranked projects from the LLM. While this file doesn't handle the fallback logic directly, poorly structured prompts can increase the likelihood of receiving unparsable responses, making template management critical to reducing fallback triggers.