# How evaluator.py Handles Compilation Errors in InterviewStreet's Hiring Agent

> Learn how evaluator.py handles compilation errors in InterviewStreet's Hiring Agent. It uses try-except blocks, logs errors with timestamps, and re-raises exceptions for effective upstream management.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: how-to-guide
- Published: 2026-07-15

---

**The evaluator.py module wraps the LLM evaluation sequence in a try-except block, logs the error with a timestamped message, and re-raises the exception for upstream handling.**

The [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) file in the open-source interviewstreet/hiring-agent repository orchestrates résumé evaluation by calling large language model (LLM) providers. Since this module does not perform traditional code compilation, compilation-like errors actually manifest as runtime exceptions during prompt construction or response parsing. Understanding how [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) handles these errors reveals the system's defensive programming patterns and safeguards against LLM provider failures.

## The Central Error Handling Pattern

The module employs a unified exception management strategy that treats all runtime failures—whether network issues, JSON parsing errors, or validation failures—as critical errors requiring immediate logging and upstream delegation.

### The Try-Except Wrapper (Lines 89-91)

In [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py), the entire LLM invocation sequence is protected by a broad exception handler. The code that builds the prompt, sends the request to the LLM provider, extracts JSON from the response, and constructs an `EvaluationData` instance is wrapped in a `try … except Exception as e:` clause spanning lines 89 through 91.

```python
def evaluate_resume(self, resume_text: str) -> EvaluationData:
    try:
        # … build prompts, call LLM provider, extract JSON …

        evaluation_dict = json.loads(response_text)
        evaluation_data = EvaluationData(**evaluation_dict)
        return evaluation_data
    except Exception as e:
        # All unexpected problems (including “compilation” errors) land here

        logger.error(f"Error evaluating resume: {str(e)}")
        raise                      # Propagate upward for higher‑level handling

```

### Logging Strategy

When an exception occurs, the handler immediately logs the failure using a timestamped error message. The code calls `logger.error(f"Error evaluating resume: {str(e)}")` to record the exact exception message. This approach ensures operators can locate failures in application logs without exposing sensitive stack traces to end-users.

### Exception Propagation

After logging, the handler re-raises the exception using a bare `raise` statement. This propagates the failure up the call stack, allowing the API layer or orchestration function to implement retry logic, fallback mechanisms, or user-friendly error messages according to the specific error type.

## Runtime Exceptions That Mimic Compilation Errors

Since [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) does not compile code, errors that resemble compilation failures occur during data transformation and validation phases. The following scenarios trigger the exception handler:

- **JSONDecodeError**: Occurs when `json.loads(response_text)` receives malformed or non-JSON text from the LLM provider.
- **Network timeouts**: Raised by the HTTP client when the LLM provider fails to respond, caught as `requests.exceptions.RequestException` or provider-specific errors.
- **Template rendering failures**: When `render_template` in [`prompts/template_manager.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompts/template_manager.py) returns `None` or malformed content, causing downstream `ValueError` exceptions.
- **Pydantic ValidationError**: Thrown during `EvaluationData(**evaluation_dict)` construction in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) when the parsed JSON lacks required fields or contains invalid data types.

In each case, the logger records the exact exception message, and the exception is re-thrown for the caller to manage.

## Supporting Files in the Error Chain

Several modules contribute to the error surface that [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) manages. Understanding these dependencies clarifies the full error propagation pathway:

- **models.py**: Defines the `EvaluationData` Pydantic model. Validation errors here bubble up to [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py)'s exception handler.
- **llm_utils.py**: Provides `extract_json_from_response()`. Failures in JSON extraction trigger the same error block in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py).
- **prompts/template_manager.py**: Renders prompt templates. File-not-found or template syntax errors propagate to the evaluator's try-except wrapper.

These files together form the error-handling pathway that safeguards [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) against any runtime failures that resemble traditional compilation errors.

## Summary

- **evaluator.py** handles compilation-like errors through a centralized try-except block spanning lines 89-91.
- **All exceptions** are logged with `logger.error()` before being re-raised to preserve the stack trace for upstream handlers.
- **Common triggers** include JSON decoding failures, network errors, template issues, and Pydantic validation errors.
- **Related files** such as [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py), [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py), and [`prompts/template_manager.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompts/template_manager.py) contribute to the error surface but rely on [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) for final error capture.

## Frequently Asked Questions

### Does evaluator.py compile code submitted by candidates?

No. Despite the name suggesting compilation error handling, [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) exclusively evaluates résumé text using LLM providers. It does not compile or execute candidate code. The "compilation error" handling actually refers to runtime exceptions during request processing and response parsing, such as JSON decoding failures or Pydantic validation errors.

### What happens when the LLM returns invalid JSON?

The `json.loads()` call raises a `JSONDecodeError`, which is caught by the exception handler in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py). The error is logged with `logger.error(f"Error evaluating resume: {str(e)}")` and then re-raised for the caller to handle, typically resulting in a 500-level error or retry attempt at the API layer.

### How does the error handling affect API responses?

The re-raise pattern ensures that [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) remains agnostic about HTTP responses. By propagating exceptions upward rather than catching and silencing them, the module allows the API layer to return appropriate error codes or trigger fallback logic while preserving the original error context in logs for debugging purposes.

### Where is the EvaluationData model defined?

The `EvaluationData` Pydantic model is defined in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py). When `evaluation_data = EvaluationData(**evaluation_dict)` executes in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py), any validation failures raise Pydantic `ValidationError` exceptions that are caught by the same try-except block handling other runtime errors.