# Can evaluator.py Detect Plagiarism or Code Similarity?

> Discover if evaluator.py detects plagiarism. This module from interviewstreet/hiring-agent analyzes résumés with LLMs, but does not check for code similarity.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: deep-dive
- Published: 2026-07-15

---

**No, the [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) module in the interviewstreet/hiring-agent repository cannot detect plagiarism or code similarity; it exclusively evaluates candidate résumés using large language model (LLM) analysis.**

The [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) module serves as the core résumé screening engine for the hiring-agent project. Despite its general naming, this file reveals a narrow scope focused on parsing résumé text through LLM prompts to generate structured feedback, with no capabilities for analyzing source code or computing similarity metrics.

## What evaluator.py Actually Does

Inside [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py), the **`ResumeEvaluator`** class orchestrates a specific workflow designed for recruitment automation. The class builds evaluation prompts from résumé text, communicates with an LLM provider, and structures the response into a typed Python object.

The evaluation flow follows this sequence:

1. **Prompt Construction**: The class uses a **template manager** (located in [`prompts/template_manager.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompts/template_manager.py)) to render evaluation criteria from `templates/resume_evaluation_criteria`.
2. **LLM Invocation**: It calls `self.provider.chat` to send the constructed prompt to the configured language model.
3. **Response Parsing**: The raw LLM output passes through **`extract_json_from_response`** (defined in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py)) to extract structured data.
4. **Data Validation**: The extracted data populates an **`EvaluationData`** Pydantic model (defined in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)), which includes fields for scores and qualitative comments.

## Architecture of the Resume Evaluation Pipeline

Understanding why plagiarism detection is absent requires examining the component architecture.

### ResumeEvaluator Class

The `ResumeEvaluator` class acts as the primary interface. When instantiated, it initializes an LLM provider through `llm_utils.initialize_llm_provider`. The class exposes an `evaluate_resume` method that accepts résumé text as a string and returns an `EvaluationData` instance. At no point does this class accept source code files or implement comparison algorithms.

### LLM Provider Abstraction

The provider system (accessed via [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py)) abstracts underlying APIs such as Gemini or OpenAI. The provider's `.chat` method processes natural language prompts only. It performs **no intrinsic code analysis**, tokenization for similarity, or fingerprinting operations.

### Template Management System

Prompt templates stored in [`prompts/template_manager.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompts/template_manager.py) and configuration in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) define static evaluation criteria. These criteria assess résumé quality factors like formatting, achievements, and role relevance—not code originality or syntactic similarity between programming solutions.

### EvaluationData Schema

The `EvaluationData` model in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) strictly defines expected output fields such as `score` and `comments`. The schema contains no plagiarism metrics, similarity scores, or reference corpus identifiers.

## Why Plagiarism Detection Is Not Implemented

Detecting code plagiarism requires specific technical components that are entirely absent from the current codebase:

- **Reference Corpus**: A database of existing solutions to compare against.
- **Similarity Algorithms**: Implementation of metrics such as Levenshtein distance, token overlap, or abstract syntax tree (AST) comparison.
- **Source Code Parsing**: Logic to extract and normalize code from submission files.

The `interviewstreet/hiring-agent` repository contains none of these elements. The module never receives file paths to code submissions, nor does it compute distance metrics between text samples.

## How to Use ResumeEvaluator

The following example demonstrates the intended usage for résumé evaluation:

```python
from evaluator import ResumeEvaluator

# Initialize the evaluator (defaults to the configured model)

evaluator = ResumeEvaluator()

# Sample résumé text

resume_text = """
John Doe
Software Engineer
Experience: 5 years at XYZ Corp...
"""

# Run the evaluation – returns a typed EvaluationData object

evaluation = evaluator.evaluate_resume(resume_text)

print("Score:", evaluation.score)
print("Comments:", evaluation.comments)

```

This snippet illustrates the module's exclusive focus on text-based résumé analysis. Attempting to pass source code through this pipeline would simply result in the LLM evaluating the code as if it were biographical text, without any similarity comparison to existing solutions.

## Summary

- **[`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py)** processes only résumé text, not source code.
- The **`ResumeEvaluator`** class relies on LLM prompts for qualitative assessment, not algorithmic similarity detection.
- No plagiarism detection infrastructure exists in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py), [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py), or template files.
- Adding plagiarism detection would require a separate pipeline with corpus management and algorithmic comparison tools.

## Frequently Asked Questions

### Does evaluator.py support code similarity checks?

No, [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) does not support code similarity checks. The module is hardcoded to evaluate résumé content through LLM analysis. It lacks the algorithms, reference databases, and file parsing logic required for plagiarism detection.

### What file types does evaluator.py process?

The module processes plain text strings representing résumé content. According to the source code in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py), the `evaluate_resume` method accepts text input directly and passes it to the LLM provider without file extension checks or syntax parsing.

### How does evaluator.py validate résumé content?

Validation occurs through the **`EvaluationData`** Pydantic model in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py). The `extract_json_from_response` function in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) parses the LLM's JSON output, and the model validates field types and constraints. This ensures structured scoring data, not code originality verification.

### Can I modify evaluator.py to detect plagiarism?

While technically possible, modifying [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) for plagiarism detection would require significant architectural changes. You would need to integrate a code corpus database, implement similarity algorithms (such as AST matching or token-based fingerprinting), and modify the input pipeline to accept source code rather than résumé text. This would effectively create a separate module beyond the current résumé-focused scope.