How Hiring Agent Selects Top GitHub Projects for Evaluation

The Hiring Agent selects top GitHub projects by combining hard-coded repository metrics with LLM-driven analysis, enforcing exactly 7 unique projects through a multi-layered fallback system.

The interviewstreet/hiring-agent repository implements an intelligent ranking algorithm that evaluates developer portfolios beyond simple star counts. When analyzing a candidate's GitHub profile, the system aggregates comprehensive repository metadata and employs a Large Language Model to interpret project quality, significance, and uniqueness.

Data Collection and Repository Classification

Before any selection occurs, the Agent gathers extensive metadata for each repository in github.py (lines 34-62). This collection phase captures:

• Repository identifiers: name, description, URL, live URL, and primary language (stored as technologies)
• Project classification: distinction between open-source (multiple contributors) and self-project (single contributor)
• Engagement metrics: star count, fork count, issue count, topics, repository size, and creation dates within the github_details block
• Contribution statistics: total contributors, the author's specific commit count, and overall commit history

This structured dataset provides the LLM with quantitative signals about project popularity and maintenance activity.

LLM-Driven Selection Process

Prompt Engineering with Jinja Templates

The collected repository data is rendered into a structured prompt using the github_project_selection.jinja template, managed by prompts/template_manager.py. In github.py (lines 78-85), this template is combined with an explicit system message that instructs the model:

"You are an expert technical recruiter analyzing GitHub repositories to identify the most impressive projects. CRITICAL: You must select exactly 7 UNIQUE projects – no duplicates allowed. Each project must be different from the others."

This hard constraint ensures the Agent returns a standardized portfolio size regardless of input volume.

JSON Parsing and Deduplication

After the LLM returns its selection, the Agent processes the response in github.py (lines 90-106):

1. Parses the JSON list of chosen projects
2. Removes any duplicate repository names
3. Validates that at least 7 distinct projects are present

If the LLM returns fewer than 7 unique entries, the system performs a star-based backfill, automatically supplementing the list with the highest-ranked repositories from the original dataset sorted by star count.

Fallback Mechanisms for Error Handling

The implementation includes robust error handling in github.py (lines 124-136). If the LLM response cannot be parsed due to JSON decode errors, or if the LLM call fails entirely, the Agent executes a safety fallback: it returns the first 7 repositories from the original list sorted by star count. This ensures the hiring workflow never breaks, even when AI services are unavailable or return malformed data.

Implementation Examples

To retrieve the top projects for a specific GitHub profile:

from github import fetch_and_display_github_info

result = fetch_and_display_github_info("https://github.com/your-username")
print("Top projects:")
for proj in result["projects"]:
    print(f"- {proj['name']} ({proj['github_url']})")

For testing or custom integrations, you can invoke the selection logic directly:

from github import fetch_all_github_repos, generate_projects_json

repos = fetch_all_github_repos("https://github.com/your-username")
top_projects = generate_projects_json(repos)  # Returns list of up to 7 projects

The generate_projects_json function encapsulates the entire pipeline—from data normalization through LLM selection to final deduplication—making it the core entry point for repository evaluation.

Summary

• The Hiring Agent aggregates comprehensive metadata including contributor statistics, project classification, and engagement metrics from github.py.
• Selection relies on an LLM interpreting a Jinja-rendered prompt with strict instructions to return exactly 7 unique projects.
• Duplicate removal and star-based backfill ensure portfolio diversity when the LLM returns fewer than 7 entries.
• A hard fallback to star-sorted repositories (lines 124-136) guarantees reliability if AI parsing fails.
• The system distinguishes between open-source and self-project classifications to provide context for the LLM's evaluation.

Frequently Asked Questions

How does the Hiring Agent handle cases where a user has fewer than 7 repositories?

If the candidate has fewer than 7 total repositories, the system returns all available unique projects. The LLM constraint for exactly 7 projects only applies when the input dataset contains 7 or more repositories; otherwise, the selection logic operates on the full available set without padding.

What distinguishes an "open-source" project from a "self-project" in the selection criteria?

The classification depends on contributor count. Open-source projects have multiple contributors, indicating community collaboration, while self-project entries have only a single contributor (the repository owner). This distinction is calculated in github.py and passed to the LLM as context for evaluating project scope and collaboration experience.

What happens if the LLM returns duplicate projects in its response?

The Agent implements explicit deduplication logic in generate_projects_json (lines 90-106). After parsing the LLM's JSON response, the system removes duplicate repository names and validates uniqueness. If duplicates are removed, causing the count to drop below 7, the system backfills with the highest-starred remaining repositories from the original dataset.

Which template engine does the Hiring Agent use to prepare repository data for the LLM?

The system uses Jinja2 templates managed through prompts/template_manager.py. Specifically, the github_project_selection.jinja template structures the repository metadata into a format optimized for the LLM's analysis, combining raw metrics with the system prompt that defines the selection criteria.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}