How the github.py Module Selects the Top 7 Projects from Candidate Repositories

The github.py module employs a hybrid LLM-driven pipeline that first sorts repositories by star count, prompts a language model to evaluate and choose the seven most impressive unique projects, and enforces the quota by backfilling from the highest-starred repos if the LLM returns fewer than seven entries.

The github.py module in the interviewstreet/hiring-agent repository orchestrates a sophisticated ranking system to surface the most relevant work from a candidate's GitHub profile. Understanding how this module selects the top 7 projects reveals a carefully balanced approach between algorithmic sorting and AI-powered evaluation, ensuring recruiters see the candidate's best contributions even when the LLM response is incomplete or malformed.

The Multi-Step Selection Pipeline

The selection process follows a deterministic sequence designed to guarantee exactly seven unique projects while maximizing quality through intelligent filtering.

Fetching and Sorting Repositories by Popularity

The pipeline begins by retrieving comprehensive repository data. The fetch_all_github_repos() function pulls the candidate's repositories, gathers contribution statistics, and constructs a rich dictionary for each repo containing metadata like stars, forks, and primary languages.

Immediately after enrichment, the module applies an initial ranking based on community validation. According to the source code in main/github.py (lines 80-82), the list is sorted by star count using:

projects.sort(key=lambda x: x["github_details"]["stars"], reverse=True)

This ensures that the most popular repositories appear first, creating a deterministic baseline that serves both as a tiebreaker and a fallback mechanism.

Constructing the LLM Prompt

Once sorted, the module prepares the data for AI evaluation. The generate_projects_json() function converts the enriched list into a JSON string and injects it into the github_project_selection prompt template (lines 38-44). This template instructs the model to analyze repository metadata—including descriptions, technologies, and complexity indicators—rather than relying solely on popularity metrics.

LLM Ranking and Deduplication

The core selection logic delegates the final curation to a Large Language Model. The system message explicitly directs the model to select exactly 7 unique projects (lines 78-84). The module invokes the model via provider.chat(**chat_params), passing the JSON-encoded repository list along with recruiter-specific evaluation criteria.

After receiving the response, the module parses the JSON output and enforces uniqueness through a deduplication tracker. As implemented in lines 100-108, the code maintains a seen set to track project names, keeping only the first occurrence of each repository and discarding any duplicates the LLM might have included.

Backfilling and Error Handling

The module includes robust safeguards to guarantee a complete set of seven projects. If the LLM returns fewer than seven unique entries, the code iterates over the original projects_data list—which remains sorted by stars—and appends the next-most-starred projects that have not yet been selected until the quota is reached (lines 110-122).

Additionally, any JSON-parsing failure or LLM error triggers an immediate fallback. The exception handler in lines 132-138 simply returns the first seven entries from the pre-sorted list (projects_data[:7]), ensuring the system never fails silently or returns an empty result set.

Key Implementation Details in github.py

The selection logic is encapsulated in several tightly coupled functions within main/github.py. Here is the critical path for repository ranking:

# From main/github.py - Initial sorting by social proof

projects_data = fetch_all_github_repos(github_url)
projects_data.sort(key=lambda x: x["github_details"]["stars"], reverse=True)

# Prepare the LLM evaluation context

projects_json = json.dumps(projects_data, indent=2)
prompt = template_manager.render_template(
    "github_project_selection", 
    projects_data=projects_json
)

The LLM invocation includes strict constraints on output format and quantity:

# System instruction enforces the 7-project limit

chat_params = {
    "model": DEFAULT_MODEL,
    "messages": [
        {
            "role": "system",
            "content": "You are an expert technical recruiter. Select exactly 7 UNIQUE projects..."
        },
        {"role": "user", "content": prompt},
    ],
    "options": model_params,
}
response = provider.chat(**chat_params)

Fallback Mechanisms for Robustness

The module implements a two-tiered fallback strategy to handle edge cases. First, the deduplication logic ensures that even if the LLM hallucinates duplicate entries, the final list contains only unique repositories. Second, the backfill mechanism guarantees that candidates with fewer than seven repos—or cases where the LLM is overly restrictive—still return a complete portfolio by supplementing with the highest-starred remaining projects.

In the event of complete LLM failure—whether from timeout, malformed JSON, or API errors—the module degrades gracefully to pure algorithmic selection, returning the top seven most-starred repositories without AI intervention.

Summary

• Primary sorting: Repositories are initially ranked by star count in descending order to establish a popularity baseline.
• AI curation: An LLM evaluates the enriched metadata to select exactly seven unique projects based on technical merit and relevance.
• Deduplication: The module enforces uniqueness by tracking seen project names and discarding duplicates from the LLM output.
• Quota enforcement: If the LLM returns fewer than seven projects, the system backfills from the remaining highest-starred repositories.
• Graceful degradation: JSON parsing errors trigger an immediate fallback to the first seven entries from the star-sorted list.

Frequently Asked Questions

How does github.py handle candidates with fewer than 7 repositories?

If a candidate has fewer than seven public repositories, the module returns all available unique projects. The backfill logic only activates when the LLM selects fewer than seven from a larger pool, iterating through the star-sorted list to reach the quota if possible, or returning the total available count if the candidate has fewer than seven repos total.

What criteria does the LLM use to rank projects instead of just using star counts?

According to the github_project_selection prompt template, the LLM evaluates repository descriptions, primary technologies, complexity indicators, and architectural patterns. The system message frames the model as an "expert technical recruiter" instructed to identify impressive, technically sophisticated work rather than merely popular repositories.

Why does the module sort by stars before sending data to the LLM?

The pre-sorting serves two purposes: it provides a deterministic ordering that ensures consistent fallback behavior if the LLM fails, and it establishes a logical default sequence for backfilling. When the LLM returns incomplete results, the system can efficiently append the next most-starred projects without re-ranking the entire dataset.

What happens if the LLM returns duplicate project names?

The parsing logic in main/github.py (lines 100-108) maintains a seen set to track project names during iteration. When encountering a duplicate name, the module skips that entry and continues processing, ensuring the final output contains only unique repositories even if the LLM includes redundant selections.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}