How the GitHub Enrichment Module Fetches and Classifies Repositories in interviewstreet/hiring-agent

The GitHub enrichment module extracts a candidate's username from their profile URL, paginates through the GitHub REST API to retrieve all public repositories, classifies each project as either open-source or self-project based on license metadata, and uses an LLM to select the seven most impressive projects for evaluation.

The interviewstreet/hiring-agent repository automates technical candidate screening by enriching resume data with public GitHub activity. At the heart of this pipeline lies the GitHub enrichment module, which transforms raw profile URLs into structured, classified project data that feeds directly into the scoring algorithm.

Extracting and Sanitizing the GitHub Username

The enrichment process begins in github.py with the extract_github_username function (lines 116-131). This helper sanitizes the input URL by removing whitespace and applies regular expression patterns to extract the username from various GitHub URL formats.

def extract_github_username(github_url: str) -> Optional[str]:
    if not github_url:
        return None
    github_url = github_url.replace(" ", "").strip()
    patterns = [r"https?://github\.com/([^/]+)", r"github\.com/([^/]+)"]
    for pat in patterns:
        m = re.search(pat, github_url)
        if m:
            return m.group(1)
    return None

The function returns None if no valid username is found, effectively halting the enrichment process for malformed URLs.

Fetching Profile and Repository Data via the GitHub REST API

Once the username is isolated, the module implements a two-phase fetching strategy to minimize API calls and handle large portfolios. The _fetch_github_api helper (lines 29-33) executes the actual HTTP requests, while a disk-based caching layer (lines 19-25) stores responses in cache/gh_githubcache_... files to avoid redundant network traffic.

To retrieve the complete repository list, fetch_all_github_repos (lines 232-280) iterates over the paginated /users/<username>/repos endpoint, requesting 100 items per page until all public repositories are exhausted.

def fetch_all_github_repos(github_url: str) -> List[Dict]:
    username = extract_github_username(github_url)
    if not username:
        return []
    repos = []
    page = 1
    while True:
        api = f"https://api.github.com/users/{username}/repos?per_page=100&page={page}"
        data = _fetch_github_api(api)
        if not data:
            break
        repos.extend(data)
        if len(data) < 100:
            break
        page += 1
    return repos

Normalizing Metadata and Classifying Project Types

After retrieval, each raw repository object is normalized into a structured dictionary containing name, description, html_url, stargazers_count, forks_count, and language. The classification logic (lines 278-283) inspects the license field to determine the project_type: repositories with a non-empty license are flagged as open_source, while those without are categorized as self_project.

def normalise_repo(raw: Dict) -> Dict:
    project_type = "open_source" if raw.get("license") else "self_project"
    return {
        "name": raw.get("name"),
        "description": raw.get("description"),
        "github_url": raw.get("html_url"),
        "github_details": {
            "stars": raw.get("stargazers_count", 0),
            "forks": raw.get("forks_count", 0),
            "language": raw.get("language"),
        },
        "project_type": project_type,
    }

The normalized list is then sorted by star count in descending order to surface the most popular work first. The module logs classification counts (lines 289-293) with status messages like "📊 Project classification: Y open source, Z self projects" to facilitate debugging.

LLM-Driven Selection of Top Projects

Rather than passing all repositories to the scoring engine, the module uses generate_projects_json (lines 334-360) to transform the list into a compact JSON structure. This payload feeds into the github_project_selection.jinja template, which prompts an LLM to identify the seven most impressive projects based on popularity, complexity, and relevance.

The selection logic (lines 383-435) handles LLM response parsing, deduplicates any repeated selections, and implements a fallback mechanism that defaults to the first seven starred repositories if the LLM fails to return valid output.

Integration with the Resume Scoring Pipeline

The fetch_and_display_github_info function (line 459) bundles the candidate's profile metadata, the full classified project list, and the LLM-selected subset into a single dictionary. This enriched payload is then consumed by score.py (lines 312-320), where it merges with traditional resume data before the final evaluation runs.

Summary

• Username extraction uses regex patterns in extract_github_username (lines 116-131) to parse GitHub URLs and isolate the candidate handle.
• API efficiency is achieved through disk caching (cache/gh_githubcache_...) and paginated fetching via fetch_all_github_repos (lines 232-280).
• Project classification depends on the presence of a license field, separating repositories into open_source or self_project categories (lines 278-283).
• Popularity ranking sorts repositories by stargazers_count to prioritize the candidate's most visible work.
• LLM curation uses the github_project_selection.jinja template to select the top seven projects for evaluation (lines 383-435).
• Pipeline integration occurs in score.py (lines 312-320), where GitHub enrichment data merges with resume content for final scoring.

Frequently Asked Questions

How does the module distinguish between open-source and self-project repositories?

The classification logic in normalise_repo (lines 278-283) checks the license field of each repository. If a license exists, the project is labeled open_source; otherwise, it is categorized as self_project. This binary classification allows the scoring pipeline to weight collaborative contributions differently from personal experiments.

What caching mechanism prevents repeated GitHub API calls?

The module implements disk-based caching in _fetch_github_api (lines 19-33), storing API responses in files prefixed with cache/gh_githubcache_. This persistence layer ensures that subsequent runs for the same candidate retrieve data from local storage rather than hitting the GitHub API again, reducing latency and avoiding rate limits.

How does the LLM select which projects to include in the final evaluation?

The generate_projects_json function (lines 334-360) prepares a structured payload of all normalized repositories. This data is passed to the github_project_selection.jinja template, which prompts the LLM to evaluate technical sophistication, community impact, and relevance. The selection logic (lines 383-435) then parses the LLM response, removes duplicates, and falls back to the top seven starred repositories if the LLM returns invalid or empty results.

Where does the enriched GitHub data integrate into the hiring workflow?

After enrichment, the fetch_and_display_github_info function (line 459) compiles the profile and project data into a dictionary. This structure is imported into score.py (lines 312-320), where it merges with the candidate's resume information to inform the final technical assessment and scoring output.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}