Checking Inconsistencies in Cloned Git Repositories with Hiring-Agent

The Hiring-Agent repository automates detection of repository inconsistencies by analyzing metadata, commit history, and file structure through an LLM-powered evaluation pipeline.

The interviewstreet/hiring-agent open-source project provides a modular AI-driven workflow specifically designed for checking inconsistencies in cloned Git repositories. This Python-based system extracts repository metadata from GitHub, validates commit patterns, and identifies structural contradictions that might indicate fabricated or misleading project history. By combining the github.py ingestion layer with the evaluator.py analysis engine, the system flags anomalies before they reach human interviewers.

How the System Validates Repository Integrity

The architecture separates data ingestion from AI analysis, allowing raw Git data to flow through a pipeline that produces structured consistency reports.

GitHub Metadata Extraction

In github.py, the GitHubClient class wraps the GitHub REST API to collect three primary consistency signals from public profiles. First, it gathers repository-level metadata including name, description, primary language, and star counts. Second, it retrieves commit activity data encompassing timestamps, frequency patterns, and author email validation. Third, it performs file-structure checks to verify the presence of typical project artifacts like README.md, setup.py, or requirements.txt.

For locally cloned repositories, the LocalRepoAnalyzer class performs analogous introspection on filesystem paths, enabling offline analysis without API rate limits.

LLM-Powered Consistency Analysis

The evaluator.py module orchestrates the project_evaluation routine, which constructs prompts using the projects.jinja template from the prompts/templates/ directory. This template embeds the collected metadata and instructs the underlying model—accessed via llm_utils.py—to detect logical contradictions. The system specifically flags repositories where the claimed primary language conflicts with actual file distributions, or where documentation claims contradict dependency manifests.

Structured Validation Results

Validation outputs conform to the ProjectConsistency Pydantic model defined in models.py, providing typed fields for boolean flags such as inconsistent_language, missing_readme, or fabricated commit history indicators. The score.py module then combines these consistency flags with other resume dimensions to produce a final numeric rating.

Detecting Specific Repository Anomalies

According to the InterviewStreet source code, the system identifies three critical inconsistency categories that suggest manipulated or low-quality repositories:

• Language Mismatches: Repositories claiming "Python" as the primary language while containing predominantly JavaScript files, or vice versa.
• Dependency Contradictions: README files stating "no external dependencies" while a requirements.txt or package.json lists several packages.
• Fabricated History: Commit timestamps that leap forward by months without intermediate activity, suggesting artificial commit backdating or history rewriting.

Implementation Examples

The following snippets demonstrate common entry points for evaluating repository consistency.

Evaluate a single GitHub username using the API-driven workflow:

from hiring_agent.github import GitHubClient
from hiring_agent.evaluator import Evaluator
from hiring_agent.config import Config

cfg = Config.load()                      # reads .env variables

gh = GitHubClient(cfg.github_token)      # authorised GitHub client

evaluator = Evaluator(cfg)

# Pull repository metadata for a user

repos = gh.list_user_repos("alice-dev")
result = evaluator.evaluate_projects(repos)

print(result.summary())

# Shows consistency warnings and an overall score

Parse a PDF résumé and run the full evaluation pipeline:

from hiring_agent.pdf import PDFParser
from hiring_agent.evaluator import Evaluator
from hiring_agent.config import Config

cfg = Config.load()
parser = PDFParser()
evaluator = Evaluator(cfg)

resume_text = parser.extract_text("alice_resume.pdf")
full_report = evaluator.evaluate_resume(resume_text)

print(full_report.to_markdown())

# Human-readable interview brief with consistency checks baked in

Perform a manual consistency check on a locally cloned repository:

import pathlib
from hiring_agent.github import LocalRepoAnalyzer
from hiring_agent.evaluator import Evaluator
from hiring_agent.config import Config

cfg = Config.load()
local_analyzer = LocalRepoAnalyzer(pathlib.Path("/tmp/alice-dev/project"))
evaluator = Evaluator(cfg)

metadata = local_analyzer.collect_metadata()
consistency = evaluator.check_project_consistency(metadata)

print(consistency.json())

# JSON payload with inconsistent_language, missing_readme, etc.

Summary

• The github.py module extracts repository metadata, commit history, and file structure via GitHubClient and LocalRepoAnalyzer.
• The evaluator.py pipeline uses Jinja2 templates from prompts/templates/ to construct LLM prompts that detect logical inconsistencies.
• Results return as structured ProjectConsistency objects defined in models.py, enabling automated scoring of repository authenticity.
• Configuration management through config.py centralizes API keys and model selection via environment variables.

Frequently Asked Questions

How does Hiring-Agent detect language inconsistencies in cloned repositories?

The system compares the primary language field returned by the GitHub API against the actual file distribution within the repository. When analyzing cloned directories through LocalRepoAnalyzer, it calculates file type frequencies and flags cases where the dominant language differs from the repository's declared primary language, storing the result in the inconsistent_language field of the ProjectConsistency model.

What file structure checks does the system perform?

The implementation in github.py verifies the presence of standard open-source artifacts including README.md, setup.py, requirements.txt, package.json, and .gitignore files. Missing expected files trigger specific flags in the evaluation output, helping identify skeleton repositories or projects that lack proper documentation despite showing recent commit activity.

Can Hiring-Agent analyze local Git repositories without GitHub API access?

Yes. The LocalRepoAnalyzer class accepts a pathlib.Path object pointing to any local Git repository and performs offline analysis of commit history, file structure, and language statistics. This approach bypasses API rate limits and enables analysis of private repositories or air-gapped environments, though it requires the repository to be already cloned to the local filesystem.

How are inconsistency results structured in the output?

The models.py file defines a ProjectConsistency Pydantic model that structures the LLM's analysis into typed fields including boolean flags for specific issues (e.g., missing_readme, inconsistent_language) and descriptive text explaining detected contradictions. This structured output integrates with the score.py ranking utilities to produce a composite authenticity rating that appears in the final evaluation report.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}