How to Manage GitHub API Rate Limiting Issues Using Personal Access Tokens

Set the GITHUB_TOKEN environment variable with a valid GitHub personal access token to increase the REST API rate limit from 60 to 5,000 requests per hour and eliminate authentication errors during candidate profile retrieval.

The interviewstreet/hiring-agent repository automates technical resume screening by querying the GitHub REST API for public candidate repositories and metadata, but unauthenticated requests hit a restrictive 60-request hourly cap. By implementing proper token-based authentication, you can effectively manage GitHub API rate limiting issues using personal access tokens without modifying core application logic, as the helper logic in github.py automatically detects and applies credentials at runtime.

Authentication Flow in github.py

The core authentication logic resides in github.py (lines 31-88), where the application detects credentials and constructs authorized request headers dynamically.

Environment Variable Detection

The helper first inspects the process environment for a token using standard library calls:

github_token = os.environ.get("GITHUB_TOKEN")

When this variable is absent, the application prints a diagnostic tip around line 88: "💡 Tip: Set GITHUB_TOKEN environment variable to increase rate limits (60/hour → 5000/hour)". This warning appears in the console output to alert operators that the process is running in unauthenticated mode.

Authorization Header Injection

If GITHUB_TOKEN exists, the code injects the Authorization: token <PAT> header into every outgoing HTTP request. This single configuration change upgrades the GitHub API quota from 60 requests per hour to 5,000 requests per hour, providing sufficient headroom for the scoring pipeline to batch-process multiple candidate profiles via score.py and its fetch_and_display_github_info orchestration method.

Configuring Your Personal Access Token

Token Generation with Minimal Scopes

Navigate to GitHub → Settings → Developer settings → Personal access tokens → Generate new token. Select the read:user and repo scopes; read-only access suffices for fetching public profile data and repository statistics required by the hiring-agent analysis.

Environment Configuration Methods

The repository includes an .env.example file demonstrating the expected variable name and format. You can supply the token via any of the following methods:

1. Shell export:

   export GITHUB_TOKEN=ghp_XXXXXXXXXXXXXXXXXXXXXXXXXXXX
   python -m main.score ./resume.pdf

2. Dotenv file: Create a .env file in the project root:

   GITHUB_TOKEN=ghp_XXXXXXXXXXXXXXXXXXXXXXXXXXXX

   The application will load this automatically if your execution environment supports dotenv files.

3. Docker Compose:

   services:
     hiring-agent:
       image: interviewstreet/hiring-agent:latest
       env_file: .env
       volumes:
         - ./cache:/app/cache

Advanced Rate Limit Management Strategies

Even with authentication, aggressive batch processing can exhaust the 5,000-request quota. The hiring-agent codebase supports additional safeguards to handle extreme throughput scenarios.

Persistent Response Caching

Fetched data is serialized to the cache/gh_githubcache_... directory structure. Persist this folder between runs to eliminate redundant API calls for previously processed candidates. In containerized deployments, mount a host volume to cache/ to maintain state across container restarts.

Handling 403 Rate Limit Exceeded Errors

If the API returns a 403 response with X-RateLimit-Remaining: 0, extend the request logic in github.py to read the X-RateLimit-Reset timestamp header. Calculate the delta between the current Unix time and the reset time, then invoke time.sleep() to pause execution until the quota refreshes.

Token Rotation for Parallel Processing

For high-throughput scenarios that exceed 5,000 requests per hour, distribute workload across multiple personal access tokens. Assign a unique GITHUB_TOKEN value to each worker process, or implement a round-robin selector in the HTTP helper to cycle through a pool of tokens and balance the request load.

Verifying Your Configuration

After exporting the variable, validate the setup by running the scoring pipeline:

export GITHUB_TOKEN=ghp_XXXXXXXXXXXXXXXXXXXXXXXXXXXX
python -m main.score ./resume.pdf

Successful execution without the console tip regarding rate limits confirms that github.py is correctly injecting the authorization header and that the orchestration layer in score.py can invoke fetch_and_display_github_info without hitting the 60-request unauthenticated cap.

Summary

• Export GITHUB_TOKEN to upgrade the GitHub API quota from 60 to 5,000 requests per hour
• The authentication logic in github.py automatically detects the variable and injects the Authorization: token <PAT> header
• Store sensitive tokens in .env files following the template provided in .env.example
• Cache responses in cache/gh_githubcache_... to minimize redundant network calls
• For extreme throughput, implement token rotation across multiple parallel worker processes

Frequently Asked Questions

What scopes does the personal access token need for hiring-agent?

The token requires only the read:user and repo scopes. These permissions provide read-only access to public profile information and repository metadata, which is sufficient for the automated resume scoring performed by score.py without granting unnecessary write access.

Where does hiring-agent store cached GitHub API responses?

The application writes serialized response data to files under the cache/gh_githubcache_... directory. You should persist this directory between runs using Docker volumes or persistent host storage to avoid redundant API requests against your 5,000-request quota.

Can I use multiple tokens to bypass the 5,000 request limit?

Yes. You can distribute workload across multiple personal access tokens by assigning different GITHUB_TOKEN values to separate worker processes, or by modifying the request builder in github.py to rotate through a pool of tokens automatically when handling large candidate batches.

Why do I still see rate limit warnings after setting GITHUB_TOKEN?

Ensure the environment variable is exported in the same shell session or loaded from the .env file before the Python process initializes. Verify that os.environ.get("GITHUB_TOKEN") in github.py returns a non-None value, and confirm that your specific token has not exceeded its own 5,000-request hourly quota or been revoked.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}