How to Add a Custom Job Portal Skill to the AI Job Search Framework

Developers can extend the AI Job Search framework by creating a new directory under .agents/skills/, adding a SKILL.md specification file, and implementing a Python scraper module that exposes standard search() and apply() functions.

The MadsLorentzen/ai-job-search repository uses a thin-pointer design that keeps every job-portal implementation in a single source-of-truth location. Adding a custom job portal skill requires following a strict convention within the hidden .agents/skills/ hierarchy, allowing the core framework to discover and load new integrations dynamically via importlib.

Understanding the Skill Architecture

The framework treats each job portal as a skill—a self-contained package of metadata and executable code. This modular approach ensures that the generic commands (scrape, rank, apply) can interact with any portal without hard-coded dependencies.

The .agents/skills Directory Structure

All portal-specific code lives hidden within the repository root:

.agents/skills/
├── <portal_name>/
│   ├── SKILL.md
│   └── scrape_<portal>.py

The folder name becomes the portal identifier used throughout the framework. According to the design documented in AGENTS.md, this structure ensures that the skill loader can enumerate available portals by scanning subdirectories.

Skill Interface Requirements

Every scraper module must conform to the skill-interface used by core commands. The module must expose:

• search(query: str) -> List[Dict] – Returns a list of job dictionaries containing fields like title, company, location, and url.
• apply(job_url: str) -> bool (Optional) – Handles automated or assisted application submission.

The framework imports these functions dynamically, so exact naming and signatures are critical for compatibility.

Step-by-Step Implementation Guide

Step 1: Create the Skill Directory

First, create a new directory under .agents/skills/ using the portal's lowercase identifier:

mkdir -p .agents/skills/awesomejobs

This directory will house the specification and implementation for the "AwesomeJobs" portal. The framework uses this folder name to reference the skill in CLI commands.

Step 2: Define the Skill Specification (SKILL.md)

Create a SKILL.md file that describes the portal's capabilities, endpoints, and dependencies. The framework's validation tool (tools/lint_skills.py) parses this file to ensure completeness.

# .agents/skills/awesomejobs/SKILL.md

# AwesomeJobs Skill

- **Portal name**: AwesomeJobs
- **Base URL**: https://www.awesomejobs.com
- **Search endpoint**: /search?q={query}
- **Authentication**: None (or API key placeholder `{{AWESOME_API_KEY}}`)
- **Supported actions**: `search`, `apply`
- **Dependencies**: `requests`, `beautifulsoup4`

Required fields include the portal name, base URL, search endpoint pattern, authentication method, supported actions list, and Python dependencies. Placeholders like {{API_KEY}} indicate where the framework should inject secrets during runtime.

Step 3: Implement the Scraper Module

Create a Python module that implements the search logic. Save this as scrape_<portal>.py within the skill directory:

# .agents/skills/awesomejobs/scrape_awesomejobs.py

import requests
from bs4 import BeautifulSoup
from typing import List, Dict

BASE_URL = "https://www.awesomejobs.com"

def search(query: str) -> List[Dict]:
    """Return a list of job dicts for the given query."""
    resp = requests.get(f"{BASE_URL}/search", params={"q": query})
    resp.raise_for_status()
    soup = BeautifulSoup(resp.text, "html.parser")
    results = []
    for card in soup.select(".job-card"):
        results.append({
            "title": card.select_one(".title").get_text(strip=True),
            "company": card.select_one(".company").get_text(strip=True),
            "location": card.select_one(".location").get_text(strip=True),
            "url": BASE_URL + card.select_one("a")["href"],
        })
    return results

def apply(job_url: str) -> bool:
    """Placeholder – most portals require manual application."""
    return True

The module must use the exact function signatures shown above. The search function handles HTTP requests and HTML parsing, while apply serves as a hook for future automation. Use helper utilities like tools/robots_check.py to respect robots.txt constraints when implementing fetch logic.

Step 4: Validate and Register the Skill

Run the lint command to validate your SKILL.md and ensure the new skill meets framework standards:

python -m tools.lint_skills

If validation passes, the skill is automatically discovered the next time you execute core commands. No additional registration steps are required because the framework scans .agents/skills/ at runtime.

Test the integration using the generic scrape command:

ai-job-search scrape --portal awesomejobs "data scientist"

Testing Your Custom Skill

Create unit tests in the tests/ directory following the pattern established in tests/test_upskill_skill.py:

# tests/test_awesomejobs_skill.py

from .agents.skills.awesomejobs.scrape_awesomejobs import search

def test_search_returns_jobs():
    results = search("software engineer")
    assert isinstance(results, list)
    assert "title" in results[0]
    assert "url" in results[0]

Run the test suite with pytest to verify that your scraper returns properly structured dictionaries before committing the new skill.

Summary

• Create a new directory under .agents/skills/<portal_name>/ to house your integration.
• Document the portal specifications in SKILL.md, including endpoints, authentication, and dependencies.
• Implement a Python module exposing search(query) and optional apply(job_url) functions with exact signatures.
• Validate your skill using python -m tools.lint_skills to ensure automatic discovery by the framework core.
• Test thoroughly using the existing test patterns in tests/ to verify field consistency and error handling.

Frequently Asked Questions

What file naming convention should I use for the scraper module?

Name the file scrape_<portal_name>.py where <portal_name> matches the directory name exactly. For example, .agents/skills/awesomejobs/scrape_awesomejobs.py. The framework uses this naming pattern when dynamically importing the module via importlib.

Does the framework support authentication tokens in SKILL.md?

Yes. Use placeholder syntax like {{API_KEY_NAME}} in the Authentication field of SKILL.md. The framework parses these placeholders and attempts to resolve them from environment variables or secure storage during runtime, though the actual secret management implementation depends on your deployment configuration.

How does the framework load custom skills dynamically?

The framework scans the .agents/skills/ directory at startup, validates each subdirectory against the SKILL.md schema using tools/lint_skills.py, and then imports the scrape_*.py module using importlib. This thin-pointer design allows the core commands to remain generic while supporting arbitrary portal implementations.

What validation checks does lint_skills.py perform?

The linter verifies that each skill directory contains a readable SKILL.md with required fields (portal name, base URL, search endpoint, supported actions, and dependencies), checks for the presence of the corresponding scrape_*.py file, and validates that the Python module syntax is correct. It also ensures that supported actions listed in the markdown match implemented functions in the scraper module.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}