Automated Tests and Validation Checks in Free-Programming-Books: A Complete Guide

The Free-Programming-Books repository runs six distinct GitHub Actions workflows that automatically lint Markdown formatting, validate external URLs, check bidirectional text markers, detect merge conflicts, and manage stale pull requests.

The EbookFoundation/free-programming-books project maintains quality through a comprehensive continuous integration pipeline. Every contribution triggers automated validation checks that enforce formatting standards, verify link integrity, and ensure proper handling of multilingual content. Understanding these automated tests helps contributors fix issues before submitting pull requests and maintainers review code efficiently.

Overview of the Continuous Integration Pipeline

The repository leverages GitHub Actions running on ubuntu-latest to execute validation checks in parallel. When a contributor opens or updates a pull request, the system triggers workflows defined in the .github/workflows/ directory. These automated tests examine Markdown syntax, URL accessibility, text directionality, and repository hygiene.

The pipeline produces GitHub annotations that appear directly on the changed files. Errors block merging, while warnings provide guidance without preventing approval. The system also posts automated comments to pull requests when validation fails, providing immediate feedback to contributors.

Markdown Linting and Format Validation

The free-programming-books-lint Tool

The primary formatting validation occurs in .github/workflows/fpb-lint.yml, which executes the npm package free-programming-books-lint. This tool enforces the canonical Markdown format required for book, course, and podcast lists.

The linter checks for:

• Correct bullet syntax and indentation
• Duplicate entries across lists
• Proper spacing after list markers
• Consistent formatting of titles and URLs

When the linter detects errors, it generates an error.log file and emits GitHub Actions annotations using the ::error file={name},line={line}::{message} format. The workflow exits with status 1 if any errors exist, preventing automatic merging.

Local Linting Setup

Contributors can run identical checks locally before submitting pull requests:

# Install the npm linter globally (same version used in CI)

npm install -g free-programming-books-lint

# Lint the entire repository or specific directories

fpb-lint books casts courses more

Local execution produces the same annotation-style output:

::error file=books/free-programming-books.md,line=42::Duplicate entry "Effective Python"
::warning file=courses/free-programming-courses.md,line=108::Missing trailing space after "- "

URL Validation and Link Checking

awesome_bot Configuration

The .github/workflows/check-urls.yml workflow validates hyperlink integrity using the Ruby gem awesome_bot. This check runs on both push and pull_request events, examining changed *.md and *.yml files.

The tool verifies:

• URL reachability (no 4xx or 5xx status codes)
• Valid SSL certificates
• Redirect handling according to repository policies
• Duplicate URL detection

Configuration flags include --allow-redirect and --allow-dupe to accommodate legitimate cases where redirects or duplicates are acceptable. Failures generate an artifact (ab-results-*.json) that subsequent steps process into GitHub annotations.

Running Link Checks Locally

Contributors can verify URLs before submission using the same tooling:

# Install the Ruby gem

gem install awesome_bot

# Validate specific files with the same flags used in CI

awesome_bot books/free-programming-books.md \
    --allow-redirect \
    --allow-dupe \
    --allow-ssl || true

Typical failure output indicates specific problematic URLs:

[ERROR] https://dead.link.example/ returned 404

Bidirectional Text Validation

RTL/LTR Linter Implementation

The repository maintains specific support for right-to-left (RTL) languages through .github/workflows/rtl-ltr-linter.yml. This workflow executes scripts/rtl_ltr_linter.py, a custom Python tool that validates bidirectional text handling in Markdown files.

The script checks for:

• Mixed RTL/LTR text without proper directionality markers
• Missing Unicode directionality markers (‏, ‎)
• Unclosed or mismatched <div dir=…> tags
• Keywords that require explicit directionality markers in RTL contexts

Configuration resides in scripts/rtl_ltr_linter_config.yml, defining keywords, symbols, and severity levels. The linter emits ::error and ::warning annotations compatible with GitHub Actions and writes detailed logs to rtl-linter-output.log.

Local RTL/LTR Testing

Contributors working with multilingual content can validate directionality locally:

# Clone the repository and install Python dependencies

python -m venv .venv && source .venv/bin/activate
pip install python-bidi PyYAML

# Execute the linter against specific files

python scripts/rtl_ltr_linter.py books/free-programming-books.md \
  --log-file rtl-linter-output.log

Sample output highlights specific directionality issues:

::warning file=books/free-programming-books.md,line=123::Keyword 'HTML' in meta 'HTML' may need trailing '‏' marker.
::error file=books/free-programming-books.md,line=210::Pure LTR text 'C++' in meta of RTL context may need trailing '‏' marker.

Pull Request Automation

Merge Conflict Detection

The .github/workflows/detect-conflicting-prs.yml workflow monitors PRs for merge conflicts using the third-party action eps1lon/actions-label-merge-conflict. Triggered on push and pull_request_target events, this automation labels PRs that cannot be merged automatically with a conflicts label.

When conflicts are detected, the workflow posts a friendly comment explaining how to resolve the issue by updating the branch with the latest changes from the main branch.

Automated PR Comments

The .github/workflows/comment-pr.yml workflow provides feedback by posting comments on pull requests when validation fails. This workflow triggers after the lint workflow completes, downloading the pr artifact containing error.log and posting its contents as a comment.

The workflow also manages the linter error label, adding it when errors exist and removing it once issues are resolved. This ensures contributors receive immediate notification of formatting issues without needing to navigate the Actions logs.

Stale Pull Request Management

Repository hygiene is maintained through .github/workflows/stale.yml, which runs daily via scheduled execution. This workflow marks PRs without recent activity as stale after 60 days and closes them after an additional 30 days unless they carry an exempt label.

The workflow adds the stale label and posts an explanatory message, helping maintainers focus on active contributions while automatically cleaning up abandoned requests. This check does not affect mergeability but helps manage the review queue.

Summary

• Six GitHub Actions workflows validate every contribution to the Free-Programming-Books repository, running automatically on pull requests and pushes.
• Markdown linting via free-programming-books-lint enforces canonical formatting, checking for duplicate entries, bullet syntax, and spacing issues.
• URL validation using awesome_bot verifies that all hyperlinks return valid status codes, have working SSL certificates, and follow redirect policies.
• RTL/LTR validation through a custom Python script ensures proper handling of bidirectional text, checking for missing directionality markers and unclosed HTML tags.
• PR automation includes conflict detection, automated commenting on validation failures, and stale PR management to maintain repository hygiene.
• Local execution is supported for all major checks, allowing contributors to validate changes before submitting pull requests.

Frequently Asked Questions

What happens if a URL check fails during automated testing?

When the URL validation workflow detects broken links, it uploads the results as a JSON artifact and generates GitHub annotations on the specific lines containing problematic URLs. The awesomebot-gh-summary-action processes these results to create a summary view. While URL failures are reported prominently, they typically appear as warnings rather than hard blockers unless configured otherwise, allowing maintainers to decide whether to merge despite transient network issues or genuinely dead resources.

Can I run the validation checks before submitting a pull request?

Yes, all major validation checks support local execution. Install the free-programming-books-lint npm package to check Markdown formatting, use the awesome_bot Ruby gem to validate URLs, and run the Python script scripts/rtl_ltr_linter.py with dependencies python-bidi and PyYAML to check bidirectional text. Running these tools locally produces identical output to the CI environment, allowing you to fix formatting errors, broken links, and directionality issues before opening your pull request.

How does the repository handle right-to-left languages?

The repository uses a custom Python linter located at scripts/rtl_ltr_linter.py to validate Markdown containing Arabic, Hebrew, or other RTL languages. This tool checks for proper Unicode directionality markers (‏ and ‎), ensures mixed RTL/LTR text displays correctly, and validates that HTML directionality tags like <div dir="rtl"> are properly closed. The workflow runs on every pull request, emitting GitHub annotations when directionality markers are missing or incorrectly applied, ensuring consistent rendering of multilingual content across the repository.

What triggers the stale pull request workflow?

The stale workflow runs on a daily schedule via GitHub Actions cron scheduling, and can also be triggered manually through workflow dispatch. It identifies pull requests that have been inactive for 60 days, adding the stale label and posting a comment to notify contributors. If no additional activity occurs within 30 days after being marked stale, the workflow automatically closes the pull request. Certain labels can exempt PRs from this process, ensuring that work-in-progress or high-priority contributions remain open despite temporary inactivity.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}