# Automated Tests and Validation Checks in Free-Programming-Books: A Complete Guide > Discover the automated tests and validation checks used in the free-programming-books repo. Learn how GitHub Actions ensure Markdown linting URL validation and more. - Repository: [Free Ebook Foundation/free-programming-books](https://github.com/EbookFoundation/free-programming-books) - Tags: how-to-guide - Published: 2026-02-23 --- **The Free-Programming-Books repository runs six distinct GitHub Actions workflows that automatically lint Markdown formatting, validate external URLs, check bidirectional text markers, detect merge conflicts, and manage stale pull requests.** The EbookFoundation/free-programming-books project maintains quality through a comprehensive continuous integration pipeline. Every contribution triggers automated validation checks that enforce formatting standards, verify link integrity, and ensure proper handling of multilingual content. Understanding these automated tests helps contributors fix issues before submitting pull requests and maintainers review code efficiently. ## Overview of the Continuous Integration Pipeline The repository leverages GitHub Actions running on `ubuntu-latest` to execute validation checks in parallel. When a contributor opens or updates a pull request, the system triggers workflows defined in the `.github/workflows/` directory. These automated tests examine Markdown syntax, URL accessibility, text directionality, and repository hygiene. The pipeline produces GitHub annotations that appear directly on the changed files. Errors block merging, while warnings provide guidance without preventing approval. The system also posts automated comments to pull requests when validation fails, providing immediate feedback to contributors. ## Markdown Linting and Format Validation ### The free-programming-books-lint Tool The primary formatting validation occurs in [`.github/workflows/fpb-lint.yml`](https://github.com/EbookFoundation/free-programming-books/blob/main/.github/workflows/fpb-lint.yml), which executes the npm package `free-programming-books-lint`. This tool enforces the canonical Markdown format required for book, course, and podcast lists. The linter checks for: - Correct bullet syntax and indentation - Duplicate entries across lists - Proper spacing after list markers - Consistent formatting of titles and URLs When the linter detects errors, it generates an `error.log` file and emits GitHub Actions annotations using the `::error file={name},line={line}::{message}` format. The workflow exits with status 1 if any errors exist, preventing automatic merging. ### Local Linting Setup Contributors can run identical checks locally before submitting pull requests: ```bash # Install the npm linter globally (same version used in CI) npm install -g free-programming-books-lint # Lint the entire repository or specific directories fpb-lint books casts courses more ``` Local execution produces the same annotation-style output: ``` ::error file=books/free-programming-books.md,line=42::Duplicate entry "Effective Python" ::warning file=courses/free-programming-courses.md,line=108::Missing trailing space after "- " ``` ## URL Validation and Link Checking ### awesome_bot Configuration The [`.github/workflows/check-urls.yml`](https://github.com/EbookFoundation/free-programming-books/blob/main/.github/workflows/check-urls.yml) workflow validates hyperlink integrity using the Ruby gem `awesome_bot`. This check runs on both `push` and `pull_request` events, examining changed `*.md` and `*.yml` files. The tool verifies: - URL reachability (no 4xx or 5xx status codes) - Valid SSL certificates - Redirect handling according to repository policies - Duplicate URL detection Configuration flags include `--allow-redirect` and `--allow-dupe` to accommodate legitimate cases where redirects or duplicates are acceptable. Failures generate an artifact (`ab-results-*.json`) that subsequent steps process into GitHub annotations. ### Running Link Checks Locally Contributors can verify URLs before submission using the same tooling: ```bash # Install the Ruby gem gem install awesome_bot # Validate specific files with the same flags used in CI awesome_bot books/free-programming-books.md \ --allow-redirect \ --allow-dupe \ --allow-ssl || true ``` Typical failure output indicates specific problematic URLs: ``` [ERROR] https://dead.link.example/ returned 404 ``` ## Bidirectional Text Validation ### RTL/LTR Linter Implementation The repository maintains specific support for right-to-left (RTL) languages through [`.github/workflows/rtl-ltr-linter.yml`](https://github.com/EbookFoundation/free-programming-books/blob/main/.github/workflows/rtl-ltr-linter.yml). This workflow executes [`scripts/rtl_ltr_linter.py`](https://github.com/EbookFoundation/free-programming-books/blob/main/scripts/rtl_ltr_linter.py), a custom Python tool that validates bidirectional text handling in Markdown files. The script checks for: - Mixed RTL/LTR text without proper directionality markers - Missing Unicode directionality markers (`‏`, `‎`) - Unclosed or mismatched `
` tags - Keywords that require explicit directionality markers in RTL contexts Configuration resides in [`scripts/rtl_ltr_linter_config.yml`](https://github.com/EbookFoundation/free-programming-books/blob/main/scripts/rtl_ltr_linter_config.yml), defining keywords, symbols, and severity levels. The linter emits `::error` and `::warning` annotations compatible with GitHub Actions and writes detailed logs to `rtl-linter-output.log`. ### Local RTL/LTR Testing Contributors working with multilingual content can validate directionality locally: ```bash # Clone the repository and install Python dependencies python -m venv .venv && source .venv/bin/activate pip install python-bidi PyYAML # Execute the linter against specific files python scripts/rtl_ltr_linter.py books/free-programming-books.md \ --log-file rtl-linter-output.log ``` Sample output highlights specific directionality issues: ``` ::warning file=books/free-programming-books.md,line=123::Keyword 'HTML' in meta 'HTML' may need trailing '‏' marker. ::error file=books/free-programming-books.md,line=210::Pure LTR text 'C++' in meta of RTL context may need trailing '‏' marker. ``` ## Pull Request Automation ### Merge Conflict Detection The [`.github/workflows/detect-conflicting-prs.yml`](https://github.com/EbookFoundation/free-programming-books/blob/main/.github/workflows/detect-conflicting-prs.yml) workflow monitors PRs for merge conflicts using the third-party action `eps1lon/actions-label-merge-conflict`. Triggered on `push` and `pull_request_target` events, this automation labels PRs that cannot be merged automatically with a `conflicts` label. When conflicts are detected, the workflow posts a friendly comment explaining how to resolve the issue by updating the branch with the latest changes from the main branch. ### Automated PR Comments The [`.github/workflows/comment-pr.yml`](https://github.com/EbookFoundation/free-programming-books/blob/main/.github/workflows/comment-pr.yml) workflow provides feedback by posting comments on pull requests when validation fails. This workflow triggers after the lint workflow completes, downloading the `pr` artifact containing `error.log` and posting its contents as a comment. The workflow also manages the `linter error` label, adding it when errors exist and removing it once issues are resolved. This ensures contributors receive immediate notification of formatting issues without needing to navigate the Actions logs. ### Stale Pull Request Management Repository hygiene is maintained through [`.github/workflows/stale.yml`](https://github.com/EbookFoundation/free-programming-books/blob/main/.github/workflows/stale.yml), which runs daily via scheduled execution. This workflow marks PRs without recent activity as stale after 60 days and closes them after an additional 30 days unless they carry an exempt label. The workflow adds the `stale` label and posts an explanatory message, helping maintainers focus on active contributions while automatically cleaning up abandoned requests. This check does not affect mergeability but helps manage the review queue. ## Summary - **Six GitHub Actions workflows** validate every contribution to the Free-Programming-Books repository, running automatically on pull requests and pushes. - **Markdown linting** via `free-programming-books-lint` enforces canonical formatting, checking for duplicate entries, bullet syntax, and spacing issues. - **URL validation** using `awesome_bot` verifies that all hyperlinks return valid status codes, have working SSL certificates, and follow redirect policies. - **RTL/LTR validation** through a custom Python script ensures proper handling of bidirectional text, checking for missing directionality markers and unclosed HTML tags. - **PR automation** includes conflict detection, automated commenting on validation failures, and stale PR management to maintain repository hygiene. - **Local execution** is supported for all major checks, allowing contributors to validate changes before submitting pull requests. ## Frequently Asked Questions ### What happens if a URL check fails during automated testing? When the URL validation workflow detects broken links, it uploads the results as a JSON artifact and generates GitHub annotations on the specific lines containing problematic URLs. The `awesomebot-gh-summary-action` processes these results to create a summary view. While URL failures are reported prominently, they typically appear as warnings rather than hard blockers unless configured otherwise, allowing maintainers to decide whether to merge despite transient network issues or genuinely dead resources. ### Can I run the validation checks before submitting a pull request? Yes, all major validation checks support local execution. Install the `free-programming-books-lint` npm package to check Markdown formatting, use the `awesome_bot` Ruby gem to validate URLs, and run the Python script [`scripts/rtl_ltr_linter.py`](https://github.com/EbookFoundation/free-programming-books/blob/main/scripts/rtl_ltr_linter.py) with dependencies `python-bidi` and `PyYAML` to check bidirectional text. Running these tools locally produces identical output to the CI environment, allowing you to fix formatting errors, broken links, and directionality issues before opening your pull request. ### How does the repository handle right-to-left languages? The repository uses a custom Python linter located at [`scripts/rtl_ltr_linter.py`](https://github.com/EbookFoundation/free-programming-books/blob/main/scripts/rtl_ltr_linter.py) to validate Markdown containing Arabic, Hebrew, or other RTL languages. This tool checks for proper Unicode directionality markers (`‏` and `‎`), ensures mixed RTL/LTR text displays correctly, and validates that HTML directionality tags like `
` are properly closed. The workflow runs on every pull request, emitting GitHub annotations when directionality markers are missing or incorrectly applied, ensuring consistent rendering of multilingual content across the repository. ### What triggers the stale pull request workflow? The stale workflow runs on a daily schedule via GitHub Actions cron scheduling, and can also be triggered manually through workflow dispatch. It identifies pull requests that have been inactive for 60 days, adding the `stale` label and posting a comment to notify contributors. If no additional activity occurs within 30 days after being marked stale, the workflow automatically closes the pull request. Certain labels can exempt PRs from this process, ensuring that work-in-progress or high-priority contributions remain open despite temporary inactivity.