How to Run the Stirling-PDF Test Suite Using the test.sh Script

Run ./test.sh from the repository root to execute the full CI-style test suite, which builds Docker images, launches Docker Compose stacks, and validates the application across Ultra-Lite, Full-Fat, and Security configurations.

The test.sh script serves as the primary entry point for running the comprehensive Stirling-PDF test suite in the Stirling-Tools/Stirling-PDF repository. Located at testing/test.sh, this orchestrator automates Docker image builds, container orchestration, and multi-phase validation to ensure both core and advanced PDF features function correctly before deployment.

Prerequisites for Running the Test Suite

Before executing the script, ensure Docker and Docker Compose (version 2 or higher) are installed and the current user has permissions to run Docker commands. The test suite builds multiple container images including ultra-lite and fat variants, requiring sufficient RAM and CPU resources—particularly for the Full-Fat image which bundles LibreOffice and heavy dependencies.

Running the Full Test Suite

To execute all four test phases (Ultra-Lite, Full-Fat+Security, Regression, and Disabled-Endpoints), navigate to the repository root and invoke the script without arguments:

cd /path/to/Stirling-PDF
./test.sh

The script performs the following actions in sequence:

• Builds the Ultra-Lite image if required (logic at lines 34-46 in test.sh)
• Builds the Fat image with security enabled (logic at lines 81-95)
• Launches each Docker-Compose configuration and waits for the HTTP health endpoint /api/v1/info/status to become reachable via the check_health function (lines 34-84)
• Executes functional, accessibility, and regression validations
• Generates a summary of passed or failed tests and creates a .failed_tests file for later re-execution

Selective Test Execution with --rerun and --rerun-failed

The script supports targeted execution to speed up development workflows without running the entire suite.

Rerunning Failed Tests Only

After a run produces failures, the script saves the names of failing suites to a .failed_tests file in the working directory. To re-execute only these tests, use the --rerun-failed flag, which invokes the load_failed_tests() function (lines 22-33) to parse the failure list:

./test.sh --rerun-failed

Running Specific Test Suites

To run individual test phases or specific combinations, provide a comma-separated list of test names using the --rerun flag. The argument parsing logic (lines 81-106) splits the input into the RERUN_TESTS array:

./test.sh --rerun "Stirling-PDF-Ultra-Lite,Webpage-Accessibility-full"

This approach skips unnecessary image builds and targets only the specified validation logic.

Understanding the Test Phases

The test.sh script validates the application through four distinct phases, each using specific Docker Compose files located in docker/embedded/compose/:

• Ultra-Lite: Tests core PDF features without extra add-ons, using docker-compose-latest-ultra-lite.yml and the ultra-lite image
• Full-Fat + Security: Exercises the complete feature set with security enabled, using the fat image and docker-compose-latest-fat-security.yml
• Regression (login + behave): Executes end-to-end UI tests using Cucumber/Behave features stored in testing/cucumber, orchestrated via test_cicd.yml
• Disabled-Endpoints: Verifies that optional API endpoints are correctly disabled, utilizing test_disabledEndpoints.sh and docker-compose-latest-fat-endpoints-disabled.yml

Additional helper scripts testing/test_webpages.sh and testing/test_disabledEndpoints.sh handle webpage accessibility and endpoint validation respectively.

How the Script Validates the Application

The check_health function (lines 34-84) performs HTTP-based health checks by polling the /api/v1/info/status endpoint until the container reports ready. Version verification steps analyze the JSON response from this endpoint to confirm the correct application version is running (lines 30-63).

If a test fails, the script automatically prints Docker logs via docker logs "$container_name" for debugging purposes and records the failure. Passed tests display in green (e.g., ✅ Stirling-PDF-Ultra-Lite) while failures appear in red with diagnostic output.

Summary

• Execute ./test.sh to run the complete Stirling-PDF test suite with automatic Docker orchestration and health validation
• Use --rerun-failed to re-execute only the tests listed in the .failed_tests file, leveraging the load_failed_tests() function
• Target specific suites with --rerun "Test1,Test2" for faster iteration and to skip costly image rebuilds
• The script validates container readiness against the /api/v1/info/status endpoint before executing test logic

Frequently Asked Questions

Where is the test.sh script located in the Stirling-PDF repository?

The script is located at testing/test.sh in the repository root. You must run it from the repository root directory so it can correctly locate the Docker Compose files in docker/embedded/compose/ and related build contexts.

How does the script determine which Docker images to build?

According to the source code in testing/test.sh (lines 34-46 and 81-95), the script conditionally builds the Ultra-Lite image for core feature tests and the Fat image with security enabled for comprehensive testing. The build logic triggers automatically based on which test suites are selected to run.

Can I run the test suite without rebuilding Docker images every time?

Yes. While the default behavior includes build steps, you can use the --rerun flag to execute specific tests against existing images. For example, running ./test.sh --rerun "Webpage-Accessibility-lite" skips the costly rebuild steps and immediately launches the container if the image already exists locally.

What happens if a test fails during execution?

When a test fails, the script records the failing suite name to a .failed_tests file in the working directory and prints relevant Docker logs for debugging. You can then use ./test.sh --rerun-failed to execute only the failed tests again, which is handled by the load_failed_tests() function at lines 22-33 of the script.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}