How to Run Tests in Onyx: Backend, Frontend, and E2E Testing Guide

You can run Onyx's full test suite using pytest for Python backend code, npm test for React frontend components, and npx playwright test for end-to-end UI flows.

Onyx (formerly Danswer) is an open-source AI question-answering platform with a comprehensive test suite covering Python APIs, React components, and full browser automation. This guide walks through executing each layer of the testing pyramid using commands and scripts defined in the onyx-dot-app/onyx repository.

Install Repository Dependencies

Before running any tests, install both Python and Node.js dependencies from the repository root.

For the backend, Onyx uses UV for dependency management:

# Install UV if you haven't already

curl -LsSf https://astral.sh/uv/install.sh | sh

# Sync Python dependencies

uv sync

The uv sync command installs exact versions from backend/requirements/ as specified in the repository.

For the frontend, ensure Node.js 18+ is installed, then fetch JavaScript dependencies:

cd web
npm ci

The npm ci command installs exact versions from package-lock.json, including testing frameworks like Jest and Playwright.

Start the Full Development Stack

Onyx requires backend services (Postgres, Redis, Vespa, etc.) to be running for most integration tests. Start the complete Docker Compose stack:

./deployment/docker_compose/install.sh
docker compose up -d

The install.sh script creates a .env file with sensible defaults, eliminating the need for manual secret configuration. Once healthy, the API is available at http://localhost:8080 and the web UI at http://localhost:3000.

Run Backend Tests with pytest

The Python test suite uses pytest with plugins including pytest-asyncio, pytest-playwright, and pytest-xdist.

Run All Backend Tests

Execute the complete Python test suite from the repository root:

pytest

Run Specific Test Files or Cases

Target individual test files or single test functions for faster feedback:

pytest backend/tests/unit/onyx/utils/test_vespa_query.py
pytest backend/tests/unit/onyx/utils/test_vespa_query.py::test_query_success

Parallel Execution

Speed up large test runs with parallel workers:

pytest -n auto

The -n auto flag utilizes pytest-xdist to distribute tests across available CPU cores.

Sandbox Container Testing

For CI parity, Onyx provides a helper script that spins up an isolated sandbox and executes tests in the same environment used by production workers:

bash backend/onyx/server/features/build/sandbox/kubernetes/docker/run-test.sh \
    backend/tests/unit/onyx/utils/test_vespa_query.py

The script prints "=== Running tests ===" before invoking pytest inside the container, as implemented in backend/onyx/server/features/build/sandbox/kubernetes/docker/run-test.sh.

Run Frontend Unit Tests with Jest

Frontend unit and integration tests use Jest with React Testing Library. All commands are defined in web/package.json.

Available npm Scripts

• npm test — Run the full suite once
• npm run test:watch — Re-run tests on file changes
• npm run test:ci — CI-friendly run with limited workers
• npm run test:changed — Run only files modified since the last commit
• npm run test:debug — Debug a single test with the Node inspector

Running Specific Tests

Execute a single test file or filter by pattern:

# Run a specific component test

npm test -- src/app/auth/login/EmailPasswordForm.test.tsx

# Run all auth-related tests

npm test -- --testPathPattern="auth"

# Generate coverage report

npm test -- --coverage

These scripts are configured in web/package.json at lines 20-27, with additional documentation available in web/tests/README.md.

Run End-to-End Tests with Playwright

E2E UI tests verify complete user flows using Playwright. Test specs live in web/tests/e2e/ and require the full development stack to be running.

Basic Playwright Commands

Run the complete E2E suite headlessly:

npx playwright test

Run a specific spec file or use interactive modes:

# Single spec file

npx playwright test web/tests/e2e/onboarding/onboarding_flow.spec.ts

# Interactive UI mode (opens a clickable interface)

npx playwright test --ui

# Watch mode for development

npx playwright test --watch

Playwright binaries are installed automatically during npm ci, but you can manually ensure browsers are present with npx playwright install.

Continuous Integration Reference

The repository defines CI workflows that mirror local testing commands. The workflow in .github/workflows/pr-jest-tests.yml executes:

npm test -- --ci --coverage --maxWorkers=50%

Backend CI utilizes the same run-test.sh sandbox script mentioned above, ensuring local and CI environments remain consistent.

Summary

• Install dependencies using uv sync for Python and npm ci for Node.js
• Start services with ./deployment/docker_compose/install.sh followed by docker compose up -d
• Run backend tests using pytest or the sandbox helper at backend/onyx/server/features/build/sandbox/kubernetes/docker/run-test.sh
• Run frontend tests using npm test with filters for specific files or patterns
• Run E2E tests using npx playwright test against the running development stack
• Accelerate testing with pytest -n auto for parallel Python execution or npm run test:watch for continuous Jest feedback

Frequently Asked Questions

How do I run a single test file in Onyx?

For Python, pass the file path directly to pytest: pytest backend/tests/unit/onyx/utils/test_vespa_query.py. For frontend, append the path after the npm command: npm test -- src/app/auth/login/EmailPasswordForm.test.tsx. Both approaches support filtering down to individual test functions using :: (Python) or pattern matching (Jest).

What is the difference between Jest and Playwright tests in Onyx?

Jest tests in web/ target individual React components and utility functions in isolation, running in a simulated DOM environment without a browser. Playwright tests in web/tests/e2e/ launch real Chromium/Firefox/WebKit browsers to interact with the live application at localhost:3000, verifying complete user workflows including network requests and navigation.

Do I need to start Docker services before running tests?

Yes, for integration and E2E tests that interact with Postgres, Redis, or Vespa. Start the stack using ./deployment/docker_compose/install.sh and docker compose up -d. However, pure unit tests that mock external dependencies can run without Docker, though the full test suite assumes services are available at the ports defined in your .env file.

How does Onyx run tests in CI/CD?

Onyx uses GitHub Actions defined in .github/workflows/. Frontend PRs trigger pr-jest-tests.yml, which runs npm test -- --ci --coverage. Backend tests execute inside the sandbox container via backend/onyx/server/features/build/sandbox/kubernetes/docker/run-test.sh, ensuring the CI environment matches production deployment conditions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}