# How to Run OpenMed Tests: A Complete Guide to Unit, Integration, and Evaluation Testing

> Learn how to run OpenMed tests with this guide. Execute unit, integration, and evaluation tests easily using pytest commands from the repository root.

- Repository: [Maziyar Panahi/openmed](https://github.com/maziyarpanahi/openmed)
- Tags: how-to-guide
- Published: 2026-06-13

---

**Run OpenMed tests using pytest after installing development dependencies with `pip install -e ".[dev]"`, then execute `pytest -m unit` for unit tests, `pytest -m integration` for integration tests, or `pytest -m eval` for evaluation tests from the repository root.**

OpenMed is a Python package for medical text processing that includes a comprehensive test suite covering unit, integration, and evaluation checks. To ensure code quality and reliability, the project uses **pytest** as its testing framework with tests organized under the `tests/` directory. This guide explains how to set up your environment and execute the OpenMed test suite efficiently according to the maziyarpanahi/openmed source code.

## Prerequisites and Installation

Before running tests, install the package in editable mode with development dependencies. The [`pyproject.toml`](https://github.com/maziyarpanahi/openmed/blob/main/pyproject.toml) file defines these under `[tool.poetry.dev-dependencies]`, including `pytest`, `pytest-cov`, `torch`, and `datasets`.

Clone the repository and run the installation command:

```bash
git clone https://github.com/maziyarpanahi/openmed.git
cd openmed
pip install -e ".[dev]"

```

The `.[dev]` extra ensures all testing utilities and compatible versions of PyTorch are available.

## Test Suite Architecture

The OpenMed test suite follows a structured directory pattern under `tests/` with separate categories for different testing levels.

### Directory Structure

The repository organizes tests into three distinct categories:

- **`tests/unit/`** – Contains **unit tests** for individual components like the core processing pipeline in [`tests/unit/test_core.py`](https://github.com/maziyarpanahi/openmed/blob/main/tests/unit/test_core.py). These tests validate isolated functionality without external dependencies.
- **`tests/integration/`** – Houses **integration tests** such as [`tests/integration/test_end_to_end.py`](https://github.com/maziyarpanahi/openmed/blob/main/tests/integration/test_end_to_end.py) that verify the full service API and component interactions.
- **`tests/eval/`** – Includes **evaluation tests** like [`tests/eval/test_metrics.py`](https://github.com/maziyarpanahi/openmed/blob/main/tests/eval/test_metrics.py) that compare model outputs against golden fixtures to ensure accuracy.

### Fixtures and Configuration

The [`tests/conftest.py`](https://github.com/maziyarpanahi/openmed/blob/main/tests/conftest.py) file defines global pytest fixtures including temporary directories, deterministic random seeds, and reusable objects like `Processor` and `ModelRegistry`. Category-specific fixtures in [`tests/unit/conftest.py`](https://github.com/maziyarpanahi/openmed/blob/main/tests/unit/conftest.py) provide mock services and sample medical texts for their respective test suites.

## Running the OpenMed Test Suite

Execute tests from the repository root using pytest commands. The test discovery pattern follows `test_*.py` conventions throughout the codebase.

### Run All Tests

To execute the complete suite including unit, integration, and evaluation tests:

```bash
pytest

```

### Run Specific Test Categories

Use pytest markers to filter by test type:

```bash

# Run only unit tests

pytest -m unit

# Run only integration tests  

pytest -m integration

# Run only evaluation tests

pytest -m eval

```

These markers correspond to the directory structure and are configured in the project configuration files.

### Run with Coverage

Generate coverage reports to identify untested code paths:

```bash
pytest --cov=openmed --cov-report=html
open htmlcov/index.html

```

The `--cov=openmed` flag tracks coverage specifically for the main package source code under the `openmed/` directory.

## CI/CD Pipeline Configuration

The repository uses GitHub Actions to automate testing on every push. The workflow file [`.github/workflows/ci.yml`](https://github.com/maziyarpanahi/openmed/blob/main/.github/workflows/ci.yml) implements the following steps:

1. **Environment Setup** – Installs the package with `pip install -e ".[dev]"`
2. **Caching** – Preserves `~/.cache` directories for torch and datasets to accelerate subsequent runs
3. **Test Execution** – Runs `pytest --cov=openmed` to collect coverage metrics

This configuration ensures that tests pass consistently across different environments and that coverage thresholds are maintained.

## Troubleshooting Common Test Failures

When running OpenMed tests, you may encounter specific errors related to dependencies or environment configuration.

**`ImportError: cannot import name 'torch'`** – Indicates PyTorch is missing or incorrectly versioned. Resolve by reinstalling with `pip install -e ".[dev]"` to pull compatible torch builds.

**Tests failing on data download** – Network restrictions or missing cached datasets prevent HuggingFace datasets from loading. Ensure internet access or pre-populate `~/.cache/huggingface/datasets` before running tests.

**Slow test execution** – Large model checkpoints downloading repeatedly causes delays. The CI caches `~/.cache` locally; ensure you reuse this path or set the appropriate cache environment variables.

**`AssertionError` in privacy-filter tests** – These tests use placeholder values and do not require the `OPENMED_PRIVACY_KEY` environment variable. No real secrets are needed for the test suite to pass.

## Summary

- **Install** OpenMed with development dependencies using `pip install -e ".[dev]"` before testing
- **Structure** includes three categories: `tests/unit/`, `tests/integration/`, and `tests/eval/`
- **Execute** specific test types with markers: `pytest -m unit`, `pytest -m integration`, or `pytest -m eval`
- **Configure** coverage reporting with `pytest --cov=openmed` to track code quality
- **Reference** [`.github/workflows/ci.yml`](https://github.com/maziyarpanahi/openmed/blob/main/.github/workflows/ci.yml) for the official continuous integration setup

## Frequently Asked Questions

### Do I need a GPU to run OpenMed tests?

No, the OpenMed test suite runs on CPU. While the package supports GPU acceleration for inference, the pytest fixtures in [`tests/conftest.py`](https://github.com/maziyarpanahi/openmed/blob/main/tests/conftest.py) configure tests to use CPU-only mode by default, ensuring compatibility across all development environments.

### How do I run a single test file instead of the entire suite?

Use the file path as an argument to pytest. For example, to run only the core unit tests: `pytest tests/unit/test_core.py`. You can also run specific test functions using the `::` syntax: `pytest tests/unit/test_core.py::test_specific_function`.

### What should I do if integration tests fail due to missing external services?

The integration tests in [`tests/integration/test_end_to_end.py`](https://github.com/maziyarpanahi/openmed/blob/main/tests/integration/test_end_to_end.py) use mock services configured in [`tests/integration/conftest.py`](https://github.com/maziyarpanahi/openmed/blob/main/tests/integration/conftest.py). If failures occur, verify that fixtures are properly loading by checking that you installed all dev dependencies. No external service endpoints are required for the test suite to execute.

### Where are the test fixtures and sample data defined?

Global fixtures reside in [`tests/conftest.py`](https://github.com/maziyarpanahi/openmed/blob/main/tests/conftest.py), while category-specific fixtures appear in subdirectories like [`tests/unit/conftest.py`](https://github.com/maziyarpanahi/openmed/blob/main/tests/unit/conftest.py). These files define reusable objects including sample processors, model registries, and medical text samples that inject deterministic test data into the suite.