# Implementing PII De-identification with shift_dates Redaction in OpenMed

> Implement PII de-identification with OpenMed shift_dates. Securely shift dates for HIPAA compliance while retaining formatting and chronological utility for analysis.

- Repository: [Maziyar Panahi/openmed](https://github.com/maziyarpanahi/openmed)
- Tags: how-to-guide
- Published: 2026-06-12

---

**The `shift_dates` method in OpenMed shifts detected DATE entities by a configurable number of days while preserving original formatting and language-specific month names, enabling HIPAA-compliant temporal de-identification without destroying chronological utility for downstream analysis.**

OpenMed provides a robust PII de-identification pipeline that supports multiple redaction strategies for sensitive clinical data. When implementing **PII de-identification with shift_dates redaction**, you transform exact temporal information by offsetting dates rather than masking them, preserving essential timeline relationships for research while removing identifiable specific dates.

## How the Pipeline Works

The de-identification process follows a three-stage architecture: entity detection, method dispatch, and date transformation.

### Entity Detection and Unified Output

The pipeline begins with a privacy-filter model (such as `openai/privacy-filter` or multilingual equivalents) that identifies PII spans in clinical text. Detection results are standardized across backends (MLX and Torch) through the `PrivacyFilterTorchPipeline` class in [`openmed/torch/privacy_filter.py`](https://github.com/maziyarpanahi/openmed/blob/main/openmed/torch/privacy_filter.py). Each detected entity returns a dictionary containing `entity_group`, `start`, `end`, `score`, and `word` keys, ensuring consistent handling regardless of the underlying inference engine.

### The Deidentify Entry Point

The core orchestration occurs in the `deidentify` function exposed from `openmed.core.pii`. This function iterates over detected entities and dispatches to specific redaction helpers based on the chosen method:

- **mask**: Replaces spans with `[MASKED]` placeholders
- **redact**: Replaces spans with generic `[REDACTED]` tags  
- **shift_dates**: Transforms DATE entities by applying a day offset while preserving format

The method validates conflicting parameters—setting `shift_dates=False` while providing `date_shift_days` raises a `ValueError` as verified in [`tests/unit/test_pii.py`](https://github.com/maziyarpanahi/openmed/blob/main/tests/unit/test_pii.py).

### Date-Shifting Implementation Details

The temporal transformation logic resides in two private helpers within [`openmed/core/pii.py`](https://github.com/maziyarpanahi/openmed/blob/main/openmed/core/pii.py):

**`_shift_date`** (line 1175): Parses date strings and applies offsets while maintaining input style. It first attempts localized month-name parsing via `_parse_localized_month_date`, then falls back to `dateutil` when available, or finally to `_shift_date_basic` for pure-regex handling.

**`_shift_date_basic`** (line 1389): A regex-driven parser supporting US (`MM/DD/YYYY`), European (`DD/MM/YYYY`), German (`DD.MM.YYYY`), ISO formats, and month-name variants. The `keep_year` flag enables partial de-identification by mutating only month and day components while preserving the year.

When parsing fails, the pipeline returns `"[DATE_SHIFTED]"` as a safe fallback placeholder.

### Language Support and Localization

Multilingual date handling relies on the `LANGUAGE_MONTH_NAMES` mapping defined in [`openmed/core/pii_i18n.py`](https://github.com/maziyarpanahi/openmed/blob/main/openmed/core/pii_i18n.py). This lookup table enables accurate shifting of dates containing localized month names in Dutch, Hindi, Telugu, Portuguese, and other languages without requiring the heavyweight `dateutil` dependency, ensuring culturally consistent output across clinical narratives.

## Practical Implementation Examples

### Basic Usage with the High-Level API

The simplest implementation uses the `deidentify` function with the `shift_dates` method:

```python
from openmed.core import deidentify

text = "Patient was admitted on 01/15/2020 and discharged on 02/20/2020."
result = deidentify(
    text,
    method="shift_dates",
    date_shift_days=30,
    keep_year=True,
)

print(result.redacted_text)

# Output: "Patient was admitted on 02/14/2020 and discharged on 03/21/2020."

```

This call path flows through [`openmed/core/pii.py`](https://github.com/maziyarpanahi/openmed/blob/main/openmed/core/pii.py) → `_redact_entity` → `_shift_date`.

### Direct Date String Manipulation

For custom pipelines requiring single-date processing, use the internal `_shift_date` helper directly:

```python
from openmed.core.pii import _shift_date

original = "15 janvier 2020"  # French format

shifted = _shift_date(original, 30, lang="fr")
print(shifted)  # Output: "14 février 2020"

```

### REST API Integration

Deploy the functionality via the FastAPI service defined in [`openmed/service/app.py`](https://github.com/maziyarpanahi/openmed/blob/main/openmed/service/app.py):

```python
import requests

payload = {
    "text": "Patient arrived on 2020-01-15.",
    "method": "shift_dates",
    "date_shift_days": 30,
}
resp = requests.post("http://localhost:8000/pii/deidentify", json=payload)
print(resp.json()["redacted_text"])

# Output: "Patient arrived on 2020-02-14."

```

### Batch Processing via CLI

Process large datasets using the built-in command-line interface:

```bash
openmed pii deidentify \
    --input-file notes.txt \
    --output-file deid_notes.txt \
    --method shift_dates \
    --date-shift-days 30

```

The CLI invokes the same `deidentify` function, ensuring consistency across interfaces.

## Key Source Files and Architecture

Understanding the codebase structure helps when extending or debugging the shift_dates functionality:

- **[`openmed/core/pii.py`](https://github.com/maziyarpanahi/openmed/blob/main/openmed/core/pii.py)**: Contains the `deidentify` function, `_shift_date` (line 1175), and `_shift_date_basic` (line 1389) implementations
- **[`openmed/core/pii_i18n.py`](https://github.com/maziyarpanahi/openmed/blob/main/openmed/core/pii_i18n.py)**: Houses `LANGUAGE_MONTH_NAMES` for multilingual month resolution
- **[`openmed/torch/privacy_filter.py`](https://github.com/maziyarpanahi/openmed/blob/main/openmed/torch/privacy_filter.py)**: Provides `PrivacyFilterTorchPipeline` for unified entity detection across backends
- **[`tests/unit/test_pii.py`](https://github.com/maziyarpanahi/openmed/blob/main/tests/unit/test_pii.py)**: Validates method behavior, alias handling, and error conditions
- **[`openmed/service/app.py`](https://github.com/maziyarpanahi/openmed/blob/main/openmed/service/app.py)**: FastAPI router exposing the REST endpoint

## Summary

- **PII de-identification with shift_dates redaction** modifies DATE content by applying configurable day offsets rather than masking or removing temporal data
- The implementation preserves original formatting and language-specific month names through the `LANGUAGE_MONTH_NAMES` mapping in [`openmed/core/pii_i18n.py`](https://github.com/maziyarpanahi/openmed/blob/main/openmed/core/pii_i18n.py)
- Core logic resides in [`openmed/core/pii.py`](https://github.com/maziyarpanahi/openmed/blob/main/openmed/core/pii.py) via `_shift_date` and `_shift_date_basic`, supporting the `keep_year` flag for partial de-identification
- Failed parsing falls back to `"[DATE_SHIFTED]"` placeholder to prevent data leakage
- Available through Python API, REST service ([`openmed/service/app.py`](https://github.com/maziyarpanahi/openmed/blob/main/openmed/service/app.py)), and CLI with consistent parameter validation

## Frequently Asked Questions

### What happens when shift_dates cannot parse a date format?

When the parser encounters unsupported formats, it returns the string `"[DATE_SHIFTED]"` as a safe fallback. This prevents original date exposure while indicating that temporal transformation was attempted. The regex-based `_shift_date_basic` handles most numeric formats (US, European, ISO), while localized month names require entries in `LANGUAGE_MONTH_NAMES`.

### Does shift_dates work with multilingual clinical notes?

Yes. The implementation resolves localized month names through `LANGUAGE_MONTH_NAMES` in [`openmed/core/pii_i18n.py`](https://github.com/maziyarpanahi/openmed/blob/main/openmed/core/pii_i18n.py), supporting languages including Dutch, Hindi, Telugu, and Portuguese. This allows accurate shifting of dates written in local languages without requiring the `dateutil` library dependency.

### How does keep_year affect the de-identification process?

Setting `keep_year=True` restricts transformation to month and day components only, leaving the year unchanged. This provides partial de-identification suitable for studies requiring temporal ordering across years while reducing re-identification risk from specific dates. The flag is processed in `_shift_date_basic` at line 1389 of [`openmed/core/pii.py`](https://github.com/maziyarpanahi/openmed/blob/main/openmed/core/pii.py).

### Can I combine shift_dates with other redaction methods?

No, the `method` parameter accepts a single strategy. However, you can implement hybrid approaches by running the pipeline twice—first with `shift_dates` to preserve temporal relationships, then with `mask` or `redact` for other entity types. The pipeline validates parameters and raises `ValueError` if you attempt conflicting configurations like setting `date_shift_days` while using `method="mask"`.