# How the Configuration Versioning System Enables Version-Specific Document Processing in AWS IDP

> Learn how the AWS IDP configuration versioning system processes documents using specific versions without redeployment. Discover how configuration snapshots and metadata enable precise document analysis.

- Repository: [aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws)
- Tags: deep-dive
- Published: 2026-02-25

---

**The AWS IDP configuration versioning system stores complete, compressed configuration snapshots in DynamoDB and attaches version metadata to S3 objects, allowing each document to be processed with its specific configuration version throughout the pipeline without stack redeployment.**

The accelerated intelligent document processing (IDP) solution on AWS uses a sophisticated **configuration versioning system** to manage OCR, classification, and extraction settings. This system enables teams to maintain multiple configuration snapshots—such as `default`, `Production`, or `Experiment-A`—and process documents against specific versions without redeploying infrastructure. Each version represents a complete, immutable snapshot stored in Amazon DynamoDB, ensuring reproducible processing across the entire document pipeline.

## How Configuration Snapshots Are Stored in DynamoDB

The system persists versioned configurations in a DynamoDB table named **ConfigurationTable**. Unlike incremental updates, each entry constitutes a **full, self-contained snapshot** containing every setting required for document processing.

### Full-Config Markers and Compression

To ensure completeness, the system marks snapshots with `_config_format: "full"` in [`lib/idp_common_pkg/idp_common/config/configuration_manager.py`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/lib/idp_common_pkg/idp_common/config/configuration_manager.py) (lines 31-34). This marker guarantees that the version contains complete OCR, classification, extraction, class, assessment, and summarization settings.

To overcome DynamoDB's 400 KB item size limit, configurations are gzip-compressed into the `_compressed_config` attribute (lines 35-39 in the same file). This compression enables storage of complex, multi-page extraction templates without truncation.

## Runtime Version Selection via S3 Metadata

Documents specify their target configuration version through **S3 object metadata**. When a file enters the system, the `config-version` metadata key determines which snapshot the pipeline loads.

### CLI and Batch Processing Injection

The `BatchProcessor` class in [`lib/idp_sdk/idp_sdk/core/batch_processor.py`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/lib/idp_sdk/idp_sdk/core/batch_processor.py) handles metadata attachment during uploads. When processing local files, it injects the version via `put_object` with the `Metadata` parameter:

```python
if config_version:
    logger.info(f"Adding config-version metadata: {config_version} to {s3_key}")
    self.s3.put_object(
        Bucket=input_bucket,
        Key=s3_key,
        Body=file_data,
        Metadata={"config-version": config_version},
    )

```

For existing S3 objects, the processor uses `copy_object` with `MetadataDirective="REPLACE"` to inject or update the version metadata (lines 62-68).

### Lambda Propagation

The `queue_sender` Lambda function in [`src/lambda/queue_sender/index.py`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/src/lambda/queue_sender/index.py) extracts the metadata and populates the `document.config_version` field (lines 49-71). This ensures the version identifier flows through the entire Step Functions workflow, making it available to every downstream processing stage.

## Version-Aware Processing in Pipeline Functions

Each processing stage—OCR, classification, extraction—retrieves its settings by calling `get_config()` with the specific version parameter.

For example, the OCR function in [`patterns/pattern-2/src/ocr_function/index.py`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/patterns/pattern-2/src/ocr_function/index.py) loads version-specific settings:

```python
from idp_common.config import ConfigurationReader

# version comes from the event payload (originally S3 metadata)

config_version = event.get("configVersion")
reader = ConfigurationReader(table_name="ConfigurationTable")
config = reader.get_merged_configuration(version=config_version, as_model=True)

```

This pattern repeats across classification, extraction, and assessment functions, ensuring each document processes against its designated configuration snapshot regardless of when it entered the system.

## Managing Versions Through the CLI and UI

The solution exposes version management through both command-line tools and the web interface.

### CLI Commands

The `idp-cli` tool provides direct version management as documented in [`docs/configuration-versions.md`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/docs/configuration-versions.md):

**Download a specific version:**

```bash
idp-cli config-download \
  --stack-name my-stack \
  --config-version Production

```

**Upload a new version:**

```bash
idp-cli config-upload \
  --stack-name my-stack \
  --config-file ./config.yaml \
  --config-version Experiment-A \
  --version-description "Testing nova-2-lite extraction prompts"

```

**Run inference with a chosen version:**

```bash
idp-cli run-inference \
  --stack-name my-stack \
  --dir ./documents/ \
  --config-version Production \
  --monitor

```

### Web Interface

The web UI provides a **Configuration Version dropdown** in the document upload, re-process, and discovery panels. This dropdown lists all versions stored in DynamoDB. When users select a version and upload a file, the UI attaches the `config-version` metadata to the S3 object, triggering the version-aware processing pipeline described above.

## Summary

The AWS IDP configuration versioning system enables precise, reproducible document processing through these key mechanisms:

- **Immutable snapshots**: Each version stores a complete, gzip-compressed configuration in DynamoDB with the `_config_format: "full"` marker, ensuring all OCR, classification, and extraction settings are self-contained.
- **Metadata-driven selection**: The `config-version` S3 object metadata determines which configuration snapshot processes each document, injected via the `BatchProcessor` in [`lib/idp_sdk/idp_sdk/core/batch_processor.py`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/lib/idp_sdk/idp_sdk/core/batch_processor.py).
- **Pipeline propagation**: The `queue_sender` Lambda extracts metadata into `document.config_version`, and each processing function calls `get_config(version=config_version)` to load settings, as seen in [`patterns/pattern-2/src/ocr_function/index.py`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/patterns/pattern-2/src/ocr_function/index.py).
- **Operational flexibility**: CLI commands and UI dropdowns enable version management without infrastructure redeployment, supporting A/B testing and staged rollouts.

## Frequently Asked Questions

### How does the system ensure a configuration version contains complete settings?

The system marks every stored version with `_config_format: "full"` in [`lib/idp_common_pkg/idp_common/config/configuration_manager.py`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/lib/idp_common_pkg/idp_common/config/configuration_manager.py). This marker guarantees that the snapshot includes all required sections—OCR engines, classification rules, extraction prompts, assessment criteria, and summarization parameters—preventing partial configuration errors during processing.

### What happens if a document is uploaded without specifying a configuration version?

If no `config-version` metadata is attached to the S3 object, the pipeline typically defaults to a version named `default` or falls back to the baseline configuration. The `get_config()` method in the configuration manager handles version resolution, ensuring documents always process against a valid configuration snapshot even when explicit versioning is omitted.

### How does the configuration versioning system handle large configuration files?

To overcome DynamoDB's 400 KB item size limit, the system compresses configuration JSON using gzip before storage. The `_compressed_config` attribute in the DynamoDB item stores this payload, while the `_config_format` field maintains the uncompressed marker. This compression enables storage of complex, multi-page extraction templates and extensive prompt libraries without truncation.

### Can I process the same document with different configuration versions for A/B testing?

Yes. The system supports A/B testing by allowing you to upload the same document multiple times with different `config-version` metadata values, or use the CLI to re-process existing documents with alternate versions. Each processing run retrieves its designated configuration snapshot from DynamoDB, enabling direct comparison of OCR accuracy, extraction quality, or classification performance across different prompt engineering experiments or model configurations.