# How to Use AWS4Auth for IAM Authentication with Amazon OpenSearch Service

> Learn how to use AWS4Auth for secure IAM authentication with Amazon OpenSearch Service. Sign HTTP requests with temporary IAM credentials for keyless access.

- Repository: [AWS Samples/sample-for-amazon-opensearch-service-tutorials-101](https://github.com/aws-samples/sample-for-amazon-opensearch-service-tutorials-101)
- Tags: how-to-guide
- Published: 2026-02-25

---

**AWS4Auth from the requests-aws4auth library signs HTTP requests using temporary IAM credentials, enabling secure, keyless authentication between AWS Lambda and Amazon OpenSearch Service without storing long-term access keys in code.**

The aws-samples/sample-for-amazon-opensearch-service-tutorials-101 repository demonstrates production-ready patterns for connecting serverless applications to Amazon OpenSearch Service. This guide explains how AWS4Auth implements IAM-based request signing to authenticate Lambda functions securely.

## Understanding AWS4Auth and IAM Authentication for OpenSearch

AWS4Auth implements the **AWS Signature Version 4** signing process, which is required to authenticate HTTP requests to Amazon OpenSearch Service when using IAM-based access control. This approach works with both classic OpenSearch domains (service name `es`) and serverless collections (service name `aoss`).

The key advantage is **zero credential storage**. Instead of embedding access keys in your code or environment variables, AWS4Auth leverages the Lambda execution role's temporary credentials, which are rotated automatically by the AWS runtime.

## Step-by-Step Implementation of AWS4Auth in Python

### Retrieving Temporary IAM Credentials with boto3

The Lambda runtime provides temporary credentials through the boto3 credential chain. In [`artifacts/search_lambda/opensearch_search.py`](https://github.com/aws-samples/sample-for-amazon-opensearch-service-tutorials-101/blob/main/artifacts/search_lambda/opensearch_search.py) and [`artifacts/index_lambda/opensearch_index.py`](https://github.com/aws-samples/sample-for-amazon-opensearch-service-tutorials-101/blob/main/artifacts/index_lambda/opensearch_index.py), the code retrieves these credentials at initialization:

```python
import boto3

# Retrieve credentials from the Lambda execution role

credentials = boto3.Session().get_credentials()

```

The `get_credentials()` method returns an object containing `access_key`, `secret_key`, and `token` (the session token required for temporary credentials).

### Creating the AWS4Auth Signer

With credentials in hand, you instantiate the `AWS4Auth` class from the requests-aws4auth package. This is implemented in lines 14-27 of both Lambda files:

```python
from requests_aws4auth import AWS4Auth

SERVICE = "es"  # Use "aoss" for OpenSearch Serverless

REGION = "us-east-1"  # Or retrieve from environment

awsauth = AWS4Auth(
    credentials.access_key,
    credentials.secret_key,
    REGION,
    SERVICE,
    session_token=credentials.token,
)

```

**Critical detail:** The `session_token` parameter is essential when running in Lambda because the execution role uses temporary credentials. Omitting this parameter results in authentication failures.

### Configuring the OpenSearch Client

The final step passes the `awsauth` signer to the OpenSearch client as the `http_auth` parameter. Lines 33-41 in the repository files show this configuration:

```python
from opensearchpy import OpenSearch, RequestsHttpConnection

ops_client = OpenSearch(
    hosts=[{"host": ENDPOINT, "port": 443}],
    http_auth=awsauth,
    use_ssl=True,
    verify_certs=True,
    connection_class=RequestsHttpConnection,
    timeout=300,
)

```

The `RequestsHttpConnection` class is required because it respects the `http_auth` parameter and automatically adds the `Authorization` header (containing the AWS4-HMAC-SHA256 signature) to every HTTP request.

## Complete Working Example from the Repository

Here is the consolidated pattern used in [`artifacts/search_lambda/opensearch_search.py`](https://github.com/aws-samples/sample-for-amazon-opensearch-service-tutorials-101/blob/main/artifacts/search_lambda/opensearch_search.py) and [`artifacts/index_lambda/opensearch_index.py`](https://github.com/aws-samples/sample-for-amazon-opensearch-service-tutorials-101/blob/main/artifacts/index_lambda/opensearch_index.py):

```python
import os
import boto3
import json
from opensearchpy import OpenSearch, RequestsHttpConnection
from requests_aws4auth import AWS4Auth

# Configuration from environment variables

ENDPOINT = os.getenv("OPENSEARCH_HOST")
REGION = os.getenv("AWS_REGION", "us-east-1")
INDEX_NAME = os.getenv("INDEX_NAME", "products")
SERVICE = "es"  # Change to "aoss" for serverless

# Initialize AWS4Auth with Lambda execution role credentials

credentials = boto3.Session().get_credentials()
awsauth = AWS4Auth(
    credentials.access_key,
    credentials.secret_key,
    REGION,
    SERVICE,
    session_token=credentials.token,
)

# Create OpenSearch client with IAM authentication

ops_client = OpenSearch(
    hosts=[{"host": ENDPOINT, "port": 443}],
    http_auth=awsauth,
    use_ssl=True,
    verify_certs=True,
    connection_class=RequestsHttpConnection,
    timeout=300,
)

def lambda_handler(event, context):
    # Example search implementation

    body = json.loads(event["body"])
    query = {
        "multi_match": {
            "query": body.get("searchTerm", ""),
            "fields": ["title^3", "description^2", "color"],
            "type": "best_fields"
        }
    }
    
    response = ops_client.search(index=INDEX_NAME, body={"query": query})
    return {
        "statusCode": 200,
        "body": json.dumps({"hits": response["hits"]["hits"]})
    }

```

The [`search_tutorials/opensearch_proxy_stack.py`](https://github.com/aws-samples/sample-for-amazon-opensearch-service-tutorials-101/blob/main/search_tutorials/opensearch_proxy_stack.py) file in the same repository defines the CDK stack that provisions these Lambda functions and attaches the necessary IAM permissions to the execution role.

## IAM Permissions and Security Considerations

For the signing process to succeed, the Lambda execution role must have appropriate permissions on the OpenSearch resource. According to the repository's CDK implementation in [`search_tutorials/opensearch_proxy_stack.py`](https://github.com/aws-samples/sample-for-amazon-opensearch-service-tutorials-101/blob/main/search_tutorials/opensearch_proxy_stack.py), the role requires:

- For classic domains: `es:ESHttpGet`, `es:ESHttpPost`, `es:ESHttpPut`, `es:ESHttpDelete` (or wildcard `es:ESHttp*`)
- For serverless collections: `aoss:APIAccess` and `aoss:CollectionAdmin` permissions

When the OpenSearch client makes a request, the `AWS4Auth` signer generates a signature based on the request parameters and the temporary credentials. The OpenSearch service validates this signature against the IAM policy attached to the credentials' role. If the signature is valid and the policy permits the action, the request is authorized.

**Security best practice:** This approach eliminates the need to store long-term AWS credentials in environment variables or code. The temporary credentials provided by the Lambda runtime are automatically rotated and have a limited lifetime, reducing the risk of credential exposure.

## Summary

- **AWS4Auth** bridges boto3's credential handling with HTTP requests to OpenSearch, implementing AWS Signature Version 4.
- The signer requires temporary credentials from `boto3.Session().get_credentials()`, including the `session_token` for Lambda execution roles.
- Service name parameters differ by deployment type: use `"es"` for managed domains and `"aoss"` for serverless collections.
- The OpenSearch client must use `RequestsHttpConnection` as the connection class to properly apply the `http_auth` signer.
- IAM permissions on the Lambda execution role (`es:ESHttp*` or `aoss:APIAccess`) are validated through the signed request, enabling zero-trust authentication without stored credentials.

## Frequently Asked Questions

### What is the difference between using "es" and "aoss" as the service parameter in AWS4Auth?

The service parameter determines the AWS service namespace used in the Signature Version 4 calculation. Use `"es"` when connecting to a managed Amazon OpenSearch Service domain (the classic Elasticsearch-compatible service). Use `"aoss"` when connecting to Amazon OpenSearch Serverless collections. The signing algorithm remains identical, but the service name affects the canonical request string and the endpoint routing validation performed by AWS.

### Why does AWS4Auth require a session_token when running in AWS Lambda?

AWS Lambda functions assume an IAM execution role at runtime, which generates temporary security credentials consisting of an access key ID, secret access key, and session token. The session token is required to validate that the credentials are temporary and to prevent replay attacks. When constructing `AWS4Auth`, you must pass `session_token=credentials.token` to include this token in the Authorization header; otherwise, OpenSearch will reject the request with a 403 Forbidden error.

### Can I use AWS4Auth with Amazon OpenSearch Serverless?

Yes, AWS4Auth fully supports Amazon OpenSearch Serverless. You must set the service parameter to `"aoss"` instead of `"es"` when instantiating the signer. Additionally, your Lambda execution role must have permissions for `aoss:APIAccess` and appropriate collection-level permissions (such as `aoss:CollectionAdmin` or `aoss:CollectionRead`). The authentication flow and code structure remain identical to managed domain implementations.

### Do I need to store AWS credentials in environment variables to use AWS4Auth?

No, storing long-term credentials in environment variables is unnecessary and discouraged when using AWS4Auth with Lambda. The library retrieves credentials dynamically from the boto3 credential provider chain, which automatically sources temporary credentials from the Lambda execution role via the instance metadata service. You only need to call `boto3.Session().get_credentials()` to access the runtime-provided credentials, eliminating the security risks associated with hardcoded or environment-stored access keys.