# GCP Elasticsearch Security and Performance: A Complete Configuration Guide

> Master GCP Elasticsearch security and performance. Learn to secure credentials, enforce TLS, and optimize with client reuse and tuned retry policies for peak efficiency.

- Repository: [elastic/elasticsearch](https://github.com/elastic/elasticsearch)
- Tags: how-to-guide
- Published: 2026-02-16

---

**When running GCP Elasticsearch, secure service-account credentials via the keystore, enforce TLS for all GCS and Vertex AI traffic, and optimize performance through client reuse, regional PUE selection, and tuned retry policies.**

Running Elasticsearch on Google Cloud Platform (GCP) integrates the search engine with Google-native services such as Cloud Storage for snapshots and Vertex AI for inference. Because GCP introduces distinct authentication models, networking patterns, and regional characteristics, configuring **gcp elasticsearch** requires specific attention to credential management, encryption, and client performance tuning.

## Securing GCP Elasticsearch: Identity and Access Management

### Service Account Credentials and Secure Settings

The `repository-gcs` module authenticates to Google Cloud Storage using service account credentials. According to the Elasticsearch source code, you must never store the JSON key file in plain text. Instead, add it to the Elasticsearch keystore as a secure setting.

In [`modules/repository-gcs/src/main/java/org/elasticsearch/repositories/gcs/GoogleCloudStorageService.java`](https://github.com/elastic/elasticsearch/blob/main/modules/repository-gcs/src/main/java/org/elasticsearch/repositories/gcs/GoogleCloudStorageService.java) (lines 84–92), the code loads credentials via:

```java
GoogleCredentials credentials = ServiceAccountCredentials.fromStream(
    Files.newInputStream(Paths.get(credentialPath)));

```

The `credentialPath` resolves from the `CREDENTIALS_FILE_SETTING` secure setting. For Vertex AI integration, the same pattern appears in [`x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/googlevertexai/request/GoogleVertexAiRequest.java`](https://github.com/elastic/elasticsearch/blob/main/x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/googlevertexai/request/GoogleVertexAiRequest.java) (lines 10–38), where `GoogleCredentials` are built from the environment or explicit service account streams.

### IAM Role Minimization

Assign the service account the minimal IAM roles required:

- **GCS Snapshots**: `roles/storage.objectAdmin` (or `roles/storage.admin` if managing buckets)
- **Vertex AI**: `roles/aiplatform.user`
- **Monitoring**: `roles/monitoring.viewer` (optional, for GCP-based metrics ingestion)

## Network Security and Encryption

### TLS Configuration and Trust Stores

All traffic to GCS and Vertex AI must traverse HTTPS. The `GoogleCloudStorageService.createClient` method (lines 48–66) constructs an `HttpTransport` with a dedicated trust store:

```java
final NetHttpTransport.Builder builder = new NetHttpTransport.Builder();
KeyStore trustStore = SecurityUtils.getJavaKeyStore();
try (InputStream ks = GoogleUtils.class.getResourceAsStream("google.jks")) {
    SecurityUtils.loadKeyStore(trustStore, ks, "notasecret");
}
builder.trustCertificates(trustStore);
HttpTransport httpTransport = builder.build();

```

This ensures TLS certificate validation against Google's root CAs.

### Proxy and Private Connectivity

If your cluster operates behind a corporate proxy, configure `gcs.client.*.proxy` settings. The code in `GoogleCloudStorageService` checks `GoogleCloudStorageClientSettings.getProxy()` and injects it into the transport builder:

```java
Proxy proxy = gcsClientSettings.getProxy();
if (proxy != null) {
    builder.setProxy(proxy);
}

```

For private VPC access without internet egress, use **Private Service Connect** or configure a custom `endpoint` in the GCS client settings to point to your VPC-SC perimeter.

## Performance Optimization for GCP Elasticsearch

### Client Caching and Thread Safety

The `GoogleCloudStorageService` maintains a cache of thread-safe clients to avoid the overhead of repeated authentication and TCP handshakes. The `client()` method (lines 10–18) returns a `MeteredStorage` instance wrapping a cached `Storage` object:

```java
public MeteredStorage client(String clientName) {
    // Returns cached, thread-safe client
    return clients.computeIfAbsent(clientName, this::createClient);
}

```

Reuse this client across snapshot operations to maximize connection pooling.

### Timeout and Retry Configuration

Tune the GCS client via settings defined in `GoogleCloudStorageClientSettings`:

```json
{
  "gcs.client.gcs_client.connect_timeout": "5s",
  "gcs.client.gcs_client.read_timeout": "30s",
  "gcs.client.gcs_client.max_retries": 5
}

```

These map to `CONNECT_TIMEOUT_SETTING`, `READ_TIMEOUT_SETTING`, and `MAX_RETRIES_SETTING`. The `getRetryStrategy()` method (lines 99–105) implements exponential back-off for transient 5xx errors and `SocketException`.

### Regional Selection and Sustainability Metrics

The `cloud-profiling` plugin contains GCP-specific Power Usage Effectiveness (PUE) and CO₂ emission factors in [`x-pack/plugin/profiling/src/main/java/org/elasticsearch/xpack/profiling/action/CloudProviders.java`](https://github.com/elastic/elasticsearch/blob/main/x-pack/plugin/profiling/src/main/java/org/elasticsearch/xpack/profiling/action/CloudProviders.java) (lines 33–67). Selecting regions with lower PUE improves both sustainability and cooling efficiency:

```java
double pue = CloudProviders.getPUEOrDefault("gcp", "northamerica-northeast1", 1.15);
double co2 = CloudProviders.getCO2TonsPerKWHOrDefault("gcp", "northamerica-northeast1", 0.0);

```

Regions like `northamerica-northeast1` report 0 t CO₂/kWh due to hydroelectric power, reducing your cluster's carbon footprint.

## Credential Lifecycle and Multi-Project Support

In multi-project (MP) clusters, the `GoogleCloudStorageClientsManager` applies cluster-state updates so each project maintains its own cached client. When credentials rotate, trigger `refreshAndClearCache` to force re-creation:

```java
googleCloudStorageService.refreshAndClearCache(newSettings);

```

This clears the client cache, forcing a fresh credential load on the next snapshot operation.

## Serverless and Observability Considerations

When deploying on GCP Cloud Run or App Engine, `DiscoveryNode.isStateless` marks the node as serverless (`isServerless` in `GoogleCloudStorageService`, lines 70–73). In this mode, certain features like snapshot repository caches are disabled because the process may be short-lived.

For observability, GCP metrics (CPU, network) can be shipped to Stackdriver via the generic monitoring exporter, though the core implementation resides in the cloud-monitoring module rather than GCP-specific code.

## Summary

- **Store credentials securely**: Use the Elasticsearch keystore for service account JSON keys, referenced by `GoogleCloudStorageService` via secure settings.
- **Enforce TLS and minimize IAM roles**: Configure HTTPS with the `google.jks` trust store and assign only `roles/storage.objectAdmin` or `roles/aiplatform.user` as needed.
- **Optimize client performance**: Reuse thread-safe GCS clients, tune `connect_timeout`, `read_timeout`, and `max_retries` in `GoogleCloudStorageClientSettings`.
- **Select sustainable regions**: Use [`CloudProviders.java`](https://github.com/elastic/elasticsearch/blob/main/CloudProviders.java) PUE and CO₂ data to choose low-impact GCP regions like `northamerica-northeast1`.
- **Handle credential rotation**: Call `refreshAndClearCache` to update clients in multi-project deployments without restarting nodes.

## Frequently Asked Questions

### How do I securely store GCP service account keys for Elasticsearch?

Add the JSON key file to the Elasticsearch keystore using `bin/elasticsearch-keystore add-file gcs.client.my_client.credentials_file`, then reference it in [`elasticsearch.yml`](https://github.com/elastic/elasticsearch/blob/main/elasticsearch.yml) with `gcs.client.my_client.credentials_file: ${file.reference}`. The `GoogleCloudStorageService` loads this via `ServiceAccountCredentials.fromStream` without exposing the key in plain text configuration files.

### What IAM roles does GCP Elasticsearch need for GCS snapshots?

Assign the service account `roles/storage.objectAdmin` for bucket object operations, or `roles/storage.admin` if the cluster must create and delete buckets. For Vertex AI inference, add `roles/aiplatform.user`. Avoid using primitive roles like `roles/editor` or `roles/owner` as they violate the principle of least privilege.

### How does GCP Elasticsearch handle credential rotation?

When credentials rotate, update the secure setting in the keystore and trigger a cache refresh. In `GoogleCloudStorageService`, the `refreshAndClearCache` method invalidates the cached `Storage` clients, forcing the next snapshot operation to load the new credentials via `ServiceAccountCredentials.fromStream`. This allows rotation without cluster restart.

### Can I run GCP Elasticsearch in a private VPC without internet access?

Yes. Configure Private Service Connect or VPC Service Controls, then set the `endpoint` parameter in your GCS client settings to point to your private endpoint. The `GoogleCloudStorageService.createClient` method respects the `endpoint` setting and can route traffic through a proxy configured via `gcs.client.*.proxy` settings, enabling air-gapped deployments.