# GKE Provisioning Gotchas: 15 Critical Pitfalls to Avoid in Google Kubernetes Engine

> Avoid GKE provisioning pitfalls. Discover 15 critical gotchas in Google Kubernetes Engine related to networking, ComputeClass, and GPU quotas. Ensure smooth deployments.

- Repository: [Google/skills](https://github.com/google/skills)
- Tags: best-practices
- Published: 2026-09-02

---

**GKE provisioning failures most often stem from immutable Day-0 networking decisions, ComputeClass misconfigurations that invalidate committed use discounts (CUDs), and missing regional quotas for GPU workloads that leave clusters in partially-provisioned states.**

Provisioning a Google Kubernetes Engine cluster appears straightforward through the Google Cloud Console or `gcloud` CLI, but the default "golden path" conceals architectural constraints that are expensive or impossible to reverse. According to the `google/skills` repository documentation, understanding these GKE provisioning subtleties before creation prevents the costly rebuilds, security exposures, and billing surprises that plague production deployments.

## Irreversible Day-0 Architecture Decisions

Several GKE configuration options become immutable after cluster creation. Changing these later requires a complete cluster rebuild, causing service downtime and migration overhead.

### Private Cluster and VPC-Native Networking

The `enablePrivateNodes` and `useIpAliases` parameters defined in [`skills/cloud/gke-cluster-creation/SKILL.md`](https://github.com/google/skills/blob/main/skills/cloud/gke-cluster-creation/SKILL.md) are **immutable flags**. If you create a public cluster initially, you cannot later convert it to a private cluster without rebuilding. Exposing the Kubernetes control plane to the public internet creates a significant attack surface that should be avoided for production workloads.

**Best practice:** Enable private nodes and master authorized networks during initial provisioning:

```bash
gcloud container clusters create-auto my-prod-cluster \
  --region us-central1 \
  --enable-private-nodes \
  --enable-master-authorized-networks \
  --enable-dns-access

```

### Release Channel Selection

Selecting the `RAPID` release channel for production environments introduces instability. As documented in the cluster creation skill, faster channels receive new features but also new bugs. Production workloads should use `REGULAR` or `STABLE` channels to limit unexpected breakages from upstream Kubernetes changes.

## ComputeClass and Cost Optimization Traps

ComputeClasses abstract node provisioning but introduce subtle billing and availability risks documented in [`skills/cloud/gke-compute-classes/references/compute-class-gotchas-and-cuds.md`](https://github.com/google/skills/blob/main/skills/cloud/gke-compute-classes/references/compute-class-gotchas-and-cuds.md).

### CUD Invalidation Through Machine Family Fallback

**ComputeClasses provision node VMs, but the discount is applied to the actual VM type that the class resolves to.** If a ComputeClass falls back to an `e2` node instead of the intended `n4` family, you lose any `n4`-family Committed Use Discounts. This silently increases compute costs despite having active CUD contracts.

Always verify which machine family your ComputeClass resolves to before relying on CUD savings.

### Spot VM Preemption Risks

Spot instances can be reclaimed at any time. According to [`skills/cloud/gke-compute-classes/references/compute-class-provisioning-methods.md`](https://github.com/google/skills/blob/main/skills/cloud/gke-compute-classes/references/compute-class-provisioning-methods.md), workloads that cannot tolerate preemptions will suffer outages if you use Spot VMs without a fallback strategy.

Implement a fallback node pool or set `maxUnavailable=1` in the upgrade strategy to ensure workload continuity during preemption events.

## GPU and TPU Quota Validation

Requesting GPU-enabled node pools without verifying regional quotas causes provisioning failures. As noted in [`skills/cloud/gke-cluster-creation/SKILL.md`](https://github.com/google/skills/blob/main/skills/cloud/gke-cluster-creation/SKILL.md), provisioning a `g2-standard-4` (L4) or `a3-highgpu-8g` (H100) requires **specific regional quota** that is not enabled by default.

The creation will fail mid-provision, leaving a partially-provisioned cluster that requires manual cleanup. Validate GPU quotas in the Google Cloud Console before executing:

```bash
cat > gpu-compute-class.yaml <<EOF
apiVersion: scheduling.gke.io/v1beta1
kind: ComputeClass
metadata:
  name: l4-gpu
spec:
  machineType: g2-standard-4
  accelerators:
  - type: nvidia-l4
    count: 1
EOF
kubectl apply -f gpu-compute-class.yaml

```

## Security and Identity Misconfigurations

### Workload Identity vs. Static Service Account Keys

Using static service-account keys for workload authentication creates credential theft risks. Keys can be leaked and require manual rotation. The security section of [`skills/cloud/gke-cluster-creation/SKILL.md`](https://github.com/google/skills/blob/main/skills/cloud/gke-cluster-creation/SKILL.md) recommends **Workload Identity**, which automatically injects short-lived tokens and dramatically reduces the blast radius of credential compromise.

## Autoscaling and Resource Management

### Cluster Autoscaler Without HPA/VPA

Enabling the **Cluster Autoscaler** without configuring the Horizontal Pod Autoscaler (HPA) or Vertical Pod Autoscaler (VPA) leads to resource starvation. As documented in [`skills/cloud/gke-cost-optimization/SKILL.md`](https://github.com/google/skills/blob/main/skills/cloud/gke-cost-optimization/SKILL.md), pods can be throttled when node capacity is insufficient, causing OOMKill loops even with autoscaling enabled.

Explicitly enable both features:
- `--enable-autoscaling` for node-level scaling
- `--enable-vertical-pod-autoscaling` for right-sizing container requests

### ResourceQuota Missing Default Requests

Defining `min` and `max` limits in a ResourceQuota without specifying `default` and `defaultRequest` values causes deployment failures. Per [`skills/cloud/gke-multitenancy/SKILL.md`](https://github.com/google/skills/blob/main/skills/cloud/gke-multitenancy/SKILL.md), pods lacking explicit resource requests are rejected by the admission controller. Always define default requests to ensure pods can schedule without manual resource specification.

## Multi-Tenancy and Network Isolation

### Default-Deny NetworkPolicy DNS Blocking

Applying a default-deny NetworkPolicy without explicit egress rules blocks all traffic, including DNS resolution. The network isolation example in [`skills/cloud/gke-multitenancy/SKILL.md`](https://github.com/google/skills/blob/main/skills/cloud/gke-multitenancy/SKILL.md) warns that this causes "pod cannot resolve host" errors.

Implement explicit DNS egress rules:

```bash
cat > allow-same-ns.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-same-namespace
  namespace: team-a
spec:
  podSelector: {}
  ingress:
  - from:
    - podSelector: {}
  egress:
  - to:
    - podSelector: {}
  - to:
    - namespaceSelector: {}
      podSelector:
        matchLabels:
          k8s-app: kube-dns
    ports:
    - protocol: UDP
      port: 53
EOF
kubectl apply -f allow-same-ns.yaml

```

### Cluster-Per-Team Cost Overruns

Jumping directly to a **Cluster-per-Team** isolation model is the most expensive and operationally heavy approach. According to [`skills/cloud/gke-multitenancy/SKILL.md`](https://github.com/google/skills/blob/main/skills/cloud/gke-multitenancy/SKILL.md), you should start with **Namespace-per-Team** and only escalate to cluster separation when compliance mandates require it. This avoids unnecessary management overhead and control plane costs.

## Cost Allocation Labeling

Forgetting to label namespaces for billing prevents cost attribution. Without `cost-center` labels, GKE cost allocation cannot break down spend by team, making charge-back impossible. Implement labels during namespace creation:

```bash
kubectl label namespace team-a cost-center=engineering
kubectl label namespace team-b cost-center=data-science

```

## Summary

- **Validate immutability:** Private cluster settings, VPC-native networking, and release channels cannot be changed after creation according to [`skills/cloud/gke-cluster-creation/SKILL.md`](https://github.com/google/skills/blob/main/skills/cloud/gke-cluster-creation/SKILL.md).
- **Verify ComputeClass resolution:** Ensure ComputeClasses resolve to the intended machine family to preserve CUD benefits.
- **Check regional quotas:** GPU and TPU workloads require explicit quota verification before provisioning.
- **Enable Workload Identity:** Replace static service account keys with short-lived tokens for security.
- **Configure complete autoscaling:** Cluster Autoscaler must be paired with HPA/VPA to prevent resource throttling.
- **Default to Namespace-per-Team:** Reserve Cluster-per-Team for strict compliance requirements only.
- **Label early:** Implement cost-center labels at namespace creation to enable charge-back.

## Frequently Asked Questions

### Can I convert a Standard GKE cluster to Autopilot after creation?

No. The choice between **Standard** and **Autopilot** modes is immutable. If you provision Standard but later require Autopilot's managed node capabilities, you must create a new cluster and migrate workloads. According to [`skills/cloud/gke-cluster-creation/SKILL.md`](https://github.com/google/skills/blob/main/skills/cloud/gke-cluster-creation/SKILL.md), Autopilot abstracts node management but cannot accommodate custom kernel modules, privileged DaemonSets, or specific GPU configurations available in Standard mode.

### Why does my ComputeClass not apply my Committed Use Discount?

ComputeClasses provision VMs based on availability and specifications, but **CUDs apply to the actual resolved VM type**, not the ComputeClass abstraction. If your ComputeClass falls back to a different machine family than your CUD covers (e.g., falling back to `e2` when you have `n4` CUDs), you pay on-demand rates. Verify the machine family mapping in [`skills/cloud/gke-compute-classes/references/compute-class-gotchas-and-cuds.md`](https://github.com/google/skills/blob/main/skills/cloud/gke-compute-classes/references/compute-class-gotchas-and-cuds.md) before assuming discount coverage.

### How do I prevent DNS resolution failures after applying NetworkPolicies?

A default-deny NetworkPolicy blocks all egress traffic, including DNS queries to kube-dns. You must explicitly allow egress to the `kube-dns` service on UDP port 53. As shown in [`skills/cloud/gke-multitenancy/SKILL.md`](https://github.com/google/skills/blob/main/skills/cloud/gke-multitenancy/SKILL.md), add a specific egress rule targeting the namespace selector with label `k8s-app: kube-dns` alongside your application traffic rules.

### What is the difference between Cluster Autoscaler and Horizontal Pod Autoscaler?

**Cluster Autoscaler** adjusts the number of nodes in your pool based on pending pod resource requests, while **Horizontal Pod Autoscaler** adjusts the number of pod replicas based on CPU or custom metrics. Enabling only the Cluster Autoscaler results in pods stuck in Pending state during traffic spikes because individual pods lack resource headroom. Enable both features as documented in [`skills/cloud/gke-cost-optimization/SKILL.md`](https://github.com/google/skills/blob/main/skills/cloud/gke-cost-optimization/SKILL.md) for efficient scaling.