# Best Practices for Deploying Oracle AI Applications on Kubernetes

> Deploy Oracle AI applications on Kubernetes using Terraform, Kustomize, OCI Vault and CI CD for production-grade security, scalability and observability.

- Repository: [Oracle Developers/oracle-ai-developer-hub](https://github.com/oracle-devrel/oracle-ai-developer-hub)
- Tags: best-practices
- Published: 2026-05-10

---

**Deploy Oracle AI applications on Kubernetes using Infrastructure-as-Code with Terraform, environment-specific Kustomize overlays, secure OCI Vault secret injection, and automated CI/CD pipelines to ensure production-grade security, scalability, and observability.**

Deploying Oracle AI workloads on Oracle Container Engine for Kubernetes (OKE) requires a cloud-native approach that balances security with operational efficiency. The reference implementation in the `oracle-devrel/oracle-ai-developer-hub` repository demonstrates a production-grade deployment pattern for generative AI applications. This guide covers the essential best practices for deploying Oracle AI applications on Kubernetes, from infrastructure provisioning to runtime security and observability.

## Infrastructure as Code with Terraform

Provision all OKE resources using **Terraform** to ensure repeatable, version-controlled infrastructure. In `apps/oci-generative-ai-jet-ui/deploy/terraform/oke.tf`, the cluster configuration automatically selects the latest supported Kubernetes version via `local.cluster_k8s_latest_version`, ensuring you run on a patched, supported release.

Define your node pool capacity explicitly using the `worker_pools` block. The reference implementation specifies `worker_pool_size: 2` with `VM.Standard.E5.Flex` shapes, providing predictable compute capacity for AI inference workloads.

```bash

# Initialise Terraform providers

terraform init

# Create the OKE cluster, VCN, subnets, and security lists

terraform apply --auto-approve

```

## Environment Management with Kustomize

Manage multiple deployment environments using **Kustomize** overlays to keep a single source of truth for base resources while customizing per-environment settings. Store environment-specific manifests under `deploy/k8s/overlays/` (e.g., `prod`), adjusting namespaces, replica counts, and image tags without duplicating YAML.

Apply the production configuration using the overlay path:

```bash

# Apply the production overlay

kubectl apply -k deploy/k8s/overlays/prod

```

The [`kustomization.yaml`](https://github.com/oracle-devrel/oracle-ai-developer-hub/blob/main/kustomization.yaml) in `apps/oci-generative-ai-jet-ui/deploy/k8s/overlays/prod/` maps the base resources to production-specific patches, enabling clean separation between development and production environments.

## Secure Secrets Management

Never hardcode credentials in container images. Instead, use the `scripts/setenv.mjs` helper script to generate [`genai.json`](https://github.com/oracle-devrel/oracle-ai-developer-hub/blob/main/genai.json), which assembles OCI Vault secrets (API keys and endpoints) for injection as environment variables. This decouples sensitive credentials from your codebase and runtime images.

Generate the secure environment configuration:

```bash

# Produce genai.json containing OCI_GENAI_API_KEY and OCI_GENAI_ENDPOINT

npx zx scripts/setenv.mjs

```

For database connections, configure **private endpoints** so traffic between your pods and Oracle AI Database never traverses the public internet, eliminating exposure to external threats.

## Networking and Ingress Configuration

Deploy a **managed NGINX Ingress controller** with TLS termination to handle external traffic securely. The repository includes a ready-made configuration in [`apps/oci-generative-ai-jet-ui/deploy/k8s/ingress/ingress-controller.yaml`](https://github.com/oracle-devrel/oracle-ai-developer-hub/blob/main/apps/oci-generative-ai-jet-ui/deploy/k8s/ingress/ingress-controller.yaml), which uses the Kubernetes-recommended `app.kubernetes.io/*` label scheme for service discovery and monitoring.

Ensure all ingress resources follow labeling standards for consistent network policy application and load balancer health check configuration.

## Resource Management and Scaling

Define explicit **CPU and memory limits** in your deployment manifests to prevent resource starvation and ensure predictable performance. The reference OKE node pool configuration provides a fixed capacity envelope that aligns with your AI workload requirements.

When scaling, leverage the node pool architecture defined in `oke.tf` to maintain adequate capacity for both inference and training workloads without over-provisioning.

## CI/CD Automation

Automate the build and deployment pipeline using the repository's helper scripts. The `scripts/release.mjs` script builds Docker images and pushes them to Oracle Container Registry, while `scripts/kustom.mjs` regenerates Kustomize overlay files with updated image tags.

Integrate these into your CI/CD pipeline (GitHub Actions, OCI Code Repository, or GitLab CI) to achieve fully automated rollouts:

```bash

# Build and push container images

npx zx scripts/release.mjs

# Regenerate overlay configurations

npx zx scripts/kustom.mjs

```

Point `kubectl` to the generated kubeconfig before deployment:

```bash
export KUBECONFIG="$(pwd)/deploy/terraform/generated/kubeconfig"
kubectl cluster-info
kubectl apply -k deploy/k8s/overlays/prod

```

Retrieve the external load balancer IP to verify service exposure:

```bash
kubectl get service -n backend -o jsonpath='{.items[?(@.spec.type=="LoadBalancer")].status.loadBalancer.ingress[0].ip}'

```

## Observability and Monitoring

Enable the OKE **Monitoring** add-on to leverage OCI Logging and Metrics for cluster-wide observability. The deployed `backend` service writes interaction records to the autonomous database, providing a built-in audit trail via queries like `SELECT * FROM interactions;`.

Monitor node pool health, pod resource utilization, and ingress traffic patterns to maintain service level objectives for your AI applications.

## Summary

- **Use Terraform** (`oke.tf`) to provision OKE clusters with automatic version selection and defined node pools (`VM.Standard.E5.Flex`, `worker_pool_size: 2`)
- **Implement Kustomize overlays** (`deploy/k8s/overlays/prod`) for environment-specific configurations without code duplication
- **Secure credentials** via `scripts/setenv.mjs` and OCI Vault injection, avoiding hardcoded secrets
- **Configure private endpoints** for database connectivity and managed NGINX Ingress with TLS for external traffic
- **Automate deployments** with `scripts/release.mjs` and `scripts/kustom.mjs` integrated into CI/CD pipelines
- **Enable observability** through OKE Monitoring add-ons and database audit trails

## Frequently Asked Questions

### How do I manage secrets for Oracle AI applications in Kubernetes?

Use the `scripts/setenv.mjs` script in the repository to generate [`genai.json`](https://github.com/oracle-devrel/oracle-ai-developer-hub/blob/main/genai.json), which pulls credentials from OCI Vault and formats them for injection as environment variables. This approach decouples secrets from container images and source code, adhering to the principle of least privilege.

### What is the recommended way to handle different deployment environments?

Store base Kubernetes manifests in `deploy/k8s/base/` and create environment-specific overlays in `deploy/k8s/overlays/` (e.g., `prod`). Use Kustomize to apply patches for namespace, replica count, and image tag variations, then deploy with `kubectl apply -k deploy/k8s/overlays/prod`.

### How do I ensure my OKE cluster runs the latest Kubernetes version?

The Terraform configuration in `apps/oci-generative-ai-jet-ui/deploy/terraform/oke.tf` uses `local.cluster_k8s_latest_version` to automatically select the newest supported Kubernetes version during provisioning. Run `terraform plan` regularly to detect version updates and apply changes through your Infrastructure-as-Code workflow.

### What networking configuration is recommended for database connectivity?

Configure **private endpoints** for Oracle AI Database connections to ensure traffic remains within the Oracle Cloud Infrastructure network and never traverses the public internet. Combine this with security lists and network policies defined in your Terraform (`oke.tf`) to enforce zero-trust networking between pods and data stores.