# How to Deploy a Model on Vertex AI Using the generative-ai Repository Examples

> Deploy a model on Vertex AI using generative-ai examples. Learn to initialize the SDK, upload artifacts, create endpoints, and deploy your model quickly.

- Repository: [Google Cloud Platform/generative-ai](https://github.com/GoogleCloudPlatform/generative-ai)
- Tags: how-to-guide
- Published: 2026-03-09

---

**To deploy a model on Vertex AI using the GoogleCloudPlatform/generative-ai repository, initialize the Vertex AI SDK, upload your model artifact to Cloud Storage, create an endpoint, and call the `deploy()` method to bind the model to that endpoint.**

The **GoogleCloudPlatform/generative-ai** repository provides a complete, runnable example that demonstrates how to deploy a model on Vertex AI programmatically. The core deployment logic resides in [`gemini/evaluation/synthetic-data-evals/setup.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/evaluation/synthetic-data-evals/setup.py), which implements the full lifecycle from model upload to live endpoint deployment.

## Prerequisites for Vertex AI Model Deployment

Before you can deploy a model on Vertex AI, ensure you have the following configured:

- **Google Cloud SDK** installed with `gcloud` authenticated to your project
- **Vertex AI API** enabled in your GCP project
- **Service account** with `Vertex AI User` IAM role (requires permissions for `aiplatform.models.upload`, `aiplatform.endpoints.create`, and `aiplatform.models.deploy`)
- **Python environment** with `google-cloud-aiplatform` installed (`pip install google-cloud-aiplatform`)

## Step-by-Step Guide to Deploy a Model on Vertex AI

The deployment process follows six distinct phases as implemented in the repository's [`setup.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/setup.py) file.

### 1. Initialize the Vertex AI SDK

First, import the `google.cloud.aiplatform` module and initialize the client with your project ID and region. This configuration applies to all subsequent API calls.

```python
from google.cloud import aiplatform

PROJECT_ID = "my-gcp-project"
REGION = "us-central1"

# Initialize Vertex AI

aiplatform.init(project=PROJECT_ID, location=REGION)

```

In [`gemini/evaluation/synthetic-data-evals/setup.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/evaluation/synthetic-data-evals/setup.py), this initialization appears at the top of the script before any model operations.

### 2. Check for Existing Models

To avoid duplicate uploads, the repository example first queries existing models using a filter on the display name. This step is optional but recommended for idempotent deployment scripts.

```python
MODEL_DISPLAY_NAME = "my-custom-model"

# Check if model already exists

existing_models = aiplatform.Model.list(
    filter=f"display_name={MODEL_DISPLAY_NAME}"
)

if existing_models:
    model = existing_models[0]
    print(f"Using existing model: {model.resource_name}")
else:
    # Proceed to upload

    pass

```

This logic corresponds to line 111 in [`setup.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/setup.py), where `aiplatform.Model.list()` filters by display name.

### 3. Upload the Model Artifact

Upload your trained model to Vertex AI by specifying the Cloud Storage path containing your model artifacts and a serving container image. The repository demonstrates this at line 135.

```python
MODEL_GCS_PATH = "gs://my-bucket/path/to/model/"

# Upload model to Vertex AI

model = aiplatform.Model.upload(
    display_name=MODEL_DISPLAY_NAME,
    artifact_uri=MODEL_GCS_PATH,
    serving_container_image_uri="us-docker.pkg.dev/vertex-ai/prediction/gpt-2:latest",
    serving_container_predict_route="/predict",
    serving_container_health_route="/health"
)

model.wait()  # Block until upload completes

```

The `artifact_uri` must point to a Google Cloud Storage bucket containing your saved model files. The `serving_container_image_uri` specifies the prediction server that will load your model.

### 4. Create a Vertex AI Endpoint

An endpoint provides a stable HTTP URL for online predictions. Create one using `Endpoint.create()`, as shown at line 175 in the repository.

```python
ENDPOINT_DISPLAY_NAME = "my-model-endpoint"

# Create endpoint

endpoint = aiplatform.Endpoint.create(display_name=ENDPOINT_DISPLAY_NAME)
print(f"Endpoint created: {endpoint.resource_name}")

```

If you prefer to use an existing endpoint, you can retrieve it using `aiplatform.Endpoint.list()` similar to the model lookup pattern.

### 5. Deploy the Model to the Endpoint

Bind your uploaded model to the endpoint with the `deploy()` method. This step provisions compute resources and starts the model serving containers. The repository implements this at line 182.

```python
DEPLOYED_MODEL_DISPLAY_NAME = f"{MODEL_DISPLAY_NAME}-v1"

# Deploy model to endpoint

deployed_model = model.deploy(
    endpoint=endpoint,
    deployed_model_display_name=DEPLOYED_MODEL_DISPLAY_NAME,
    machine_type="n1-standard-4",
    min_replica_count=1,
    max_replica_count=1,
    traffic_split={"0": 100}  # Route 100% traffic to this model

)

print(f"Model deployed to endpoint: {endpoint.resource_name}")

```

The `machine_type` parameter specifies the compute resources (e.g., `n1-standard-4` for 4 vCPUs and 15 GB RAM). The `traffic_split` dictionary controls what percentage of requests route to this model version.

### 6. Verify the Deployment

After deployment, verify that the endpoint is serving predictions. The repository includes a verification print statement at line 188.

```python

# Verify deployment

print(f"Deployed model ID: {deployed_model.id}")
print(f"Endpoint URI: {endpoint.gca_resource.deployed_models[0].service_account}")

# Test prediction (example)

# prediction = endpoint.predict(instances=[{"input": "test"}])

```

You can now call `endpoint.predict()` to send inference requests to your deployed model.

## Complete Deployment Script Example

Here is the consolidated, runnable script adapted from [`gemini/evaluation/synthetic-data-evals/setup.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/evaluation/synthetic-data-evals/setup.py) that implements the full deployment pipeline:

```python
from google.cloud import aiplatform

# Configuration

PROJECT_ID = "my-gcp-project"
REGION = "us-central1"
MODEL_DISPLAY_NAME = "my-custom-model"
MODEL_GCS_PATH = "gs://my-bucket/path/to/model/"
ENDPOINT_DISPLAY_NAME = "my-model-endpoint"

# 1. Initialize Vertex AI

aiplatform.init(project=PROJECT_ID, location=REGION)

# 2. Check for existing model

existing_models = aiplatform.Model.list(filter=f"display_name={MODEL_DISPLAY_NAME}")
if existing_models:
    model = existing_models[0]
else:
    # 3. Upload new model

    model = aiplatform.Model.upload(
        display_name=MODEL_DISPLAY_NAME,
        artifact_uri=MODEL_GCS_PATH,
        serving_container_image_uri="us-docker.pkg.dev/vertex-ai/prediction/gpt-2:latest",
    )
    model.wait()

# 4. Create endpoint

endpoints = aiplatform.Endpoint.list(filter=f"display_name={ENDPOINT_DISPLAY_NAME}")
if endpoints:
    endpoint = endpoints[0]
else:
    endpoint = aiplatform.Endpoint.create(display_name=ENDPOINT_DISPLAY_NAME)

# 5. Deploy model to endpoint

deployed_model = model.deploy(
    endpoint=endpoint,
    deployed_model_display_name=f"{MODEL_DISPLAY_NAME}-v1",
    machine_type="n1-standard-4",
    min_replica_count=1,
    max_replica_count=1,
    traffic_split={"0": 100},
)

print(f"✅ Model deployed successfully to: {endpoint.resource_name}")

```

## Key Repository Files for Vertex AI Deployment

The **GoogleCloudPlatform/generative-ai** repository contains several relevant files for understanding Vertex AI deployment patterns:

- **[`gemini/evaluation/synthetic-data-evals/setup.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/evaluation/synthetic-data-evals/setup.py)** – The primary reference implementation demonstrating model upload, endpoint creation, and deployment with traffic splitting.
- **[`setup-env/README.md`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/setup-env/README.md)** – Instructions for configuring Vertex AI Workbench environments, useful for interactive deployment testing.
- **[`tools/llmevalkit/README.md`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/tools/llmevalkit/README.md)** – Documentation for Vertex AI Prompt Optimization, showing advanced serving patterns.
- **[`search/web-app/README.md`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/search/web-app/README.md)** – Examples of enabling Vertex AI Search API, demonstrating complementary serving infrastructure.

## Summary

- **Initialize** the Vertex AI SDK with `aiplatform.init()` to configure your project and region.
- **Upload** model artifacts using `aiplatform.Model.upload()`, specifying the Cloud Storage path and serving container image.
- **Create** an endpoint with `aiplatform.Endpoint.create()` to provide a stable prediction URL.
- **Deploy** using `model.deploy()` to bind the model to the endpoint with specified machine types and traffic allocation.
- **Verify** deployment by checking the endpoint resource name and testing predictions.

## Frequently Asked Questions

### What permissions are required to deploy a model on Vertex AI?

You need the **Vertex AI User** IAM role or equivalent custom permissions including `aiplatform.models.upload`, `aiplatform.endpoints.create`, and `aiplatform.models.deploy`. Additionally, you need `storage.objects.get` access to the Cloud Storage bucket containing your model artifacts.

### Can I deploy multiple models to the same endpoint?

Yes. You can deploy multiple models to a single endpoint and control traffic splitting between them using the `traffic_split` parameter in the `deploy()` method. For example, set `traffic_split={"0": 70, "1": 30}` to route 70% of requests to the first model and 30% to the second.

### What serving container images should I use for custom models?

Vertex AI provides pre-built container images for common frameworks like TensorFlow, PyTorch, scikit-learn, and XGBoost. For custom prediction servers, you can use any container image that implements the Vertex AI Predict protocol. The repository example uses `us-docker.pkg.dev/vertex-ai/prediction/gpt-2:latest`, but you should replace this with an image matching your model's framework and version.