How to Deploy a Model on Vertex AI Using the generative-ai Repository Examples

To deploy a model on Vertex AI using the GoogleCloudPlatform/generative-ai repository, initialize the Vertex AI SDK, upload your model artifact to Cloud Storage, create an endpoint, and call the deploy() method to bind the model to that endpoint.

The GoogleCloudPlatform/generative-ai repository provides a complete, runnable example that demonstrates how to deploy a model on Vertex AI programmatically. The core deployment logic resides in gemini/evaluation/synthetic-data-evals/setup.py, which implements the full lifecycle from model upload to live endpoint deployment.

Prerequisites for Vertex AI Model Deployment

Before you can deploy a model on Vertex AI, ensure you have the following configured:

• Google Cloud SDK installed with gcloud authenticated to your project
• Vertex AI API enabled in your GCP project
• Service account with Vertex AI User IAM role (requires permissions for aiplatform.models.upload, aiplatform.endpoints.create, and aiplatform.models.deploy)
• Python environment with google-cloud-aiplatform installed (pip install google-cloud-aiplatform)

Step-by-Step Guide to Deploy a Model on Vertex AI

The deployment process follows six distinct phases as implemented in the repository's setup.py file.

1. Initialize the Vertex AI SDK

First, import the google.cloud.aiplatform module and initialize the client with your project ID and region. This configuration applies to all subsequent API calls.

from google.cloud import aiplatform

PROJECT_ID = "my-gcp-project"
REGION = "us-central1"

# Initialize Vertex AI

aiplatform.init(project=PROJECT_ID, location=REGION)

In gemini/evaluation/synthetic-data-evals/setup.py, this initialization appears at the top of the script before any model operations.

2. Check for Existing Models

To avoid duplicate uploads, the repository example first queries existing models using a filter on the display name. This step is optional but recommended for idempotent deployment scripts.

MODEL_DISPLAY_NAME = "my-custom-model"

# Check if model already exists

existing_models = aiplatform.Model.list(
    filter=f"display_name={MODEL_DISPLAY_NAME}"
)

if existing_models:
    model = existing_models[0]
    print(f"Using existing model: {model.resource_name}")
else:
    # Proceed to upload

    pass

This logic corresponds to line 111 in setup.py, where aiplatform.Model.list() filters by display name.

3. Upload the Model Artifact

Upload your trained model to Vertex AI by specifying the Cloud Storage path containing your model artifacts and a serving container image. The repository demonstrates this at line 135.

MODEL_GCS_PATH = "gs://my-bucket/path/to/model/"

# Upload model to Vertex AI

model = aiplatform.Model.upload(
    display_name=MODEL_DISPLAY_NAME,
    artifact_uri=MODEL_GCS_PATH,
    serving_container_image_uri="us-docker.pkg.dev/vertex-ai/prediction/gpt-2:latest",
    serving_container_predict_route="/predict",
    serving_container_health_route="/health"
)

model.wait()  # Block until upload completes

The artifact_uri must point to a Google Cloud Storage bucket containing your saved model files. The serving_container_image_uri specifies the prediction server that will load your model.

4. Create a Vertex AI Endpoint

An endpoint provides a stable HTTP URL for online predictions. Create one using Endpoint.create(), as shown at line 175 in the repository.

ENDPOINT_DISPLAY_NAME = "my-model-endpoint"

# Create endpoint

endpoint = aiplatform.Endpoint.create(display_name=ENDPOINT_DISPLAY_NAME)
print(f"Endpoint created: {endpoint.resource_name}")

If you prefer to use an existing endpoint, you can retrieve it using aiplatform.Endpoint.list() similar to the model lookup pattern.

5. Deploy the Model to the Endpoint

Bind your uploaded model to the endpoint with the deploy() method. This step provisions compute resources and starts the model serving containers. The repository implements this at line 182.

DEPLOYED_MODEL_DISPLAY_NAME = f"{MODEL_DISPLAY_NAME}-v1"

# Deploy model to endpoint

deployed_model = model.deploy(
    endpoint=endpoint,
    deployed_model_display_name=DEPLOYED_MODEL_DISPLAY_NAME,
    machine_type="n1-standard-4",
    min_replica_count=1,
    max_replica_count=1,
    traffic_split={"0": 100}  # Route 100% traffic to this model

)

print(f"Model deployed to endpoint: {endpoint.resource_name}")

The machine_type parameter specifies the compute resources (e.g., n1-standard-4 for 4 vCPUs and 15 GB RAM). The traffic_split dictionary controls what percentage of requests route to this model version.

6. Verify the Deployment

After deployment, verify that the endpoint is serving predictions. The repository includes a verification print statement at line 188.

# Verify deployment

print(f"Deployed model ID: {deployed_model.id}")
print(f"Endpoint URI: {endpoint.gca_resource.deployed_models[0].service_account}")

# Test prediction (example)

# prediction = endpoint.predict(instances=[{"input": "test"}])

You can now call endpoint.predict() to send inference requests to your deployed model.

Complete Deployment Script Example

Here is the consolidated, runnable script adapted from gemini/evaluation/synthetic-data-evals/setup.py that implements the full deployment pipeline:

from google.cloud import aiplatform

# Configuration

PROJECT_ID = "my-gcp-project"
REGION = "us-central1"
MODEL_DISPLAY_NAME = "my-custom-model"
MODEL_GCS_PATH = "gs://my-bucket/path/to/model/"
ENDPOINT_DISPLAY_NAME = "my-model-endpoint"

# 1. Initialize Vertex AI

aiplatform.init(project=PROJECT_ID, location=REGION)

# 2. Check for existing model

existing_models = aiplatform.Model.list(filter=f"display_name={MODEL_DISPLAY_NAME}")
if existing_models:
    model = existing_models[0]
else:
    # 3. Upload new model

    model = aiplatform.Model.upload(
        display_name=MODEL_DISPLAY_NAME,
        artifact_uri=MODEL_GCS_PATH,
        serving_container_image_uri="us-docker.pkg.dev/vertex-ai/prediction/gpt-2:latest",
    )
    model.wait()

# 4. Create endpoint

endpoints = aiplatform.Endpoint.list(filter=f"display_name={ENDPOINT_DISPLAY_NAME}")
if endpoints:
    endpoint = endpoints[0]
else:
    endpoint = aiplatform.Endpoint.create(display_name=ENDPOINT_DISPLAY_NAME)

# 5. Deploy model to endpoint

deployed_model = model.deploy(
    endpoint=endpoint,
    deployed_model_display_name=f"{MODEL_DISPLAY_NAME}-v1",
    machine_type="n1-standard-4",
    min_replica_count=1,
    max_replica_count=1,
    traffic_split={"0": 100},
)

print(f"✅ Model deployed successfully to: {endpoint.resource_name}")

Key Repository Files for Vertex AI Deployment

The GoogleCloudPlatform/generative-ai repository contains several relevant files for understanding Vertex AI deployment patterns:

• gemini/evaluation/synthetic-data-evals/setup.py – The primary reference implementation demonstrating model upload, endpoint creation, and deployment with traffic splitting.
• setup-env/README.md – Instructions for configuring Vertex AI Workbench environments, useful for interactive deployment testing.
• tools/llmevalkit/README.md – Documentation for Vertex AI Prompt Optimization, showing advanced serving patterns.
• search/web-app/README.md – Examples of enabling Vertex AI Search API, demonstrating complementary serving infrastructure.

Summary

• Initialize the Vertex AI SDK with aiplatform.init() to configure your project and region.
• Upload model artifacts using aiplatform.Model.upload(), specifying the Cloud Storage path and serving container image.
• Create an endpoint with aiplatform.Endpoint.create() to provide a stable prediction URL.
• Deploy using model.deploy() to bind the model to the endpoint with specified machine types and traffic allocation.
• Verify deployment by checking the endpoint resource name and testing predictions.

Frequently Asked Questions

What permissions are required to deploy a model on Vertex AI?

You need the Vertex AI User IAM role or equivalent custom permissions including aiplatform.models.upload, aiplatform.endpoints.create, and aiplatform.models.deploy. Additionally, you need storage.objects.get access to the Cloud Storage bucket containing your model artifacts.

Can I deploy multiple models to the same endpoint?

Yes. You can deploy multiple models to a single endpoint and control traffic splitting between them using the traffic_split parameter in the deploy() method. For example, set traffic_split={"0": 70, "1": 30} to route 70% of requests to the first model and 30% to the second.

What serving container images should I use for custom models?

Vertex AI provides pre-built container images for common frameworks like TensorFlow, PyTorch, scikit-learn, and XGBoost. For custom prediction servers, you can use any container image that implements the Vertex AI Predict protocol. The repository example uses us-docker.pkg.dev/vertex-ai/prediction/gpt-2:latest, but you should replace this with an image matching your model's framework and version.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}