# genai-sagemaker vs Other Model Deployment Examples: AWS and GCP Compared

> Compare genai-sagemaker AWS deployment to Google Cloud examples like Vertex AI. Understand key differences in model deployment tools and services for each cloud platform.

- Repository: [Google Cloud Platform/generative-ai](https://github.com/GoogleCloudPlatform/generative-ai)
- Tags: comparison
- Published: 2026-03-09

---

**genai-sagemaker is the AWS-centric deployment path that uses the SageMaker SDK and IAM roles, while other examples in the repository target Google Cloud Platform services such as Vertex AI, Cloud Run, and GKE using GCP-specific tooling.**

The `GoogleCloudPlatform/generative-ai` repository provides multi-cloud reference implementations for deploying generative AI models. While the majority of examples demonstrate native Google Cloud integrations, the `genai-sagemaker` directory offers equivalent workflows for Amazon Web Services. Understanding the architectural differences between genai-sagemaker and other model deployment examples enables teams to select the appropriate infrastructure for hybrid cloud environments or AWS-specific production requirements.

## Target Platforms and SDKs

The fundamental distinction lies in the cloud provider ecosystem and the corresponding SDKs used for deployment.

### AWS SageMaker vs Google Cloud Services

**genai-sagemaker** targets **Amazon Web Services (AWS)** exclusively. All scripts, Dockerfiles, and CloudFormation templates in [`genai-sagemaker/README.md`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/genai-sagemaker/README.md) and supporting files are built around the SageMaker SDK, SageMaker-specific IAM roles, and SageMaker endpoint APIs.

Other deployment examples target **Google Cloud Platform (GCP)** services. These include Vertex AI for managed model serving, Cloud Run for serverless containers, GKE for Kubernetes-orchestrated inference, and Cloud Functions for event-driven workloads. Each uses GCP-native resources such as Cloud Storage buckets and Vertex AI endpoints.

### Authentication Mechanisms

AWS deployments require **AWS credentials** (`aws_access_key_id`, `aws_secret_access_key`) and SageMaker execution IAM roles. The `sagemaker` Python library automatically signs requests using these credentials.

GCP examples rely on **Application Default Credentials (ADC)** or service account keys through the `google-auth` library. The `google-cloud-aiplatform` SDK handles authentication to Vertex AI and other GCP services.

## Deployment Architecture Differences

The workflow patterns for provisioning and serving models differ significantly between the two platforms.

### SageMaker Endpoint Creation

In [`genai-sagemaker/deploy_hf_sagemaker.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/genai-sagemaker/deploy_hf_sagemaker.py), deployment follows a four-step SageMaker workflow:

1. **Create a SageMaker Model** using `sagemaker.Model` or `HuggingFaceModel` with specific container versions
2. **Define an EndpointConfig** specifying instance types like `ml.g5.2xlarge` and autoscaling policies
3. **Deploy the Endpoint** via `model.deploy()` to create a real-time inference endpoint
4. **Invoke via boto3** or the SageMaker SDK using AWS-specific API signatures

SageMaker supports both real-time endpoints and batch transform jobs for large-scale offline inference, integrating with AWS data pipelines such as S3, Glue, and Athena.

### GCP Deployment Patterns

According to [`vertex_ai/quickstart_deploy.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/vertex_ai/quickstart_deploy.py) and related documentation, GCP deployment typically involves:

1. **Uploading models to Vertex AI** Model Garden or Cloud Storage buckets
2. **Creating a Vertex AI Model resource** using `aiplatform.Model`
3. **Deploying to a Vertex endpoint** with `model.deploy()` on machine types like `n1-standard-4`
4. **Invoking via REST** or `aiplatform.PredictionServiceClient`

Alternatively, containers can be deployed to **Cloud Run** for serverless scaling or **GKE** with KServe for multi-model orchestration, as documented in [`cloud-run/README.md`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/cloud-run/README.md) and [`gke/kserve/README.md`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gke/kserve/README.md).

## Code Implementation Comparison

The implementation details reveal platform-specific conventions for model packaging and serving.

### SageMaker SDK Deployment

The [`genai-sagemaker/deploy_hf_sagemaker.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/genai-sagemaker/deploy_hf_sagemaker.py) script demonstrates AWS deployment using the Hugging Face integration:

```python
import sagemaker
from sagemaker.huggingface import HuggingFaceModel

# Define IAM role for SageMaker execution

role = "arn:aws:iam::123456789012:role/SageMakerExecutionRole"

# Configure SageMaker-compatible container

hf_model = HuggingFaceModel(
    transformers_version="4.31.0",
    pytorch_version="2.0.0",
    py_version="py310",
    model_data="s3://my-bucket/model.tar.gz",
    env={"HF_MODEL_ID": "google/flan-t5-xl"},
    role=role,
)

# Deploy to GPU instance

predictor = hf_model.deploy(
    initial_instance_count=1,
    instance_type="ml.g5.2xlarge",
    endpoint_name="flan-t5-xl-endpoint"
)

# Invoke endpoint

response = predictor.predict({
    "inputs": "Translate English to French: Hello, how are you?",
    "parameters": {"max_new_tokens": 64}
})

```

### Vertex AI SDK Deployment

In contrast, [`vertex_ai/quickstart_deploy.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/vertex_ai/quickstart_deploy.py) uses the Google Cloud SDK:

```python
from vertexai.preview import language_models
import vertexai

# Initialize Vertex AI

vertexai.init(project="my-gcp-project", location="us-central1")

# Load model from Model Garden

model = language_models.TextGenerationModel.from_pretrained("text-bison@001")

# Deploy to endpoint

endpoint = model.deploy(
    machine_type="n1-standard-4",
    min_replica_count=1,
    max_replica_count=2
)

# Generate prediction

response = model.predict(
    "Translate English to French: Hello, how are you?",
    temperature=0.2,
    max_output_tokens=64
)

```

### SageMaker Inference Handler

The [`genai-sagemaker/inference.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/genai-sagemaker/inference.py) file implements the required SageMaker inference contract with four specific functions:

```python
import json
import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_id = "google/flan-t5-xl"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id).to("cuda")

def model_fn(model_dir):
    """SageMaker calls this to load the model."""
    return model

def input_fn(request_body, request_content_type):
    """Parse incoming request."""
    if request_content_type == "application/json":
        data = json.loads(request_body)
        return data["inputs"]
    raise ValueError(f"Unsupported content type: {request_content_type}")

def predict_fn(inputs, model):
    """Run inference."""
    inputs = tokenizer(inputs, return_tensors="pt").to("cuda")
    outputs = model.generate(**inputs, max_new_tokens=64)
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

def output_fn(prediction, response_content_type):
    """Format response."""
    return json.dumps({"generated_text": prediction})

```

GCP containers typically use standard HTTP servers with `/predict` endpoints or TGI (Text Generation Inference) containers without requiring these specific handler functions.

## Container and Runtime Specifications

**genai-sagemaker** requires **SageMaker-compatible Docker images** that expose port 8080 and implement the SageMaker inference toolkit contract. The [`genai-sagemaker/requirements.txt`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/genai-sagemaker/requirements.txt) specifies dependencies including `sagemaker`, `transformers`, and `torch`.

Other examples utilize **Vertex AI-compatible containers** such as `gcr.io/cloud-aiplatform/prediction/tgi` or generic Cloud Run containers. These follow standard HTTP serving conventions rather than SageMaker-specific interfaces.

## Summary

- **genai-sagemaker** targets **AWS SageMaker** using the `sagemaker` Python SDK, IAM roles, and SageMaker endpoints, while other examples target **GCP Vertex AI**, **Cloud Run**, **GKE**, and **Cloud Functions** using `google-cloud-aiplatform`.
- **Authentication** differs between AWS credentials and IAM roles versus GCP Application Default Credentials and service accounts.
- **Deployment workflows** in [`genai-sagemaker/deploy_hf_sagemaker.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/genai-sagemaker/deploy_hf_sagemaker.py) use `HuggingFaceModel` and `model.deploy()` to create SageMaker endpoints on instances like `ml.g5.2xlarge`, whereas GCP examples deploy to Vertex AI endpoints or serverless containers.
- **Inference handlers** in SageMaker require specific functions (`model_fn`, `input_fn`, `predict_fn`, `output_fn`) implemented in [`genai-sagemaker/inference.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/genai-sagemaker/inference.py), while GCP examples use standard HTTP endpoints or KServe mediators.
- **Use genai-sagemaker** when integrating with AWS data pipelines or existing AWS infrastructure; use GCP examples for native Vertex AI features, Gemini integration, or Google Kubernetes Engine workloads.

## Frequently Asked Questions

### What is the primary difference between genai-sagemaker and Vertex AI deployment examples?

**genai-sagemaker** focuses exclusively on **AWS SageMaker** infrastructure, utilizing the SageMaker Python SDK, AWS IAM roles, and SageMaker-specific container formats to deploy models on instances like `ml.g5.2xlarge`. Vertex AI examples use the `google-cloud-aiplatform` library and target GCP-managed endpoints with different authentication and scaling mechanisms.

### Can I deploy the same model using both genai-sagemaker and Cloud Run?

Yes, the underlying model artifacts (such as Hugging Face checkpoints) are portable, but the **containerization and serving contracts differ**. genai-sagemaker requires a container with a `serve` script and specific inference handlers as shown in [`genai-sagemaker/inference.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/genai-sagemaker/inference.py), while Cloud Run accepts standard HTTP servers or prebuilt TGI containers without SageMaker-specific interfaces.

### Which SDK should I use for batch inference: genai-sagemaker or GCP examples?

For **AWS batch inference**, use **genai-sagemaker** with `sagemaker.transformer.Transformer` to create batch transform jobs that read from S3. For GCP batch workloads, use Vertex AI Batch Prediction or process jobs with Cloud Functions, depending on the specific example in the repository.

### Does genai-sagemaker support the same model formats as Vertex AI examples?

Both support popular frameworks like PyTorch and TensorFlow, but **genai-sagemaker** specifically integrates with **Hugging Face Deep Learning Containers** and expects model artifacts in SageMaker-compatible tar.gz formats stored in S3. Vertex AI examples can load models directly from Model Garden or Cloud Storage buckets using `aiplatform.Model` without requiring SageMaker-specific packaging.