How to Run genai-sagemaker Examples on Amazon SageMaker

The genai-sagemaker directory does not exist in the GoogleCloudPlatform/generative-ai repository, but you can adapt the existing Vertex AI and open-models examples to run on Amazon SageMaker using the sagemaker Python SDK and Hugging Face inference containers.

The GoogleCloudPlatform/generative-ai repository is the authoritative source for generative AI implementations on Google Cloud, containing end-to-end notebooks for Vertex AI and Gemini. While developers searching for native genai-sagemaker examples will not find a dedicated folder in the current main branch, the architectural patterns in the open-models and gemini directories provide a complete blueprint for deploying these same workloads on Amazon SageMaker with minimal modifications.

Why the Repository Lacks a genai-sagemaker Directory

The repository's primary focus is Google Cloud's Generative AI services. According to the source code structure, SageMaker-specific notebooks are not part of the public main tree, which is why a genai-sagemaker directory cannot be found through repository search.

However, the open-models notebooks demonstrate framework-agnostic patterns for loading models from the Hugging Face hub and deploying them to managed endpoints. These patterns translate directly to SageMaker's deployment workflow.

Adapting Vertex AI Patterns for SageMaker Deployment

You can recreate the SageMaker workflow by replacing Google Cloud SDK calls with AWS equivalents while preserving the model loading and inference logic.

Step 1: Configure Dependencies and Environment

Start by installing the required Python SDKs in your SageMaker notebook instance. While the repository does not ship a SageMaker-specific requirements.txt, you can adapt the generic dependencies found in search/web-app/requirements.txt and add AWS-specific packages:

• sagemaker
• boto3
• transformers

Step 2: Select a Pretrained Model

Choose a model from the Hugging Face hub, following the pattern in open-models/use-cases/vertex_ai_deepseek_smolagents.ipynb. This notebook illustrates how to load models like google/gemma-2b using the transformers library, which works identically on SageMaker.

Step 3: Create the SageMaker Model Object

Use the sagemaker Python SDK to define a Model object that points to a Docker container capable of running your chosen framework. This parallels the Vertex AI model creation logic in open-models/serving/vertex_ai_tgi_gemma_multi_lora_adapters_deployment.ipynb.

For Hugging Face models, use the official huggingface-pytorch-inference container from the AWS ECR registry:

container_image = (
    "763104351884.dkr.ecr.us-west-2.amazonaws.com"
    "/huggingface-pytorch-inference:1.13.1-transformers4.30.2-gpu-py39-cu118-ubuntu20.04"
)

Step 4: Deploy the Endpoint

Call model.deploy() with your desired instance type, mirroring the deployment sections in the open-models serving notebooks. This provisions the inference infrastructure and exposes a runtime endpoint.

Step 5: Invoke the Endpoint

Use the SageMaker runtime client or the Predictor object to send JSON payloads containing your prompts. The request-response handling follows the same structure as the Gemini Chat API demonstrated in gemini/getting-started/intro_gemini_chat.ipynb, using inputs and parameters keys.

Step 6: Optional REST API Wrapper

The repository includes a minimal Flask application in search/web-app/main.py that wraps Vertex AI predictions. You can reuse this structure to expose your SageMaker endpoint as a REST API by replacing the Vertex AI client initialization with boto3.client('runtime.sagemaker') calls.

Step 7: Clean Up Resources

Delete the endpoint when finished to avoid ongoing charges using sagemaker.Session().delete_endpoint(endpoint_name). This cleanup pattern appears implicitly in every deployment notebook in the repository.

Complete SageMaker Deployment Script

Below is a minimal, self-contained script that implements the full workflow on SageMaker, adapted from the patterns in open-models/serving/vertex_ai_tgi_gemma_multi_lora_adapters_deployment.ipynb and gemini/getting-started/intro_gemini_chat.ipynb.

# Install dependencies (run once in your SageMaker notebook)

# !pip install -q sagemaker boto3 transformers

import json
import boto3
import sagemaker
from sagemaker import Model
from sagemaker.session import Session
from sagemaker.predictor import Predictor

# -------------------------------------------------

# 1️⃣  Choose a pre-trained model from Hugging Face

# -------------------------------------------------

model_id = "google/gemma-2b"  # replace with any HF model id

# -------------------------------------------------

# 2️⃣  Define the SageMaker container (Hugging Face)

# -------------------------------------------------

container_image = (
    "763104351884.dkr.ecr.us-west-2.amazonaws.com"
    "/huggingface-pytorch-inference:1.13.1-transformers4.30.2-gpu-py39-cu118-ubuntu20.04"
)

# -------------------------------------------------

# 3️⃣  Create the SageMaker Model object

# -------------------------------------------------

role = sagemaker.get_execution_role()  # SageMaker notebook role

sess = Session()

hf_model = Model(
    image_uri=container_image,
    model_data=None,  # No model tarball – HF container pulls from hub at runtime

    role=role,
    sagemaker_session=sess,
    env={"HF_MODEL_ID": model_id, "HF_TASK": "text-generation"},
)

# -------------------------------------------------

# 4️⃣  Deploy an endpoint

# -------------------------------------------------

predictor: Predictor = hf_model.deploy(
    initial_instance_count=1,
    instance_type="ml.m5.large",
    endpoint_name="gemma-2b-endpoint",
)

# -------------------------------------------------

# 5️⃣  Invoke the endpoint

# -------------------------------------------------

payload = {
    "inputs": "Write a short poem about clouds.",
    "parameters": {"max_new_tokens": 64, "temperature": 0.7},
}
response = predictor.predict(json.dumps(payload))
generated = json.loads(response.decode("utf-8"))
print(generated[0]["generated_text"])

# -------------------------------------------------

# 6️⃣  Clean up (delete endpoint when done)

# -------------------------------------------------

# predictor.delete_endpoint()

Key Files to Reference for SageMaker Migration

When building SageMaker examples, consult these specific files from the GoogleCloudPlatform/generative-ai repository:

• search/web-app/requirements.txt – Lists core Python packages; add sagemaker and boto3 to adapt for AWS.

• open-models/use-cases/vertex_ai_deepseek_smolagents.ipynb – Demonstrates loading models from the Hugging Face hub and preparing them for managed endpoints.

• open-models/serving/vertex_ai_tgi_gemma_multi_lora_adapters_deployment.ipynb – Contains the detailed deployment workflow (model object creation, endpoint configuration, and invocation) that maps to SageMaker patterns.

• gemini/getting-started/intro_gemini_chat.ipynb – Shows the JSON payload structure for LLM chat requests, compatible with Hugging Face inference containers.

• search/web-app/main.py – Provides a Flask wrapper structure useful for exposing SageMaker endpoints via REST API.

Summary

• The genai-sagemaker directory is not present in the repository's main branch.
• Adaptation requires replacing Vertex AI SDK calls with SageMaker SDK (sagemaker and boto3).
• Use Hugging Face inference containers for deploying open models from the Hub.
• Request/response JSON structures from the Gemini examples transfer directly to SageMaker endpoints.
• Always delete endpoints after use to avoid AWS infrastructure charges.

Frequently Asked Questions

Does the GoogleCloudPlatform/generative-ai repository contain a genai-sagemaker directory?

No. The repository focuses exclusively on Google Cloud services, and the genai-sagemaker folder does not exist in the public main branch. You must manually adapt the Vertex AI and open-models examples for Amazon SageMaker.

Which repository file shows the best pattern for SageMaker endpoint deployment?

The open-models/serving/vertex_ai_tgi_gemma_multi_lora_adapters_deployment.ipynb notebook provides the most relevant blueprint. It demonstrates model object creation, endpoint deployment, and invocation patterns that translate directly to SageMaker's Model.deploy() workflow.

Can I use the Gemini chat examples on Amazon SageMaker?

Yes. While you cannot use the Gemini API itself on SageMaker, the request formatting logic in gemini/getting-started/intro_gemini_chat.ipynb (using structured JSON with inputs and parameters keys) is compatible with Hugging Face inference containers deployed on SageMaker, requiring minimal modification to your application code.

How do I handle authentication when adapting these examples for AWS?

Replace the Google Cloud authentication (google.auth) with standard AWS credential chains. In a SageMaker notebook environment, use sagemaker.get_execution_role() to obtain the IAM role, as shown in the deployment script above. For local development, configure the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables or use the boto3 credential provider chain.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}