# What Is the genai-sagemaker Directory in Google Cloud's Generative AI Repository?

> Discover the genai-sagemaker directory in Google Cloud's generative-ai repo. Learn how to deploy Google Generative AI models on AWS SageMaker using Docker containers and CloudFormation.

- Repository: [Google Cloud Platform/generative-ai](https://github.com/GoogleCloudPlatform/generative-ai)
- Tags: deep-dive
- Published: 2026-03-09

---

**The genai-sagemaker directory provides a complete reference implementation for deploying Google Generative AI models on AWS SageMaker, bridging Google Cloud AI capabilities with AWS infrastructure through Docker containers, CloudFormation templates, and automated CI/CD pipelines.**

The `genai-sagemaker` folder within the `GoogleCloudPlatform/generative-ai` repository enables teams operating primarily on AWS to leverage Google's GenAI models—such as Gemini—without managing GCP-only infrastructure. This reference architecture packages the Google GenAI SDK into SageMaker-compatible container images, automates deployment via Infrastructure as Code, and provides sample clients for inference.

## Core Purpose and Architecture

The directory serves as a **hybrid-cloud bridge**, allowing organizations to maintain their existing AWS-centric machine learning pipelines while accessing Google's latest foundation models. According to the source code, the implementation focuses on four primary objectives.

### End-to-End Deployment Pipeline

The reference implementation provides everything needed to package a GenAI model and launch it as a SageMaker endpoint. The `Dockerfile` in the repository root defines the container environment that pre-installs the `google-genai` client libraries and their dependencies. The accompanying `Makefile` automates the build and push process to Amazon ECR.

```bash

# From the genai-sagemaker directory

make build          # builds Docker image with GenAI SDK installed

make push           # pushes image to your Amazon ECR repo

```

The `infrastructure/` directory contains CloudFormation templates—specifically [`infrastructure/sagemaker-endpoint.yml`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/infrastructure/sagemaker-endpoint.yml)—that provision the SageMaker model entity, endpoint configuration, and necessary IAM roles.

### Bridging Google GenAI SDK with SageMaker

The implementation handles the critical integration layer between AWS and Google Cloud authentication. Example code in [`scripts/invoke_endpoint.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/scripts/invoke_endpoint.py) demonstrates how to configure the `google.generativeai` library inside the SageMaker environment, managing service-account keys and token-refresh logic securely.

```python
import boto3
import json
import google.generativeai as genai

# Load GCP credentials (service-account JSON) from a secret manager / env var

genai.configure(api_key=os.getenv("GOOGLE_API_KEY"))

client = boto3.client('sagemaker-runtime')
payload = {"prompt": "Write a haiku about clouds"}

resp = client.invoke_endpoint(
    EndpointName="genai-sagemaker-endpoint",
    ContentType="application/json",
    Body=json.dumps(payload)
)

answer = json.loads(resp["Body"].read())
print(answer["completion"])

```

## Key Components and File Structure

The directory organizes assets into logical groups that support the full ML operations lifecycle:

- **[`README.md`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/README.md)** – High-level overview, prerequisites, and step-by-step deployment guide
- **`Dockerfile`** – Container definition installing the Google GenAI SDK and entrypoint scripts
- **`Makefile`** – Convenience targets for container lifecycle management (build, push, clean)
- **`infrastructure/`** – CloudFormation/SAM templates for SageMaker model, endpoint configuration, and IAM roles
- **`scripts/`** – Helper utilities including [`build_image.sh`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/build_image.sh), [`deploy_endpoint.sh`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/deploy_endpoint.sh), and [`invoke_endpoint.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/invoke_endpoint.py)
- **`notebooks/`** – Jupyter notebooks demonstrating end-to-end data preparation, model invocation, and result visualization
- **`gateway/`** – Optional FastAPI application ([`gateway/app.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gateway/app.py)) that exposes the SageMaker endpoint as a REST API
- **[`.github/workflows/deploy.yml`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/.github/workflows/deploy.yml)** – GitHub Actions workflow automating container builds and endpoint deployment on repository pushes

## Deployment Workflow Examples

### Infrastructure as Code Deployment

Deploy the SageMaker endpoint using the provided CloudFormation template after pushing your container to ECR:

```bash
aws cloudformation deploy \
  --template-file infrastructure/sagemaker-endpoint.yml \
  --stack-name genai-sagemaker-stack \
  --parameter-overrides \
    ImageUri=<your-ecr-repo>:latest \
    ModelName=genai-model \
    InstanceType=ml.m5.large

```

### FastAPI Gateway Integration

For applications requiring a RESTful interface, the [`gateway/app.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gateway/app.py) file implements a FastAPI wrapper around the SageMaker runtime client:

```python
from fastapi import FastAPI, Request
import boto3, json, os

app = FastAPI()
sm_runtime = boto3.client("sagemaker-runtime")
ENDPOINT = os.getenv("SM_ENDPOINT_NAME")

@app.post("/generate")
async def generate(req: Request):
    body = await req.json()
    resp = sm_runtime.invoke_endpoint(
        EndpointName=ENDPOINT,
        ContentType="application/json",
        Body=json.dumps(body)
    )
    return json.loads(resp["Body"].read())

```

## Summary

- The **genai-sagemaker** directory enables deployment of Google GenAI models on AWS SageMaker for hybrid-cloud scenarios.
- It provides a complete **CI/CD pipeline** via GitHub Actions, Docker, and CloudFormation templates found in `infrastructure/`.
- Authentication between AWS and Google Cloud is handled through environment variables and service account keys configured in the container.
- The reference includes **production-ready components**: a FastAPI gateway, Python inference clients, and automated build scripts.
- All source files are located at `https://github.com/GoogleCloudPlatform/generative-ai/tree/main/genai-sagemaker`.

## Frequently Asked Questions

### What models can I deploy using genai-sagemaker?

The directory supports any model accessible through the Google GenAI SDK, including Gemini Pro, Gemini Pro Vision, and PaLM 2. The container architecture is model-agnostic; you specify the model variant through environment variables in the SageMaker endpoint configuration.

### How does authentication work between AWS SageMaker and Google GenAI?

Authentication relies on GCP service account keys injected as environment variables or secrets. The `Dockerfile` installs the Google GenAI SDK, and the deployment scripts configure `genai.configure(api_key=...)` using credentials stored in AWS Secrets Manager or passed during endpoint creation. Token refresh logic is handled automatically by the SDK.

### Can I use genai-sagemaker in production environments?

Yes. The repository includes production-oriented features: a FastAPI gateway in [`gateway/app.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gateway/app.py) for load balancing, CloudFormation templates for consistent infrastructure provisioning, and a GitHub Actions workflow in [`.github/workflows/deploy.yml`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/.github/workflows/deploy.yml) for automated, reproducible deployments. You should implement additional monitoring and adjust the auto-scaling policies in the CloudFormation templates for your specific workload.

### What are the cost implications of running Google GenAI models on SageMaker?

Costs include AWS SageMaker hosting charges (compute instance hours) plus Google Cloud API usage fees for the GenAI models. The reference implementation uses standard SageMaker inference instances (e.g., `ml.m5.large`), allowing you to leverage Reserved Instances or Savings Plans for predictable workloads, potentially reducing compute costs compared to on-demand pricing.