# How to Fine-Tune Foundation Models Using the GenAI-SageMaker Directory

> Learn how to fine-tune foundation models with the genai-sagemaker directory a comprehensive toolchain for supervised fine-tuning on Amazon SageMaker and automated deployment.

- Repository: [Google Cloud Platform/generative-ai](https://github.com/GoogleCloudPlatform/generative-ai)
- Tags: how-to-guide
- Published: 2026-03-09

---

**The genai-sagemaker directory provides a complete toolchain for supervised fine-tuning of foundation models on Amazon SageMaker using Google's Gen AI SDK, including containerized training environments, YAML-based job configurations, and automated deployment utilities.**

The GoogleCloudPlatform/genai-sagemaker repository delivers production-ready infrastructure for fine-tuning large language models on AWS infrastructure. This tooling bridges Google’s Gen AI SDK with Amazon SageMaker's managed training capabilities, enabling you to customize models like Gemma or Llama using proprietary datasets while maintaining full control over hyperparameters and compute resources.

## Prerequisites and Installation

Before submitting training jobs, install the helper package and configure your AWS environment.

Install the package from the repository root:

```bash
pip install -e .

```

Ensure your AWS credentials are configured and that you have an IAM Role with AmazonSageMakerFullAccess permissions. The `roleArn` specified in your training configuration must have permissions to read from S3 and write training artifacts.

## Preparing Training Data for Fine-Tuning

The GenAI-SageMaker workflow requires JSON Lines (`.jsonl`) files where each line contains a `prompt` and the expected `completion`. This schema aligns with the Gen AI Supervised-Tuning documentation.

Reference the example structure in `data/sample_dataset.jsonl`:

```jsonl
{"prompt": "Translate the following English text to French: 'Hello world'", "completion": "Bonjour le monde"}
{"prompt": "Summarize: The quick brown fox...", "completion": "A fox jumps over a lazy dog."}

```

Upload your dataset to Amazon S3 using the AWS CLI or the provided S3 helper utilities:

```bash
aws s3 cp my_training_data.jsonl s3://my-bucket/genai-finetune/input/

```

The [`src/utils/s3_helper.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/src/utils/s3_helper.py) module contains additional utility functions for programmatically uploading and downloading data to S3 buckets.

## Configuring the SageMaker Training Job

Define your training configuration in [`config/training_job.yaml`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/config/training_job.yaml). This file specifies the Docker image, compute resources, hyperparameters, and data locations required by SageMaker's CreateTrainingJob API.

Key configuration parameters include:

- **trainingJobName**: Unique identifier for the training job
- **roleArn**: IAM role ARN with SageMaker execution permissions
- **inputDataConfig**: S3 URI pointing to your training data
- **resourceConfig**: Instance type (e.g., `ml.p3.2xlarge`) and count
- **hyperParameters**: Fine-tuning variables including `learning_rate`, `epochs`, and `batch_size`

Example configuration snippet:

```yaml
trainingJobName: genai-gemma-finetune-001
roleArn: arn:aws:iam::123456789012:role/SageMakerExecutionRole
inputDataConfig:
  - channelName: train
    dataSource:
      s3DataSource:
        s3Uri: s3://my-bucket/genai-finetune/input/
        s3DataType: S3Prefix
outputDataConfig:
  s3OutputPath: s3://my-bucket/genai-finetune/output/
resourceConfig:
  instanceType: ml.p3.2xlarge
  instanceCount: 1
  volumeSizeInGB: 50
hyperParameters:
  learning_rate: "5e-5"
  epochs: "3"
  batch_size: "8"

```

The Docker image referenced in the configuration is built from `docker/Dockerfile`, which bundles the Gen AI SDK, TRL/PEFT libraries, and the training execution script into the Google-GenAI-SageMaker container.

## Launching the Fine-Tuning Job

Use [`src/run_finetune.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/src/run_finetune.py) to submit the training job to SageMaker. This script acts as a CLI wrapper that translates your YAML configuration into a SageMaker CreateTrainingJob API call.

Launch the job using the CLI entry point:

```bash
genai-sagemaker run-finetune --config config/training_job.yaml

```

Alternatively, import the Python module directly for programmatic execution:

```python
from src.run_finetune import main
main()

```

The training container automatically pulls the specified foundation model, applies your dataset for supervised fine-tuning, and pushes the resulting checkpoint to the S3 output location defined in `outputDataConfig`.

## Monitoring Training Progress

Track job status through the SageMaker console, CloudWatch logs, or the command-line interface.

Check job status using the CLI:

```bash
genai-sagemaker describe-job --name genai-gemma-finetune-001

```

The helper scripts in `src/utils/` provide additional functionality for streaming logs and polling job completion. Monitor metrics such as loss convergence and training throughput via CloudWatch Logs to detect convergence issues or resource constraints during execution.

## Deploying the Fine-Tuned Model

Once training completes, the tuned model artifacts reside in the S3 output path specified in your configuration. Use [`src/deploy_endpoint.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/src/deploy_endpoint.py) to create a SageMaker endpoint for real-time inference.

Deploy the model using the CLI:

```bash
genai-sagemaker deploy-endpoint \
  --model-artifact s3://my-bucket/genai-finetune/output/model.tar.gz \
  --endpoint-name gemma-finetuned-endpoint \
  --instance-type ml.m5.large

```

This script creates a SageMaker Model object and deploys it to an HTTPS endpoint, making your fine-tuned model available for inference requests through AWS SDKs or boto3.

## Summary

- **Data Format**: Prepare JSON Lines files with `prompt` and `completion` fields, referencing `data/sample_dataset.jsonl` for schema guidance.

- **Configuration**: Define training parameters in [`config/training_job.yaml`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/config/training_job.yaml), specifying instance types, hyperparameters, and S3 locations for input/output data.

- **Execution**: Submit jobs via [`src/run_finetune.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/src/run_finetune.py) or the `genai-sagemaker run-finetune` CLI command, which invokes the SageMaker CreateTrainingJob API.

- **Containerization**: The training environment uses a custom Docker image from `docker/Dockerfile` containing the Gen AI SDK and PEFT libraries.

- **Deployment**: Create inference endpoints using [`src/deploy_endpoint.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/src/deploy_endpoint.py) after training artifacts are written to the S3 output path.

## Frequently Asked Questions

### What data format does genai-sagemaker require for fine-tuning?

The tooling requires JSON Lines (`.jsonl`) files where each line contains a JSON object with `prompt` and `completion` keys. This format matches the Gen AI Supervised-Tuning API schema, allowing you to provide input-output pairs for supervised fine-tuning of foundation models.

### How do I customize hyperparameters for my training job?

Edit the `hyperParameters` section in [`config/training_job.yaml`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/config/training_job.yaml) to adjust values such as `learning_rate`, `epochs`, and `batch_size`. These parameters are passed directly to the training container built from `docker/Dockerfile`, which uses TRL/PEFT libraries to configure the fine-tuning process.

### Where are the fine-tuned model artifacts stored?

Upon job completion, the training container pushes the model checkpoint to the S3 URI specified in `outputDataConfig.s3OutputPath` within your YAML configuration. You reference this S3 path when deploying the model using `genai-sagemaker deploy-endpoint` or [`src/deploy_endpoint.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/src/deploy_endpoint.py).

### Can I modify the training container for custom dependencies?

Yes. The `docker/Dockerfile` defines the Google-GenAI-SageMaker container environment. You can extend this Dockerfile to include additional Python packages or custom training scripts, then reference your rebuilt image in the training configuration's Docker image field before submitting jobs via [`src/run_finetune.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/src/run_finetune.py).