# Key Components in the generative-ai-gcp Directory for Building AI Solutions on Google Cloud

> Explore the generative-ai-gcp directory to discover Vertex AI SDK helpers, search infra, model serving, and fine-tuning for building AI solutions on Google Cloud.

- Repository: [Google Cloud Platform/generative-ai](https://github.com/GoogleCloudPlatform/generative-ai)
- Tags: architecture
- Published: 2026-03-09

---

**The generative-ai-gcp directory provides a comprehensive toolkit of Vertex AI SDK helpers, search infrastructure, model serving pipelines, and fine-tuning workflows for building production-grade generative AI applications on Google Cloud.**

The GoogleCloudPlatform/generative-ai repository hosts a curated collection of reference implementations for generative AI on GCP. Within the generative-ai-gcp directory structure, developers will find modular components ranging from low-level SDK wrappers to complete web applications, enabling end-to-end AI solution development.

## Vertex AI SDK and Evaluation Components

### LLM Prompting Interface

The [`tools/llmevalkit/src/gcp_prompt.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/tools/llmevalkit/src/gcp_prompt.py) file provides a Python wrapper around the Vertex AI SDK that simplifies interaction with Gemini, PaLM, and custom models. This helper class handles model initialization, token limit management, and streaming response processing.

```python
from tools.llmevalkit.src.gcp_prompt import GcpPrompt

# Initialise the helper – it reads the GOOGLE_APPLICATION_CREDENTIALS env var

prompt = GcpPrompt(
    project_id="my-gcp-project",
    location="us-central1",
    model_name="gemini-1.5-pro"
)

response = prompt.run(
    system="You are a helpful assistant.",
    user="Explain Retrieval‑Augmented Generation in two sentences."
)

print(response.text)   # → “RAG combines …”

```

### Dataset and Evaluation Helpers

Complementing the prompting interface, [`tools/llmevalkit/src/gcp_dataset.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/tools/llmevalkit/src/gcp_dataset.py) and [`tools/llmevalkit/src/gcp_evaluation.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/tools/llmevalkit/src/gcp_evaluation.py) provide utilities for constructing evaluation datasets, submitting batch inference jobs to Vertex AI, and parsing evaluation results. These modules form the backbone of the LLM Eval Kit, enabling systematic model benchmarking on GCP.

## Search and Retrieval Infrastructure

### Vertex AI Search Web Application

The [`search/web-app/main.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/search/web-app/main.py) file implements a minimal Flask-based UI that demonstrates searchable document retrieval using Vertex AI Search. This component showcases relevance boosting, hybrid search capabilities, and metadata filtering through a practical web interface.

```python
from flask import Flask, request, jsonify
import vertexai.search as search

app = Flask(__name__)

@app.route("/search")
def search_docs():
    query = request.args.get("q")
    # Uses the default search engine configured in `consts.py`

    results = search.search(
        query=query,
        max_documents=5,
        query_expansion_spec=search.QueryExpansionSpec(
            boost_spec=search.BoostSpec(condition_boosts=[
                search.ConditionBoost(
                    condition="metadata.author == 'Lavi'",
                    boost=2.0
                )
            ])
        )
    )
    return jsonify([r.document.id for r in results])

```

The accompanying [`search/web-app/consts.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/search/web-app/consts.py) defines the searchable data schema, metadata fields, and boost rules, while the `Dockerfile` in the same directory enables containerized deployment to Cloud Run or GKE.

### Vector Search and Embeddings

For developers building custom retrieval systems, the `embeddings/vector-search-quickstart.ipynb` notebook provides an end-to-end guide for generating embeddings via Vertex AI and performing similarity search. This component demonstrates integration between Gemini embedding models and vector databases.

```python
from google.cloud import bigquery

client = bigquery.Client(project="my-gcp-project")

# Generate embeddings with Vertex AI (Gemini text‑embedding model)

embedding_sql = """
SELECT
  GENERATE_TEXT_EMBEDDING(
    "I love pizza",  -- document text
    "textembedding-gecko@001"
  ) AS vec
"""
vec = client.query(embedding_sql).to_dataframe().vec[0]

# Find the 5 nearest neighbours in a pre‑populated vector table

search_sql = f"""
SELECT id, cosine_distance(vec, @query_vec) AS dist
FROM `my_dataset.my_vectors`
ORDER BY dist LIMIT 5
"""
job = client.query(search_sql, job_config=bigquery.QueryJobConfig(
    query_parameters=[
        bigquery.ScalarQueryParameter("query_vec", "ARRAY<FLOAT64>", vec)
    ]))
print(job.result().to_dataframe())

```

## Model Serving and Deployment

### Model Garden and TGI Serving

The `open-models/serving/vertex_ai_tgi_gemma_multi_lora_adapters_deployment.ipynb` notebook demonstrates deploying open-source models like Llama 3 and Gemma on Vertex AI using the Text Generation Inference (TGI) container. This component supports LoRA (Low-Rank Adaptation) adapter deployment for efficient model customization.

```python
from google.cloud import aiplatform

# 1️⃣ Upload the model artifact (saved in Cloud Storage)

model = aiplatform.Model.upload(
    display_name="llama3-finetuned",
    artifact_uri="gs://my-bucket/llama3-finetuned/",
    serving_container_image_uri="us-docker.pkg.dev/vertex-ai/prediction/tgi:latest"
)

# 2️⃣ Deploy to an endpoint

endpoint = model.deploy(
    machine_type="n1-standard-8",
    accelerator_type="NVIDIA_TESLA_T4",
    accelerator_count=1,
    traffic_split={"0": 100}
)

# 3️⃣ Invoke the endpoint

response = endpoint.predict(instances=[{
    "prompt": "Summarize the plot of *The Great Gatsby* in 30 words."
}])
print(response)

```

### Containerized Deployment Patterns

For production deployment, the repository includes Dockerfiles and Terraform snippets for containerizing LLM inference services. The `search/web-app/Dockerfile` provides a pattern for Flask applications, while [`partner-models/claude/computer-use-demo/deployment.yaml`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/partner-models/claude/computer-use-demo/deployment.yaml) demonstrates GKE deployment with service account binding via the `iam.gke.io/gcp-service-account` annotation.

## Fine-Tuning and Customization

The `open-models/fine-tuning/vertex_ai_trl_fine_tuning_gemma.ipynb` notebook provides a complete workflow for instruction-tuning large language models using TensorFlow Reinforcement Learning from Human Feedback (TRL) on Vertex AI. This component demonstrates how to customize open-source models like Gemma with domain-specific datasets while leveraging managed training infrastructure and custom container capabilities.

## Data Integration and Partner Models

### BigQuery Integration

The `workshops/rag-ops/2.2_mvp_chunk_embeddings.ipynb` notebook demonstrates tight integration between BigQuery and Vertex AI, including the IAM configuration required for service accounts to invoke generative models directly from SQL queries. Line 933 specifically highlights the necessary permissions for cross-service authentication, enabling vector operations and text generation within data warehouse workflows.

### Partner Model Support

The repository extends beyond Google-native models to include partner implementations. The [`partner-models/claude/computer-use-demo/deployment.yaml`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/partner-models/claude/computer-use-demo/deployment.yaml) file provides a complete GKE deployment specification for Anthropic's Claude, including Cloud Build configurations and service account wiring for enterprise deployments that require multi-cloud AI capabilities.

## Summary

- The **generative-ai-gcp directory** provides modular SDK wrappers in `tools/llmevalkit/src/` for prompting, dataset management, and evaluation on Vertex AI.
- **Search and retrieval** capabilities are demonstrated through the Flask-based Vertex AI Search web application ([`search/web-app/main.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/search/web-app/main.py)) and BigQuery vector search notebooks.
- **Model serving** options range from managed TGI containers for open-source models to containerized Cloud Run and GKE deployment patterns.
- **Fine-tuning workflows** leverage TRL on Vertex AI for customizing Gemma and other open models with domain-specific data.
- **Enterprise integrations** include BigQuery SQL-based generation and partner model deployments with proper IAM service account binding.

## Frequently Asked Questions

### What is the generative-ai-gcp directory in the GoogleCloudPlatform/generative-ai repository?

The generative-ai-gcp directory refers to the collection of Google Cloud Platform-specific resources within the repository, organized into folders like `tools/llmevalkit`, `search`, `embeddings`, and `open-models`. These components provide reference implementations for building, serving, and evaluating generative AI applications using Vertex AI, BigQuery, and containerized deployment platforms.

### How does the Vertex AI Search web application demonstrate enterprise search capabilities?

The [`search/web-app/main.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/search/web-app/main.py) file implements a Flask application that interfaces with Vertex AI Search to provide hybrid search functionality, relevance boosting, and metadata filtering. The accompanying [`consts.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/consts.py) defines searchable schemas and boost rules, while the Dockerfile enables containerized deployment to Cloud Run or GKE, demonstrating a complete production-ready search interface.

### What options are available for deploying open-source models using the generative-ai-gcp components?

The repository provides multiple deployment paths for open-source models including the Text Generation Inference (TGI) container on Vertex AI as shown in `open-models/serving/vertex_ai_tgi_gemma_multi_lora_adapters_deployment.ipynb`, which supports LoRA adapters. Additionally, containerized deployment patterns for Cloud Run and GKE are demonstrated in the search web app and partner model examples, offering flexibility from serverless to Kubernetes-based serving.

### How does the repository support fine-tuning workflows on Vertex AI?

The `open-models/fine-tuning/vertex_ai_trl_fine_tuning_gemma.ipynb` notebook provides a complete implementation for instruction-tuning models using TensorFlow Reinforcement Learning from Human Feedback (TRL). This workflow demonstrates how to customize open-source models like Gemma with domain-specific datasets while leveraging Vertex AI's managed training infrastructure and custom container capabilities.