Key Components in the generative-ai-gcp Directory for Building AI Solutions on Google Cloud

The generative-ai-gcp directory provides a comprehensive toolkit of Vertex AI SDK helpers, search infrastructure, model serving pipelines, and fine-tuning workflows for building production-grade generative AI applications on Google Cloud.

The GoogleCloudPlatform/generative-ai repository hosts a curated collection of reference implementations for generative AI on GCP. Within the generative-ai-gcp directory structure, developers will find modular components ranging from low-level SDK wrappers to complete web applications, enabling end-to-end AI solution development.

Vertex AI SDK and Evaluation Components

LLM Prompting Interface

The tools/llmevalkit/src/gcp_prompt.py file provides a Python wrapper around the Vertex AI SDK that simplifies interaction with Gemini, PaLM, and custom models. This helper class handles model initialization, token limit management, and streaming response processing.

from tools.llmevalkit.src.gcp_prompt import GcpPrompt

# Initialise the helper – it reads the GOOGLE_APPLICATION_CREDENTIALS env var

prompt = GcpPrompt(
    project_id="my-gcp-project",
    location="us-central1",
    model_name="gemini-1.5-pro"
)

response = prompt.run(
    system="You are a helpful assistant.",
    user="Explain Retrieval‑Augmented Generation in two sentences."
)

print(response.text)   # → “RAG combines …”

Dataset and Evaluation Helpers

Complementing the prompting interface, tools/llmevalkit/src/gcp_dataset.py and tools/llmevalkit/src/gcp_evaluation.py provide utilities for constructing evaluation datasets, submitting batch inference jobs to Vertex AI, and parsing evaluation results. These modules form the backbone of the LLM Eval Kit, enabling systematic model benchmarking on GCP.

Search and Retrieval Infrastructure

Vertex AI Search Web Application

The search/web-app/main.py file implements a minimal Flask-based UI that demonstrates searchable document retrieval using Vertex AI Search. This component showcases relevance boosting, hybrid search capabilities, and metadata filtering through a practical web interface.

from flask import Flask, request, jsonify
import vertexai.search as search

app = Flask(__name__)

@app.route("/search")
def search_docs():
    query = request.args.get("q")
    # Uses the default search engine configured in `consts.py`

    results = search.search(
        query=query,
        max_documents=5,
        query_expansion_spec=search.QueryExpansionSpec(
            boost_spec=search.BoostSpec(condition_boosts=[
                search.ConditionBoost(
                    condition="metadata.author == 'Lavi'",
                    boost=2.0
                )
            ])
        )
    )
    return jsonify([r.document.id for r in results])

The accompanying search/web-app/consts.py defines the searchable data schema, metadata fields, and boost rules, while the Dockerfile in the same directory enables containerized deployment to Cloud Run or GKE.

Vector Search and Embeddings

For developers building custom retrieval systems, the embeddings/vector-search-quickstart.ipynb notebook provides an end-to-end guide for generating embeddings via Vertex AI and performing similarity search. This component demonstrates integration between Gemini embedding models and vector databases.

from google.cloud import bigquery

client = bigquery.Client(project="my-gcp-project")

# Generate embeddings with Vertex AI (Gemini text‑embedding model)

embedding_sql = """
SELECT
  GENERATE_TEXT_EMBEDDING(
    "I love pizza",  -- document text
    "textembedding-gecko@001"
  ) AS vec
"""
vec = client.query(embedding_sql).to_dataframe().vec[0]

# Find the 5 nearest neighbours in a pre‑populated vector table

search_sql = f"""
SELECT id, cosine_distance(vec, @query_vec) AS dist
FROM `my_dataset.my_vectors`
ORDER BY dist LIMIT 5
"""
job = client.query(search_sql, job_config=bigquery.QueryJobConfig(
    query_parameters=[
        bigquery.ScalarQueryParameter("query_vec", "ARRAY<FLOAT64>", vec)
    ]))
print(job.result().to_dataframe())

Model Serving and Deployment

Model Garden and TGI Serving

The open-models/serving/vertex_ai_tgi_gemma_multi_lora_adapters_deployment.ipynb notebook demonstrates deploying open-source models like Llama 3 and Gemma on Vertex AI using the Text Generation Inference (TGI) container. This component supports LoRA (Low-Rank Adaptation) adapter deployment for efficient model customization.

from google.cloud import aiplatform

# 1️⃣ Upload the model artifact (saved in Cloud Storage)

model = aiplatform.Model.upload(
    display_name="llama3-finetuned",
    artifact_uri="gs://my-bucket/llama3-finetuned/",
    serving_container_image_uri="us-docker.pkg.dev/vertex-ai/prediction/tgi:latest"
)

# 2️⃣ Deploy to an endpoint

endpoint = model.deploy(
    machine_type="n1-standard-8",
    accelerator_type="NVIDIA_TESLA_T4",
    accelerator_count=1,
    traffic_split={"0": 100}
)

# 3️⃣ Invoke the endpoint

response = endpoint.predict(instances=[{
    "prompt": "Summarize the plot of *The Great Gatsby* in 30 words."
}])
print(response)

Containerized Deployment Patterns

For production deployment, the repository includes Dockerfiles and Terraform snippets for containerizing LLM inference services. The search/web-app/Dockerfile provides a pattern for Flask applications, while partner-models/claude/computer-use-demo/deployment.yaml demonstrates GKE deployment with service account binding via the iam.gke.io/gcp-service-account annotation.

Fine-Tuning and Customization

The open-models/fine-tuning/vertex_ai_trl_fine_tuning_gemma.ipynb notebook provides a complete workflow for instruction-tuning large language models using TensorFlow Reinforcement Learning from Human Feedback (TRL) on Vertex AI. This component demonstrates how to customize open-source models like Gemma with domain-specific datasets while leveraging managed training infrastructure and custom container capabilities.

Data Integration and Partner Models

BigQuery Integration

The workshops/rag-ops/2.2_mvp_chunk_embeddings.ipynb notebook demonstrates tight integration between BigQuery and Vertex AI, including the IAM configuration required for service accounts to invoke generative models directly from SQL queries. Line 933 specifically highlights the necessary permissions for cross-service authentication, enabling vector operations and text generation within data warehouse workflows.

Partner Model Support

The repository extends beyond Google-native models to include partner implementations. The partner-models/claude/computer-use-demo/deployment.yaml file provides a complete GKE deployment specification for Anthropic's Claude, including Cloud Build configurations and service account wiring for enterprise deployments that require multi-cloud AI capabilities.

Summary

• The generative-ai-gcp directory provides modular SDK wrappers in tools/llmevalkit/src/ for prompting, dataset management, and evaluation on Vertex AI.
• Search and retrieval capabilities are demonstrated through the Flask-based Vertex AI Search web application (search/web-app/main.py) and BigQuery vector search notebooks.
• Model serving options range from managed TGI containers for open-source models to containerized Cloud Run and GKE deployment patterns.
• Fine-tuning workflows leverage TRL on Vertex AI for customizing Gemma and other open models with domain-specific data.
• Enterprise integrations include BigQuery SQL-based generation and partner model deployments with proper IAM service account binding.

Frequently Asked Questions

What is the generative-ai-gcp directory in the GoogleCloudPlatform/generative-ai repository?

The generative-ai-gcp directory refers to the collection of Google Cloud Platform-specific resources within the repository, organized into folders like tools/llmevalkit, search, embeddings, and open-models. These components provide reference implementations for building, serving, and evaluating generative AI applications using Vertex AI, BigQuery, and containerized deployment platforms.

How does the Vertex AI Search web application demonstrate enterprise search capabilities?

The search/web-app/main.py file implements a Flask application that interfaces with Vertex AI Search to provide hybrid search functionality, relevance boosting, and metadata filtering. The accompanying consts.py defines searchable schemas and boost rules, while the Dockerfile enables containerized deployment to Cloud Run or GKE, demonstrating a complete production-ready search interface.

What options are available for deploying open-source models using the generative-ai-gcp components?

The repository provides multiple deployment paths for open-source models including the Text Generation Inference (TGI) container on Vertex AI as shown in open-models/serving/vertex_ai_tgi_gemma_multi_lora_adapters_deployment.ipynb, which supports LoRA adapters. Additionally, containerized deployment patterns for Cloud Run and GKE are demonstrated in the search web app and partner model examples, offering flexibility from serverless to Kubernetes-based serving.

How does the repository support fine-tuning workflows on Vertex AI?

The open-models/fine-tuning/vertex_ai_trl_fine_tuning_gemma.ipynb notebook provides a complete implementation for instruction-tuning models using TensorFlow Reinforcement Learning from Human Feedback (TRL). This workflow demonstrates how to customize open-source models like Gemma with domain-specific datasets while leveraging Vertex AI's managed training infrastructure and custom container capabilities.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}