Gemini API Examples in Google Cloud: Navigating the generative-ai Repository

The GoogleCloudPlatform/generative-ai repository does not contain a generative-ai-gcp directory, but it houses extensive Gemini API examples across the workshops, gemini, and open-models folders.

The GoogleCloudPlatform/generative-ai repository serves as the definitive reference for implementing Google's Gemini API on Vertex AI. While the specific generative-ai-gcp directory does not exist, the repository provides production-ready code samples spanning from basic SDK initialization to advanced multimodal RAG systems and fine-tuning workflows.

Where to Find Gemini API Examples in the Repository

Despite the absence of a generative-ai-gcp folder, Gemini API implementations are distributed strategically throughout the repository's architecture. The primary locations include the workshops directory for instructional notebooks, the gemini folder for use-case-specific implementations, and open-models for deployment scenarios. These locations collectively demonstrate the full lifecycle of Gemini API requests, from authentication and model selection to streaming responses and token-count optimization.

Core Gemini API Implementation Patterns

The repository showcases consistent architectural patterns for Gemini API integration using the Vertex AI Python SDK. Each pattern targets specific operational requirements while maintaining standardized initialization workflows.

SDK Initialization and Authentication

Every notebook begins with the vertexai module import and explicit project configuration. In workshops/rag-ops/1_prototyping_gemini.ipynb, the initialization follows this pattern:

import vertexai
vertexai.init(project="my-gcp-project", location="us-central1")

The SDK automatically respects default environment variables when explicit parameters are omitted, enabling seamless transitions between development and production environments.

Model Instantiation and Configuration

Gemini model access occurs through the GenerativeModel class imported from vertexai.generative_models. The repository demonstrates instantiation with specific model IDs such as gemini-2.0-flash or custom fine-tuned endpoints. For example, gemini/use-cases/retrieval-augmented-generation/intro_multimodal_rag.ipynb illustrates loading both standard and specialized model versions to handle multimodal RAG pipelines.

Multimodal Content and Prompt Construction

The examples extensively utilize the Part class for constructing complex prompts that combine text, images, video, and documents. In workshops/ai-agents/ai_agents_for_engineers.ipynb, prompts are assembled as mixed-type lists:

from vertexai.generative_models import Part

prompt = [
    Part.from_image_uri("gs://my-bucket/sample.jpg"),
    "Describe the scene in plain English, then list three key insights."
]

This pattern supports ingestion of PDFs, audio files, and video content through methods like Part.from_file() and Part.from_image().

Safety Settings and Generation Configuration

Production implementations require explicit safety filters and generation parameters. The repository consistently implements GenerationConfig for controlling temperature, maximum output tokens, and top-p sampling thresholds. Safety constraints are enforced using HarmCategory enumerations paired with HarmBlockThreshold levels, as shown in the prototyping workshops:

from vertexai.generative_models import GenerationConfig, HarmCategory, HarmBlockThreshold

gen_config = GenerationConfig(
    temperature=0.7,
    max_output_tokens=1024,
    top_p=0.95,
)

safety_settings = {
    HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
    HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
    HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
    HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
}

Complete Gemini API Implementation Example

The following implementation synthesizes patterns from workshops/rag-ops/1_prototyping_gemini.ipynb and related notebooks, demonstrating SDK initialization, multimodal prompting, streaming responses, and token counting:

# 1️⃣ Initialise the Vertex AI SDK (uses default env vars if set)

import vertexai
vertexai.init(project="my-gcp-project", location="us-central1")

# 2️⃣ Import Gemini classes

from vertexai.generative_models import (
    GenerativeModel,
    GenerationConfig,
    HarmCategory,
    HarmBlockThreshold,
    Part,
)

# 3️⃣ Choose a Gemini model (flash for quick prototyping)

MODEL_ID = "gemini-2.0-flash"
model = GenerativeModel(MODEL_ID)

# 4️⃣ Define generation parameters and safety filters

gen_config = GenerationConfig(
    temperature=0.7,
    max_output_tokens=1024,
    top_p=0.95,
)

safety_settings = {
    HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
    HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
    HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
    HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
}

# 5️⃣ Build a multimodal prompt (image + text)

prompt = [
    Part.from_image_uri("gs://my-bucket/sample.jpg"),
    "Describe the scene in plain English, then list three key insights."
]

# 6️⃣ Generate a response (streaming version)

response = model.generate_content(
    prompt,
    generation_config=gen_config,
    safety_settings=safety_settings,
    stream=True,
)

# 7️⃣ Consume streamed chunks

for chunk in response:
    print(chunk.text)          # partial output

    print("-" * 80)

# 8️⃣ Token-count helper (cost estimation)

token_info = model.count_tokens(prompt)
print(f"Tokens: {token_info.total_tokens}, Billable chars: {token_info.total_billable_characters}")

Key Notebooks for Specific Use Cases

The repository organizes Gemini API examples by functional domain, allowing developers to locate relevant implementations efficiently:

• workshops/rag-ops/1_prototyping_gemini.ipynb: Demonstrates fundamental API patterns including model loading, streaming responses, and token utilization tracking.
• workshops/rag-ops/2.3_mvp_rag.ipynb: Implements end-to-end Retrieval-Augmented Generation using Gemini with chunked embeddings and context stitching.
• gemini/use-cases/retrieval-augmented-generation/intro_multimodal_rag.ipynb: Shows multimodal RAG implementations combining text, PDF documents, and audio inputs.
• gemini/use-cases/video-analysis/video_analysis_with_youtube_data_api_and_batch_prediction.ipynb: Illustrates video-to-text extraction pipelines and subsequent Gemini summarization.
• gemini/tuning/sft_gemini_visual_defect_detection.ipynb: Loads fine-tuned Gemini endpoints via GenerativeModel(custom_endpoint_name) for specialized visual inspection tasks.
• workshops/ai-agents/ai_agents_for_engineers.ipynb: Covers zero-shot, few-shot, and chain-of-thought prompting patterns.
• setup-env/README.md: Provides environment configuration instructions for GCP project setup and service account permissions.

Summary

• The generative-ai-gcp directory does not exist within the GoogleCloudPlatform/generative-ai repository.
• Gemini API examples are distributed across workshops, gemini, and open-models directories.
• All implementations utilize the vertexai Python SDK with consistent initialization via vertexai.init().
• The GenerativeModel class supports both standard model IDs (gemini-2.0-flash) and custom fine-tuned endpoints.
• Multimodal inputs are handled through the Part class, supporting images, video, audio, and documents.
• Production deployments require explicit GenerationConfig and HarmCategory safety configurations.
• The count_tokens() method enables precise cost estimation before API invocation.

Frequently Asked Questions

Is there a generative-ai-gcp directory in the GoogleCloudPlatform/generative-ai repository?

No, the repository does not contain a directory named generative-ai-gcp. Instead, Gemini API examples are organized within the workshops, gemini, and open-models directories. These folders contain comprehensive notebooks demonstrating various implementation patterns.

How do I initialize the Gemini API using the examples provided?

Initialize the SDK by importing vertexai and calling vertexai.init(project="your-project", location="your-region"). The examples in workshops/rag-ops/1_prototyping_gemini.ipynb demonstrate this pattern, including support for default environment variables when explicit parameters are not provided.

Can I process multimodal inputs using these Gemini API examples?

Yes, the repository extensively demonstrates multimodal processing using the Part class from vertexai.generative_models. Notebooks like workshops/ai-agents/ai_agents_for_engineers.ipynb show how to combine text, images, PDFs, and video content within a single prompt using methods such as Part.from_image() and Part.from_file().

Where can I find examples of fine-tuned Gemini models in the repository?

The gemini/tuning/sft_gemini_visual_defect_detection.ipynb notebook demonstrates loading and utilizing fine-tuned Gemini endpoints. It shows how to instantiate GenerativeModel with a custom endpoint name rather than a standard model ID, enabling specialized inference for specific domains like visual defect detection.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}