Google Cloud Services Required for the LLM Demos in the generative-ai Repository

To run the LLM demos in the GoogleCloudPlatform/generative-ai repository, you must enable Vertex AI Search, Vertex AI LLM (Gemini/PaLM), Cloud Run, and Cloud Storage, along with the Discovery Engine API and appropriate IAM roles.

The generative-ai repository contains production-ready demonstration applications that showcase retrieval-augmented generation (RAG) patterns. The primary LLM demo located at search/retrieval-augmented-generation/rag_with_dual_llms integrates multiple Google Cloud services to provide a complete question-answering system with dual language models and optional evaluation capabilities.

Core Google Cloud Services for the LLM Demos

The LLM demos require a specific set of mandatory services to function correctly. These services handle document storage, semantic search, language model inference, web hosting, and data management.

Vertex AI Search and Discovery Engine

Vertex AI Search serves as the document corpus storage and semantic retrieval engine for the demos. The underlying Discovery Engine API (discoveryengine.googleapis.com) powers the search functionality, enabling vector-based document retrieval that feeds context to the language models.

According to the source code in search/retrieval-augmented-generation/rag_with_dual_llms/README.md, you must enable this service before deploying any components:

gcloud services enable discoveryengine.googleapis.com

The demo application uses the Discovery Engine to perform semantic retrieval on datasets like CUAD PDFs and Alphabet earnings reports stored in Cloud Storage.

Vertex AI LLM (Gemini and PaLM)

Vertex AI LLM provides the generative capabilities through either Gemini or PaLM models. In search/retrieval-augmented-generation/rag_with_dual_llms/src/vertex_rag_demo_dual_llms_with_judge.py, the application initializes model connections using the VertexAI class with specific model names:

VertexAI(model_name="gemini-pro")  # or palm model variants

The demo supports dual-LLM configurations where one model generates answers and an optional "judge" model evaluates response quality. This requires the Vertex AI API to be enabled in your Google Cloud project.

Cloud Run for Web Hosting

Cloud Run hosts the Streamlit web interface that users interact with. The deployment configuration in the README specifies Cloud Run as the target platform for serving the vertex_rag_demo_dual_llms_with_judge.py application.

The deployment command from the repository root uses the source-based deployment feature:

cd search/retrieval-augmented-generation/rag_with_dual_llms/src
gcloud run deploy vertex-ai-search-demo \
  --source . \
  --region us-central1 \
  --project $PROJECT_ID

Cloud Storage for Data Assets

Cloud Storage holds the sample datasets referenced by the demo applications. The retrieval system accesses documents via Google Cloud Storage URIs (e.g., gs://cloud-samples-data/...) to populate the Vertex AI Search index.

While the demo can work with custom data, the provided examples rely on publicly available sample data stored in Cloud Storage buckets.

IAM and Service Account Configuration

Proper IAM permissions are mandatory for the Cloud Run service account to access the Vertex AI Search index. The repository documentation specifies that the default Compute Engine service account requires the roles/discoveryengine.viewer role.

Configure this permission using the project number:

gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:$(gcloud projects describe $PROJECT_ID \
    --format='value(projectNumber)')-compute@developer.gserviceaccount.com" \
  --role="roles/discoveryengine.viewer"

This grants the necessary read permissions for the search index while maintaining the principle of least privilege.

Optional Google Cloud Services

While the core functionality requires the services above, the demos support additional capabilities through optional Google Cloud services.

Enterprise Knowledge Graph API

The Enterprise Knowledge Graph API enables advanced knowledge graph lookups within the demo interface. To activate this feature, enable the service:

gcloud services enable enterpriseknowledgegraph.googleapis.com

This allows the application to perform entity resolution and knowledge-based augmentations beyond standard vector search.

Cloud Build for Automated Deployment

Cloud Build facilitates one-click deployment from source code to Cloud Run. While optional for local testing, Cloud Build streamlines the CI/CD pipeline when deploying production instances directly from the repository.

Setup and Deployment Commands

Complete the following steps to prepare your Google Cloud environment for the LLM demos.

Enable all required APIs:

gcloud services enable discoveryengine.googleapis.com \
                         enterpriseknowledgegraph.googleapis.com \
                         run.googleapis.com

Authenticate your environment:

gcloud auth login
gcloud auth application-default login

Configure environment variables:

export PROJECT_ID="my-gcp-project"
export LOCATION="us-central1"

Deploy to Cloud Run:

cd search/retrieval-augmented-generation/rag_with_dual_llms/src
gcloud run deploy vertex-ai-search-demo \
  --source . \
  --region $LOCATION \
  --project $PROJECT_ID

Run locally for development:

cd search/retrieval-augmented-generation/rag_with_dual_llms/src
pip install -r requirements.txt
streamlit run vertex_rag_demo_dual_llms_with_judge.py

Add the -- --judge flag when running locally to enable the evaluation model feature.

Source Code Architecture

The implementation relies on specific files that demonstrate the service integration patterns.

• search/retrieval-augmented-generation/rag_with_dual_llms/README.md: Contains the complete setup guide including API enablement, IAM configuration, and deployment instructions.

• search/retrieval-augmented-generation/rag_with_dual_llms/src/vertex_rag_demo_dual_llms_with_judge.py: The main application file that wires together Vertex AI Search, dual LLM instances, and the optional judge model for response evaluation.

• search/web-app/README.md: Documents an alternative web interface implementation using the same core services (Vertex AI Search, Cloud Run, Cloud Storage).

• vision/use-cases/hey_llm/README.md: Shows Google Sheets integration requiring Vertex AI (Gemini) for spreadsheet-based LLM functions.

• tools/llmevalkit/README.md: Demonstrates LLM output evaluation using the Vertex AI Evaluation service.

Summary

• Vertex AI Search and the Discovery Engine API provide the semantic retrieval backbone required for RAG functionality.
• Vertex AI LLM (Gemini or PaLM) powers the generative response capabilities through the VertexAI class interface.
• Cloud Run serves as the hosting platform for the Streamlit demo interface.
• Cloud Storage supplies the document datasets accessed by the retrieval system.
• IAM roles (specifically roles/discoveryengine.viewer) grant necessary permissions to the Cloud Run service account.
• Optional services include the Enterprise Knowledge Graph API for enhanced lookups and Cloud Build for automated deployment pipelines.

Frequently Asked Questions

Do I need to enable billing for these Google Cloud services?

Yes, all listed services require an active billing account. Vertex AI Search, Vertex AI LLM, and Cloud Run incur costs based on usage, storage, and compute time. Enable billing in your Google Cloud project before attempting deployment.

Can I run the LLM demos locally without Cloud Run?

Yes, you can run the demos locally using Streamlit for development and testing. Execute streamlit run vertex_rag_demo_dual_llms_with_judge.py from the source directory after installing requirements. However, the application still requires cloud-based services (Vertex AI Search and LLM APIs) to function, even when running locally.

What IAM role is specifically needed for Vertex AI Search?

The Cloud Run service account requires the roles/discoveryengine.viewer role to read from the search index. According to the repository setup instructions, you must bind this role to the default Compute Engine service account (projectNumber-compute@developer.gserviceaccount.com) using gcloud projects add-iam-policy-binding.

Is Vertex AI Gemini required, or can I use PaLM?

The demo supports both models. The vertex_rag_demo_dual_llms_with_judge.py file accepts various model names through the VertexAI(model_name=...) initialization. You can configure PaLM models or Gemini variants depending on your project requirements and availability in your region.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}