# How to Run Google Cloud LLM Demos Locally: Vertex AI Search and RAG Examples

> Run Google Cloud LLM demos locally like Vertex AI Search and RAG. Clone the repo, install dependencies, authenticate, and start the server for hands-on generative AI examples.

- Repository: [Google Cloud Platform/generative-ai](https://github.com/GoogleCloudPlatform/generative-ai)
- Tags: how-to-guide
- Published: 2026-03-09

---

**You can run the Vertex AI Search and RAG dual-LLM demos locally by cloning the GoogleCloudPlatform/generative-ai repository, installing dependencies from the demo-specific [`requirements.txt`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/requirements.txt), authenticating with `gcloud auth application-default login`, and launching the Streamlit or FastAPI server.**

The GoogleCloudPlatform/generative-ai repository hosts self-contained demonstration applications that showcase Vertex AI capabilities, including a **Vertex AI Search web app** and a **RAG dual-LLM Streamlit demo**. You can run these LLM demos locally on your workstation to experiment with generative AI features, test custom prompts, and validate configurations before deploying to production.

## Clone the Repository and Select a Demo

First, obtain the source code by cloning the repository. The demos reside in specific subdirectories within the `search/` folder.

```bash
git clone https://github.com/GoogleCloudPlatform/generative-ai.git
cd generative-ai

```

The repository contains two primary demonstration applications:

- **Vertex AI Search Web App**: Located in `search/web-app/`, this FastAPI application demonstrates enterprise search capabilities using Vertex AI Search.
- **RAG Dual-LLM Demo**: Located in `search/retrieval-augmented-generation/rag_with_dual_llms/`, this Streamlit application implements retrieval-augmented generation with two LLMs and an optional judge model for evaluation.

## Configure the Python Environment

Each demo requires an isolated Python environment to prevent dependency conflicts. Navigate to your chosen demo directory and create a virtual environment before installing any packages.

For the RAG dual-LLM demo:

```bash
cd search/retrieval-augmented-generation/rag_with_dual_llms
python3 -m venv .venv
source .venv/bin/activate

```

For the Vertex AI Search web app:

```bash
cd search/web-app
python3 -m venv .venv
source .venv/bin/activate

```

## Install Demo-Specific Dependencies

Install the required packages using the [`requirements.txt`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/requirements.txt) file provided in each demo folder. This ensures you receive compatible versions of the Vertex AI SDK, Streamlit, LangChain, FastAPI, and other dependencies.

```bash
pip install --upgrade pip
pip install -r requirements.txt

```

The [`requirements.txt`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/requirements.txt) in `search/retrieval-augmented-generation/rag_with_dual_llms/` specifically pins versions for Streamlit and the Google Cloud AI Platform client library. The [`search/web-app/requirements.txt`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/search/web-app/requirements.txt) includes FastAPI and Uvicorn for the web server, along with the Google Cloud Discovery Engine client.

## Authenticate to Google Cloud

The demos invoke Vertex AI services including Gemini, Search, and Embeddings APIs. You must authenticate using Application Default Credentials (ADC) tied to your GCP project.

Run the following command to open a browser-based authentication flow:

```bash
gcloud auth application-default login

```

After authentication, set the `GOOGLE_CLOUD_PROJECT` environment variable so the code knows which project to bill and which resources to access:

```bash
export GOOGLE_CLOUD_PROJECT=<YOUR_PROJECT_ID>

```

## Launch the Demos Locally

### Run the RAG Dual-LLM Streamlit Demo

Navigate to the source directory and launch the Streamlit application. The main entry point is [`src/vertex_rag_demo_dual_llms.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/src/vertex_rag_demo_dual_llms.py).

```bash
streamlit run src/vertex_rag_demo_dual_llms.py

```

To enable the judge model that evaluates and scores the two LLM responses, use the `with_judge` variant and pass the `--judge` flag:

```bash
streamlit run src/vertex_rag_demo_dual_llms_with_judge.py -- --judge

```

Once running, open your browser to `http://localhost:8501` to interact with the RAG interface.

### Run the Vertex AI Search Web App

For the FastAPI-based web application, launch the development server using Uvicorn. The entry point is `main:app` in the `search/web-app/` directory.

```bash
uvicorn main:app --reload

```

Alternatively, deploy directly to Cloud Run from source without running a local server:

```bash
gcloud run deploy vertex-ai-search-demo --source .

```

When running locally with Uvicorn, visit `http://127.0.0.1:8000` to access the search interface.

## Summary

- Clone the **GoogleCloudPlatform/generative-ai** repository to access demo source code in `search/web-app/` and `search/retrieval-augmented-generation/rag_with_dual_llms/`.
- Create a Python virtual environment and install dependencies from the demo-specific [`requirements.txt`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/requirements.txt) files to ensure compatible versions of Streamlit, FastAPI, and the Vertex AI SDK.
- Authenticate using `gcloud auth application-default login` and set `GOOGLE_CLOUD_PROJECT` to enable API access.
- Launch the RAG demo with `streamlit run src/vertex_rag_demo_dual_llms.py` or the web app with `uvicorn main:app --reload`.
- Access the interfaces at `http://localhost:8501` for Streamlit or `http://127.0.0.1:8000` for the FastAPI server.

## Frequently Asked Questions

### What are the system requirements for running these LLM demos locally?

You need Python 3.9 or higher, approximately 500 MB of disk space for the virtual environment, and a Google Cloud project with the Vertex AI API enabled. The demos run on macOS, Linux, and Windows (with WSL recommended for Windows users).

### How do I troubleshoot authentication errors when connecting to Vertex AI?

Ensure you have run `gcloud auth application-default login` and that the `GOOGLE_CLOUD_PROJECT` environment variable matches a project where you have Vertex AI Administrator or Editor permissions. Verify the Vertex AI API is enabled in the Google Cloud Console under "APIs & Services > Library".

### Can I modify the LLM models used in the demos?

Yes. The README files in each demo folder document how to customize the application, including swapping the LLM model name and providing custom prompts. You can edit the model configuration in [`src/vertex_rag_demo_dual_llms.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/src/vertex_rag_demo_dual_llms.py) or set environment variables as described in the specific demo's README to point to different Gemini or PaLM models in Vertex AI.

### How do I clean up resources after running the demos?

If you deployed the Vertex AI Search web app to Cloud Run using `gcloud run deploy`, delete the service to avoid ongoing charges by running `gcloud run services delete vertex-ai-search-demo`. For local development servers, simply deactivate the virtual environment with `deactivate` and delete the `.venv` directory if desired.