# RAG Examples in the Google Cloud Generative AI Repository: LLM-Demos Directory Explained

> Explore Retrieval Augmented Generation RAG examples within the GoogleCloudPlatform/generative-ai repository. Discover RAG implementations in rag-ops and qa-ops directories.

- Repository: [Google Cloud Platform/generative-ai](https://github.com/GoogleCloudPlatform/generative-ai)
- Tags: tutorial
- Published: 2026-03-09

---

**The GoogleCloudPlatform/generative-ai repository does not contain an `LLM-demos` directory; instead, Retrieval-Augmented Generation (RAG) examples are organized across `workshops/rag-ops/`, `search/retrieval-augmented-generation/`, and `workshops/qa-ops/`.**

The GoogleCloudPlatform/generative-ai repository provides production-ready implementations of **Retrieval-Augmented Generation (RAG)** using Vertex AI Search and Gemini models. While developers frequently search for RAG examples in a hypothetical `LLM-demos` folder, the actual implementations are distributed across specialized directories that demonstrate end-to-end retrieval patterns, multimodal pipelines, and dual-LLM evaluation frameworks.

## RAG Implementation Locations (Instead of LLM-Demos)

The repository organizes RAG functionality into four primary locations. Each directory serves a distinct architectural purpose, from educational notebooks to production Streamlit applications.

### workshops/rag-ops/ (MVP RAG Pipeline)

The `workshops/rag-ops/` directory contains a step-by-step "MVP" series that builds a multimodal RAG pipeline using Vertex Gemini. The key file is **`2.3_mvp_rag.ipynb`**, which demonstrates end-to-end retrieval, citation generation, and evaluation metrics.

This notebook implements the core RAG pattern: document chunking, embedding generation, vector storage, and context-aware generation with source citations.

### search/retrieval-augmented-generation/ (Production Demos)

The `search/retrieval-augmented-generation/` directory houses Streamlit-based demos that illustrate how to combine Vertex AI Search (retrieval) with LLMs. This location contains two significant implementations:

**Single-LLM RAG Flow**

The `examples/question_answering.ipynb` notebook demonstrates a basic RAG implementation using LangChain integration:

```python

# Initialise Vertex AI LLM (Gemini)

from vertexai.preview.language_models import ChatModel
chat_model = ChatModel.from_pretrained("gemini-1.5-flash-001")
llm = chat_model.start_chat()

# Initialise Vertex AI Search retriever

from langchain.retrievers import VertexAISearchRetriever
retriever = VertexAISearchRetriever(
    project_id="YOUR_PROJECT",
    location="us-central1",
    data_store_id="my_datastore",          # pre‑created Vertex AI Search datastore

    max_chunks=5,                           # number of chunks to fetch per query

)

# RAG query

query = "How does TPU v5e scaling impact LLM training cost?"
docs = retriever.get_relevant_documents(query)      # retrieval step

context = "\n".join([d.page_content for d in docs]) # concatenate chunks

# Prompt with retrieved context

prompt = f"""You are a helpful AI assistant.
Context:
{context}

Answer the following question using only the above context:
{query}"""

response = llm.send_message(prompt)
print(response.text)

```

*Key points* – Retrieval is performed first; the retrieved chunks are injected into the prompt before the LLM call. The notebook expands this pattern with LangChain `ConversationalRetrievalChain`, memory handling, and evaluation metrics.

**Dual-LLM RAG Comparison**

The [`rag_with_dual_llms/src/vertex_rag_demo_dual_llms_with_judge.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/rag_with_dual_llms/src/vertex_rag_demo_dual_llms_with_judge.py) file implements a sophisticated comparison pattern where two different models answer the same query using shared retrieved context:

```python

# Initialize two LLMs (Gemini and PaLM2) for comparison

from vertexai.preview.language_models import ChatModel
llm_left  = ChatModel.from_pretrained("gemini-1.5-flash-001").start_chat()
llm_right = ChatModel.from_pretrained("text-bison-001").start_chat()

# Retrieve context once (shared) using Vertex AI Search

docs = retriever.get_relevant_documents(user_query)
shared_context = "\n".join(d.page_content for d in docs)

def ask_llm(llm, query, context):
    prompt = f"""Context:
{context}

Answer the question:
{query}"""
    return llm.send_message(prompt).text

left_answer  = ask_llm(llm_left,  user_query, shared_context)
right_answer = ask_llm(llm_right, user_query, shared_context)

# Optional judge model to decide which answer is better

judge = ChatModel.from_pretrained("gemini-1.5-pro-001").start_chat()
judge_prompt = f"""You are a judge. Compare the two answers to the same question and output the better one.

Question: {user_query}
Answer A: {left_answer}
Answer B: {right_answer}"""
best_answer = judge.send_message(judge_prompt).text

```

*Key points* – Retrieval is performed **once**, then the same context is fed to two different LLMs. An optional "judge" LLM ranks the responses.

### workshops/qa-ops/ (Multimodal RAG)

The `workshops/qa-ops/` directory demonstrates both text-only and **multimodal RAG (mRAG)** techniques. The `building_DIY_multimodal_qa_system_with_mRAG.ipynb` notebook implements text + image chunking, embeddings, and RAG-driven QA:

```python

# 1️⃣ Extract text & image embeddings with Vertex AI Embeddings

from vertexai.preview.vision_models import ImageModel
image_model = ImageModel.from_pretrained("imagen-2.0-001")

def embed_document(text, image_bytes):
    txt_emb = vertexai.preview.language_models.TextEmbeddingModel.from_pretrained(
                 "textembedding-gecko@001").get_embeddings([text])[0]
    img_emb = image_model.get_embeddings([image_bytes])[0]
    return txt_emb, img_emb

# 2️⃣ Store both embeddings in a vector DB (e.g., Vertex AI Matching Engine)

# (code omitted for brevity – uses `google-cloud-aiplatform` client)

# 3️⃣ Retrieval: fetch top‑k text + image chunks that match the query embedding

# 4️⃣ Prompt: combine textual snippets and a short description of the images

prompt = f"""You are a multimodal assistant.
Context (text chunks):
{text_chunks}

Context (image descriptions):
{image_descriptions}

Answer the user question:
{user_question}"""
response = llm.send_message(prompt)

```

*Key points* – Both modalities are embedded, stored together, and retrieved simultaneously; the LLM receives a mixed-modality context.

## Additional RAG Resources

Beyond the primary implementation directories, the repository includes supporting utilities for RAG deployment.

### search/web-app/ (RAG Toggle UI)

The `search/web-app/` directory contains a Flask-style web UI that toggles RAG on and off. The [`main.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/main.py) file includes a `"🦜️🔗 Retrieval Augmented Generation (RAG)"` UI element that allows users to compare grounded versus ungrounded generation.

### search/vertexai-search-options/ (Retrieval Engine Configuration)

The `search/vertexai-search-options/` directory describes how Vertex AI Search can be configured as the retrieval engine for RAG systems. The `vertexai_search_options.ipynb` notebook covers search add-on configuration for LLM-grounded answers.

## Summary

- The **GoogleCloudPlatform/generative-ai** repository does not contain an `LLM-demos` directory; RAG implementations are distributed across specialized folders.
- **`workshops/rag-ops/2.3_mvp_rag.ipynb`** provides the foundational MVP notebook demonstrating end-to-end multimodal RAG with citations.
- **`search/retrieval-augmented-generation/`** contains production Streamlit demos, including single-LLM flows and dual-LLM comparison frameworks with judge models.
- **`workshops/qa-ops/`** implements **multimodal RAG (mRAG)** supporting both text and image retrieval using Vertex AI Embeddings.
- **[`search/web-app/main.py`](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/search/web-app/main.py)** offers a simple Flask UI for toggling RAG functionality on and off.
- All implementations utilize **Vertex AI Search** or **Vertex AI Matching Engine** as the retrieval backend, paired with **Gemini** or **PaLM2** models for generation.

## Frequently Asked Questions

### Why can't I find the LLM-demos directory in the Google Cloud generative-ai repository?

The repository does not include an `LLM-demos` folder because the project organizes demonstrations by functional domain rather than by generic LLM categories. RAG-specific code resides in `workshops/rag-ops/`, `search/retrieval-augmented-generation/`, and `workshops/qa-ops/` directories that reflect specific architectural patterns and use cases.

### What is the difference between the RAG implementations in workshops/rag-ops and search/retrieval-augmented-generation?

The `workshops/rag-ops/` directory focuses on educational, step-by-step MVP builds that teach multimodal RAG concepts using Vertex Gemini, including citation generation and evaluation frameworks. In contrast, `search/retrieval-augmented-generation/` contains production-oriented Streamlit applications that demonstrate integration with Vertex AI Search, including advanced patterns like dual-LLM comparison and judge-based answer selection.

### How does the multimodal RAG implementation handle images and text together?

The multimodal RAG implementation in `workshops/qa-ops/building_DIY_multimodal_qa_system_with_mRAG.ipynb` processes both modalities by generating separate embeddings for text (using `textembedding-gecko@001`) and images (using `imagen-2.0-001`), storing both in a vector database such as Vertex AI Matching Engine, and retrieving top-k matches from both modalities simultaneously to create a combined context prompt for the LLM.

### Can I use Vertex AI Search as the retrieval engine for these RAG examples?

Yes, multiple RAG implementations in the repository use Vertex AI Search as the primary retrieval backend. The `search/retrieval-augmented-generation/examples/question_answering.ipynb` notebook specifically demonstrates using `VertexAISearchRetriever` from LangChain to fetch relevant chunks from a Vertex AI Search datastore before passing them to Gemini models for grounded generation.