RAG Examples in the Google Cloud Generative AI Repository: LLM-Demos Directory Explained

The GoogleCloudPlatform/generative-ai repository does not contain an LLM-demos directory; instead, Retrieval-Augmented Generation (RAG) examples are organized across workshops/rag-ops/, search/retrieval-augmented-generation/, and workshops/qa-ops/.

The GoogleCloudPlatform/generative-ai repository provides production-ready implementations of Retrieval-Augmented Generation (RAG) using Vertex AI Search and Gemini models. While developers frequently search for RAG examples in a hypothetical LLM-demos folder, the actual implementations are distributed across specialized directories that demonstrate end-to-end retrieval patterns, multimodal pipelines, and dual-LLM evaluation frameworks.

RAG Implementation Locations (Instead of LLM-Demos)

The repository organizes RAG functionality into four primary locations. Each directory serves a distinct architectural purpose, from educational notebooks to production Streamlit applications.

workshops/rag-ops/ (MVP RAG Pipeline)

The workshops/rag-ops/ directory contains a step-by-step "MVP" series that builds a multimodal RAG pipeline using Vertex Gemini. The key file is 2.3_mvp_rag.ipynb, which demonstrates end-to-end retrieval, citation generation, and evaluation metrics.

This notebook implements the core RAG pattern: document chunking, embedding generation, vector storage, and context-aware generation with source citations.

search/retrieval-augmented-generation/ (Production Demos)

The search/retrieval-augmented-generation/ directory houses Streamlit-based demos that illustrate how to combine Vertex AI Search (retrieval) with LLMs. This location contains two significant implementations:

Single-LLM RAG Flow

The examples/question_answering.ipynb notebook demonstrates a basic RAG implementation using LangChain integration:

# Initialise Vertex AI LLM (Gemini)

from vertexai.preview.language_models import ChatModel
chat_model = ChatModel.from_pretrained("gemini-1.5-flash-001")
llm = chat_model.start_chat()

# Initialise Vertex AI Search retriever

from langchain.retrievers import VertexAISearchRetriever
retriever = VertexAISearchRetriever(
    project_id="YOUR_PROJECT",
    location="us-central1",
    data_store_id="my_datastore",          # pre‑created Vertex AI Search datastore

    max_chunks=5,                           # number of chunks to fetch per query

)

# RAG query

query = "How does TPU v5e scaling impact LLM training cost?"
docs = retriever.get_relevant_documents(query)      # retrieval step

context = "\n".join([d.page_content for d in docs]) # concatenate chunks

# Prompt with retrieved context

prompt = f"""You are a helpful AI assistant.
Context:
{context}

Answer the following question using only the above context:
{query}"""

response = llm.send_message(prompt)
print(response.text)

Key points – Retrieval is performed first; the retrieved chunks are injected into the prompt before the LLM call. The notebook expands this pattern with LangChain ConversationalRetrievalChain, memory handling, and evaluation metrics.

Dual-LLM RAG Comparison

The rag_with_dual_llms/src/vertex_rag_demo_dual_llms_with_judge.py file implements a sophisticated comparison pattern where two different models answer the same query using shared retrieved context:

# Initialize two LLMs (Gemini and PaLM2) for comparison

from vertexai.preview.language_models import ChatModel
llm_left  = ChatModel.from_pretrained("gemini-1.5-flash-001").start_chat()
llm_right = ChatModel.from_pretrained("text-bison-001").start_chat()

# Retrieve context once (shared) using Vertex AI Search

docs = retriever.get_relevant_documents(user_query)
shared_context = "\n".join(d.page_content for d in docs)

def ask_llm(llm, query, context):
    prompt = f"""Context:
{context}

Answer the question:
{query}"""
    return llm.send_message(prompt).text

left_answer  = ask_llm(llm_left,  user_query, shared_context)
right_answer = ask_llm(llm_right, user_query, shared_context)

# Optional judge model to decide which answer is better

judge = ChatModel.from_pretrained("gemini-1.5-pro-001").start_chat()
judge_prompt = f"""You are a judge. Compare the two answers to the same question and output the better one.

Question: {user_query}
Answer A: {left_answer}
Answer B: {right_answer}"""
best_answer = judge.send_message(judge_prompt).text

Key points – Retrieval is performed once, then the same context is fed to two different LLMs. An optional "judge" LLM ranks the responses.

workshops/qa-ops/ (Multimodal RAG)

The workshops/qa-ops/ directory demonstrates both text-only and multimodal RAG (mRAG) techniques. The building_DIY_multimodal_qa_system_with_mRAG.ipynb notebook implements text + image chunking, embeddings, and RAG-driven QA:

# 1️⃣ Extract text & image embeddings with Vertex AI Embeddings

from vertexai.preview.vision_models import ImageModel
image_model = ImageModel.from_pretrained("imagen-2.0-001")

def embed_document(text, image_bytes):
    txt_emb = vertexai.preview.language_models.TextEmbeddingModel.from_pretrained(
                 "textembedding-gecko@001").get_embeddings([text])[0]
    img_emb = image_model.get_embeddings([image_bytes])[0]
    return txt_emb, img_emb

# 2️⃣ Store both embeddings in a vector DB (e.g., Vertex AI Matching Engine)

# (code omitted for brevity – uses `google-cloud-aiplatform` client)

# 3️⃣ Retrieval: fetch top‑k text + image chunks that match the query embedding

# 4️⃣ Prompt: combine textual snippets and a short description of the images

prompt = f"""You are a multimodal assistant.
Context (text chunks):
{text_chunks}

Context (image descriptions):
{image_descriptions}

Answer the user question:
{user_question}"""
response = llm.send_message(prompt)

Key points – Both modalities are embedded, stored together, and retrieved simultaneously; the LLM receives a mixed-modality context.

Additional RAG Resources

Beyond the primary implementation directories, the repository includes supporting utilities for RAG deployment.

search/web-app/ (RAG Toggle UI)

The search/web-app/ directory contains a Flask-style web UI that toggles RAG on and off. The main.py file includes a "🦜️🔗 Retrieval Augmented Generation (RAG)" UI element that allows users to compare grounded versus ungrounded generation.

search/vertexai-search-options/ (Retrieval Engine Configuration)

The search/vertexai-search-options/ directory describes how Vertex AI Search can be configured as the retrieval engine for RAG systems. The vertexai_search_options.ipynb notebook covers search add-on configuration for LLM-grounded answers.

Summary

• The GoogleCloudPlatform/generative-ai repository does not contain an LLM-demos directory; RAG implementations are distributed across specialized folders.
• workshops/rag-ops/2.3_mvp_rag.ipynb provides the foundational MVP notebook demonstrating end-to-end multimodal RAG with citations.
• search/retrieval-augmented-generation/ contains production Streamlit demos, including single-LLM flows and dual-LLM comparison frameworks with judge models.
• workshops/qa-ops/ implements multimodal RAG (mRAG) supporting both text and image retrieval using Vertex AI Embeddings.
• search/web-app/main.py offers a simple Flask UI for toggling RAG functionality on and off.
• All implementations utilize Vertex AI Search or Vertex AI Matching Engine as the retrieval backend, paired with Gemini or PaLM2 models for generation.

Frequently Asked Questions

Why can't I find the LLM-demos directory in the Google Cloud generative-ai repository?

The repository does not include an LLM-demos folder because the project organizes demonstrations by functional domain rather than by generic LLM categories. RAG-specific code resides in workshops/rag-ops/, search/retrieval-augmented-generation/, and workshops/qa-ops/ directories that reflect specific architectural patterns and use cases.

What is the difference between the RAG implementations in workshops/rag-ops and search/retrieval-augmented-generation?

The workshops/rag-ops/ directory focuses on educational, step-by-step MVP builds that teach multimodal RAG concepts using Vertex Gemini, including citation generation and evaluation frameworks. In contrast, search/retrieval-augmented-generation/ contains production-oriented Streamlit applications that demonstrate integration with Vertex AI Search, including advanced patterns like dual-LLM comparison and judge-based answer selection.

How does the multimodal RAG implementation handle images and text together?

The multimodal RAG implementation in workshops/qa-ops/building_DIY_multimodal_qa_system_with_mRAG.ipynb processes both modalities by generating separate embeddings for text (using textembedding-gecko@001) and images (using imagen-2.0-001), storing both in a vector database such as Vertex AI Matching Engine, and retrieving top-k matches from both modalities simultaneously to create a combined context prompt for the LLM.

Can I use Vertex AI Search as the retrieval engine for these RAG examples?

Yes, multiple RAG implementations in the repository use Vertex AI Search as the primary retrieval backend. The search/retrieval-augmented-generation/examples/question_answering.ipynb notebook specifically demonstrates using VertexAISearchRetriever from LangChain to fetch relevant chunks from a Vertex AI Search datastore before passing them to Gemini models for grounded generation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}