How LLM Demos in Google Cloud's Generative AI Repository Showcase Core Capabilities

The LLM demos in the GoogleCloudPlatform/generative-ai repository demonstrate end-to-end generative AI capabilities through production-ready examples that combine Vertex AI services, Gemini models, retrieval-augmented generation (RAG), and multi-agent orchestration.

These runnable demos illustrate how to stitch together the core building blocks of modern LLM applications. Each example in the repository—from dual-LLM comparison tools to agentic concierges—shows the complete flow from user input through retrieval, processing, and LLM generation, all deployed on Google Cloud's Vertex AI platform.

Dual-LLM RAG Comparison: Evaluating Retrieval-Augmented Generation

The Dual-LLM RAG comparison demo showcases how to implement and evaluate retrieval-augmented generation by running two separate Gemini models against the same Vertex AI Search index. This pattern demonstrates multi-model evaluation and LLM-as-a-judge techniques.

Architecture and Implementation

The demo in search/retrieval-augmented-generation/rag_with_dual_llms/src/vertex_rag_demo_dual_llms_with_judge.py implements a four-stage pipeline:

1. Vertex AI Search retrieves relevant documents from the index
2. Two independent Gemini LLM back-ends process the same query with retrieved context
3. Optional LLM-as-Judge scores and ranks the two responses
4. Streamlit front-end provides interactive side-by-side comparison

To launch the demo with the evaluation judge enabled:

cd search/retrieval-augmented-generation/rag_with_dual_llms
streamlit run vertex_rag_demo_dual_llms_with_judge.py -- --judge

Vertex AI Search Web-App: Production RAG Deployment

The Vertex AI Search web-app demo illustrates how to deploy a full-stack, production-ready RAG interface. This example highlights the "Search + LLM" pattern with real-time user event logging and Cloud Run deployment capabilities.

Full-Stack Implementation

Located in search/web-app/, this demo combines:

• Backend: Python Flask-style FastAPI application (app.py) that initializes the Gemini LLM
• Retrieval: Vertex AI Search API for document corpus queries
• Generation: Gemini model generates answers with citations from retrieved data
• Frontend: Jinja templates (templates/search.html) rendering the chat UI
• Observability: Optional Vertex AI "VAIS" event logging for relevance feedback

The LLM initialization pattern used throughout the demo:

from vertexai.preview.language_models import ChatModel

def init_llm():
    # Gemini 1.5 Flash (cheaper) – replace with any Gemini model ID

    model_name = "gemini-1.5-flash-001"
    return ChatModel.from_pretrained(model_name)

GenKit Postcard-Generator: Multimodal AI Product Development

The GenKit postcard-generator demo showcases rapid prototyping of generative AI products using the GenKit SDK. This example demonstrates prompt-template management, multimodal generation (text + image), and the Streamlit development workflow.

GenKit Architecture

Found in genkit/postcard-generator/docs/demo.md, this demo implements:

1. GenKit SDK (genkit package) for orchestrating AI flows
2. Prompt templates using the @prompts.prompt decorator for reusable text generation
3. Gemini 2.5 Flash for creative text generation
4. Imagen 2 for generating custom images based on text prompts
5. Streamlit UI for live preview and postcard export

The core generation logic:

import genkit as gk
from genkit import prompts

@prompts.prompt
def postcard_prompt(name: str):
    return f"Write a short, friendly postcard for {name} visiting Paris."

def generate_postcard(name: str):
    # Gemini 2.5 Flash for text, Imagen for image

    text = gk.run(postcard_prompt(name))
    image = gk.run("generate_image", prompt=f"{name} in front of the Eiffel Tower")
    return {"text": text, "image_url": image}

Concierge LangGraph Demo: Multi-Agent Orchestration

The Concierge LangGraph demo demonstrates advanced agentic AI patterns by implementing a "concierge" that routes queries between expert assistants. This example highlights LLM-based intent classification, function calling, and multi-agent orchestration using LangGraph.

Agent Architecture

Located in gemini/agents/genai-experience-concierge/, this demo features:

• LangGraph workflow: State-machine graph defining agent transitions
• Intent classifier: Gemini LLM routes queries to "Retail Search" or "Customer Support" experts
• Function calling: Structured schema to query synthetic BigQuery stores
• Memory integration: Context retention across conversation turns
• Frontend: Flask + Jinja interface (langgraph-demo/frontend/concierge_ui/demo_page.py)

The routing logic implemented in the LangGraph state machine:

from langgraph.graph import StateGraph, END

def route_intent(state: dict):
    # LLM-based intent classifier returns "retail" or "support"

    intent = llm_classifier(state["user_input"])
    if intent == "retail":
        return "RetailSearchAgent"
    else:
        return "SupportAgent"

graph = StateGraph(state_schema=dict)
graph.add_node("RetailSearchAgent", retail_agent)
graph.add_node("SupportAgent", support_agent)
graph.add_conditional_edges(
    start="Router",  # entry node

    cond_fn=route_intent,
    true_node="RetailSearchAgent",
    false_node="SupportAgent",
)
graph.set_entry_point("Router")
graph.add_edge("RetailSearchAgent", END)
graph.add_edge("SupportAgent", END)
app = graph.compile()

Summary

The LLM demos in the GoogleCloudPlatform/generative-ai repository showcase generative AI capabilities through concrete, deployable examples:

• Dual-LLM RAG comparison demonstrates retrieval-augmented generation with multi-model evaluation and LLM-as-a-judge patterns using Vertex AI Search and Gemini.
• Vertex AI Search web-app provides a production-ready full-stack implementation combining FastAPI, Jinja templates, and real-time event logging for grounded Q&A.
• GenKit postcard-generator illustrates multimodal AI product development with prompt templates, text generation (Gemini 2.5 Flash), and image generation (Imagen 2).
• Concierge LangGraph demo exhibits advanced agent orchestration with LLM-based intent classification, function calling, and state-machine workflows routing between expert assistants.

Frequently Asked Questions

What is the primary purpose of the Dual-LLM RAG demo?

The Dual-LLM RAG demo primarily demonstrates how to evaluate retrieval-augmented generation quality by running two separate Gemini models against the same Vertex AI Search index and comparing their outputs. It showcases the LLM-as-a-judge pattern where a third model can score responses, providing a framework for systematic RAG evaluation.

How does the Concierge LangGraph demo handle query routing?

The Concierge LangGraph demo implements an LLM-based intent classifier that analyzes user input and routes queries to either a Retail Search Agent or Customer Support Agent. This routing logic is implemented as a conditional edge in a LangGraph state machine, where the route_intent function returns the target node based on the classified intent, enabling dynamic multi-agent orchestration.

What technologies are used in the GenKit postcard-generator demo?

The GenKit postcard-generator demo combines the GenKit SDK for AI workflow orchestration, Gemini 2.5 Flash for creative text generation, and Imagen 2 for image synthesis. The implementation uses Streamlit for the user interface and demonstrates prompt template management through Python decorators, providing a complete multimodal AI product development example.

Can these demos be deployed to production environments?

Yes, these demos are designed as production-ready examples. The Vertex AI Search web-app specifically includes Cloud Run deployment configurations and real-time event logging for relevance feedback. The Dual-LLM RAG demo uses Streamlit for rapid prototyping but can be containerized, while the Concierge LangGraph demo provides a Flask-based frontend suitable for production deployment with proper scaling and security configurations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}