Discovery Agent Query Generation and Result Reranking Mechanisms in Google Cloud Knowledge Catalog

The Discovery Agent delegates query understanding to Gemini while relying entirely on Dataplex's built-in semantic search and ranking engine, passing results through without client-side reranking unless explicitly customized.

The Discovery Agent in the GoogleCloudPlatform/knowledge-catalog repository demonstrates how large language models can interface with Data Catalog APIs to transform natural language into structured data discovery. Understanding the Discovery agent query generation and result reranking mechanisms reveals a clean separation between LLM-based intent parsing and backend search optimization, with extension points available for custom ranking logic.

Agent Architecture and Query Generation Pipeline

The Discovery Agent follows a straightforward pattern: the LLM interprets user intent and delegates the actual search execution to a bound tool function.

LLM Agent Configuration

In samples/discovery/agent.py, the agent is instantiated using the google.adk.agents.llm_agent.Agent class configured with the Gemini-3 Flash model (GEMINI_MODEL). The agent loads its behavioral instructions from samples/discovery/SKILL.md, which defines how to handle user requests and when to invoke the search tool.

# Conceptual structure from samples/discovery/agent.py

agent = Agent(
    model=GEMINI_MODEL,
    name="discovery_agent",
    instruction="".join(open("samples/discovery/SKILL.md").readlines()),
    tools=[knowledge_catalog_search]
)

Tool Binding and Query Forwarding

The agent binds to a single tool defined in samples/discovery/tools.py. When the LLM decides to perform a search, it calls knowledge_catalog_search with the exact text the user typed as the query argument. The agent does not rewrite, expand, or optimize the query string; it relies entirely on Dataplex's semantic search capabilities to interpret the natural language.

Dataplex Semantic Search and Result Ranking

Rather than implementing custom ranking algorithms, the Discovery Agent leverages Google's managed Dataplex service for both semantic query interpretation and relevance scoring.

Backend Search Implementation

The knowledge_catalog_search function creates a CatalogServiceClient targeting dataplex.googleapis.com. It constructs the parent resource name as projects/<consumer-project>/locations/global (retrieving the project ID via samples/discovery/utils.py), then invokes search_entries with semantic search enabled:

def knowledge_catalog_search(query: str) -> dict:
    client = CatalogServiceClient()
    parent_name = f"projects/{get_consumer_project()}/locations/global"
    
    response = client.search_entries(
        request={
            "name": parent_name,
            "query": query,
            "page_size": 50,
            "semantic_search": True,
        }
    )
    # ...

Setting semantic_search=True instructs Dataplex to interpret the query using vector-based semantic matching rather than simple keyword filtering.

Result Extraction and Default Ordering

The tool extracts a lightweight dictionary for each result (lines 47-55 in samples/discovery/tools.py), capturing entry_name, system, resource_id, and display_name. The function preserves the exact ordering returned by Dataplex, which already contains relevance scores computed by Dataplex's internal ranking engine. No additional client-side reranking occurs in the default implementation.

Implementing Custom Reranking Logic

While the default implementation passes Dataplex results through unchanged, the tool structure supports custom augmentation.

Extension Points in the Tool Function

Developers can insert reranking logic immediately after the list comprehension in knowledge_catalog_search (after line 55). Potential customizations include:

• Custom scoring: Inject business-specific relevance scores based on entry metadata
• Deduplication: Remove duplicate entries across different systems
• User-feedback loops: Reorder results based on historical click-through rates

Code Examples and Implementation

Invoke the Discovery Agent locally using the exported agent instance:

from samples.discovery.agent import discovery_agent

response = discovery_agent.run(
    {"query": "Find all BigQuery tables that contain sales data for 2023"}
)
print(response)

To implement custom reranking, modify samples/discovery/tools.py to adjust the result ordering before returning:

def knowledge_catalog_search(query: str) -> dict:
    # ... existing Dataplex call ...

    entries = [
        {
            "entry_name": r.dataplex_entry.name,
            "system": r.dataplex_entry.entry_source.system,
            "resource_id": r.dataplex_entry.entry_source.resource,
            "display_name": r.dataplex_entry.entry_source.display_name,
            "score": getattr(r, "ranking_score", 0),
        }
        for r in response.results
    ]
    
    # Custom rerank: boost entries with "2023" in display name

    entries.sort(
        key=lambda e: (e["score"], "2023" in e["display_name"]),
        reverse=True,
    )
    return {"results": entries}

The implementation also includes error handling that distinguishes permission errors (returned with clear messaging) from generic exceptions (logged and returned as errors).

Summary

• The Discovery Agent in samples/discovery/agent.py uses Gemini-3 Flash to interpret user intent but forwards the exact query text to Dataplex without rewriting it.
• Query generation relies on Dataplex's semantic_search=True parameter to perform vector-based natural language matching.
• Result reranking is delegated to Dataplex's internal ranking engine; the agent preserves the returned ordering.
• The knowledge_catalog_search tool in samples/discovery/tools.py provides a clean extension point for custom reranking logic after result extraction.
• The agent targets dataplex.googleapis.com with a parent resource pattern of projects/<consumer-project>/locations/global.

Frequently Asked Questions

Does the Discovery Agent rewrite user queries before searching?

No. According to the implementation in samples/discovery/agent.py, the agent passes the user's exact input text directly to the knowledge_catalog_search tool. The semantic interpretation happens within Dataplex's search_entries API when called with semantic_search=True, not through LLM-based query rewriting.

How does the agent handle the ranking of search results?

The agent relies entirely on Dataplex's built-in ranking engine. The search_entries method returns results already ordered by relevance scores computed by Google's backend infrastructure. The knowledge_catalog_search function extracts entry metadata and returns the list in the same order without applying additional client-side scoring.

Where can I modify the result ordering in the Knowledge Catalog Discovery Agent?

You can customize reranking in samples/discovery/tools.py immediately after the list comprehension that extracts entry data (around lines 47-55). This location allows you to access the full Dataplex response objects, apply custom scoring algorithms, and reorder the entries list before returning it to the agent.

What model does the Discovery Agent use for query understanding?

The agent uses the Gemini-3 Flash model (GEMINI_MODEL), configured in samples/discovery/agent.py. This model powers the LLM agent's ability to determine when to invoke the search tool and how to present results, while the actual data retrieval depends on the Dataplex Catalog API.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}