How to Manage Prompts in the Agent Platform: A Complete Guide to Inference and Fine-Tuning

The Agent Platform expects prompts in a structured messages format—containing role-content pairs for user, assistant, and system—or as raw strings for single-turn requests, with fine-tuning workflows requiring validated JSON-Lines datasets.

Managing prompts effectively in the Agent Platform (formerly Gemini Enterprise Agent Platform) requires understanding its conversational message schema and dataset preparation utilities. The platform, maintained in the google/skills repository, provides specific SDK patterns for runtime inference and dedicated Python scripts for formatting and validating training data. This guide explains the end-to-end prompt lifecycle, from generating single-turn responses to preparing multi-turn datasets for supervised fine-tuning.

Understanding the Agent Platform Prompt Format

The Agent Platform processes prompts using two distinct patterns depending on your use case.

Message-based prompting is the preferred format for production workloads. Each prompt is a JSON object containing a role (either user, assistant, or system) and a content string. When you pass a list of these objects to generate_content, the model maintains conversational context across turns and respects system-level instructions.

Single-turn prompting accepts a raw text string for straightforward queries. The SDK internally wraps this string as a single user message before sending it to the model endpoint, making it suitable for stateless requests that do not require conversation history.

Sending Prompts to the Agent Platform

Single-Turn Inference with the Vertex AI SDK

For simple, one-off requests, initialize the Vertex AI client and call generate_content with a plain string. The openmaas_vertexai_sdk.py script in the repository demonstrates this pattern for the OpenMaaS wrapper:

import google.auth
import vertexai
from vertexai.generative_models import GenerativeModel

# Initialize the client (project inferred from ADC)

_, project_id = google.auth.default()
vertexai.init(project=project_id, location="global")

# Build the model reference and send a single-turn prompt

model = GenerativeModel("publishers/zai-org/models/glm-5-maas")
response = model.generate_content("Explain quantum computing.")
print(response.text)

This approach leverages the GenerativeModel class from vertexai.generative_models and automatically handles authentication via Application Default Credentials (ADC).

Multi-Turn Conversations with Message History

To maintain context across multiple exchanges, construct a messages list and pass it to the messages parameter. This pattern, supported by both the Gemini and OpenMaaS SDKs, enables system-prompt injection and conversational memory:

from vertexai.generative_models import GenerativeModel

model = GenerativeModel("publishers/google/models/gemini-1.5-pro")
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the capital of France?"},
]

response = model.generate_content(messages=messages)
print(response.text)  # → "Paris."

The platform appends the model's reply as an assistant message to the history, allowing you to pass the updated list back for subsequent turns.

Preparing Prompts for Fine-Tuning and Evaluation

Converting Tabular Data to JSON-Lines

Fine-tuning jobs require datasets in JSON-Lines (.jsonl) format where each line represents one training example. The prepare_dataset.py script located at skills/cloud/agent-platform-tuning/scripts/prepare_dataset.py automates conversion from CSV, JSON, or Parquet files.

The script filters out rows with empty or NaN values in the prompt or completion columns, then writes each record using either the messages schema or a simple prompt/completion pair schema:

python -m skills.cloud.agent_platform_tuning.scripts.prepare_dataset \
  --input my_data.csv \
  --output my_data.jsonl \
  --format_type messages \
  --prompt_col user_prompt \
  --completion_col model_answer

Behind the scenes, the convert_to_jsonl function uses datasets.load_dataset to read the input and produces lines following the messages format:

{
  "messages": [
    {"role": "user", "content": "Explain quantum computing."},
    {"role": "assistant", "content": "Quantum computing ..."}
  ]
}

Validating Dataset Format

Before launching a tuning job, validate your JSON-Lines file using validate_dataset.py from the evaluation flywheel (skills/cloud/agent-platform-eval-flywheel/scripts/validate_dataset.py). This utility checks that each line contains the required fields—either a messages list or both prompt and completion keys—and that no values are empty:

python -m skills.cloud.agent_platform_eval_flywheel.scripts.validate_dataset \
  --input my_data.jsonl \
  --format_type messages

The validator outputs a summary indicating the count of valid versus invalid entries, allowing you to catch formatting errors before they cause job failures.

Running Production Evaluation on Prompts

For batch evaluation of stored prompts against a deployed endpoint, the repository provides endpoint_evaluation.py (skills/cloud/agent-platform-eval-flywheel/scripts/endpoint_evaluation.py). This script reads a dataset containing a prompt column, calls the endpoint for each entry, and stores responses in a new response column:

import pandas as pd
from endpoint_evaluation import run_inference

df = pd.read_json("prompts.jsonl", lines=True)   # expects a 'prompt' column

df["response"] = df["prompt"].apply(
    lambda p: run_inference(p, endpoint_url="https://my-endpoint", token="YOUR_TOKEN")
)
df.to_json("responses.jsonl", orient="records", lines=True)

This pattern supports downstream metric calculation (such as BLEU or ROUGE scores) by pairing original prompts with model-generated outputs.

Summary

• The Agent Platform accepts both raw strings for single-turn requests and structured messages lists (role-content pairs) for multi-turn conversations.
• Use generate_content from the Vertex AI SDK for runtime inference, passing either a string or a messages list depending on context requirements.
• Fine-tuning requires JSON-Lines format; use prepare_dataset.py to convert CSV/Parquet files and validate_dataset.py to check for missing fields before job submission.
• For production evaluation, endpoint_evaluation.py provides a robust loop for batch-processing prompts against deployed endpoints.

Frequently Asked Questions

What format does the Agent Platform expect for prompts?

The platform accepts two formats: a raw string for single-turn requests, which the SDK wraps internally as {"role": "user", "content": "<prompt>"}, or a list of message objects with explicit role (user, assistant, or system) and content fields for multi-turn conversations.

How do I convert my existing CSV dataset for Agent Platform fine-tuning?

Use the prepare_dataset.py script with --format_type messages and specify your column names using --prompt_col and --completion_col. This utility filters empty rows and outputs a .jsonl file where each line contains the properly structured messages array required by the tuning service.

What validation steps should I run before submitting a tuning job?

Run validate_dataset.py against your .jsonl file to ensure every line contains either a valid messages list or both prompt and completion fields with non-empty values. This prevents job failures caused by malformed training examples.

Can I use raw strings instead of the messages format for multi-turn conversations?

While the SDK accepts raw strings for single-turn queries, multi-turn conversations require the messages format to maintain context. Passing a raw string resets the conversation history, whereas the messages list preserves previous turns and system instructions across calls to generate_content.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}