# How to Manage Prompts in the Agent Platform: A Complete Guide to Inference and Fine-Tuning

> Learn to manage prompts in Agent Platform for inference and fine-tuning. Master structured messages, raw strings, and JSON-Lines datasets for efficient prompt handling.

- Repository: [Google/skills](https://github.com/google/skills)
- Tags: how-to-guide
- Published: 2026-09-05

---

**The Agent Platform expects prompts in a structured *messages* format—containing role-content pairs for user, assistant, and system—or as raw strings for single-turn requests, with fine-tuning workflows requiring validated JSON-Lines datasets.**

Managing prompts effectively in the Agent Platform (formerly Gemini Enterprise Agent Platform) requires understanding its conversational message schema and dataset preparation utilities. The platform, maintained in the `google/skills` repository, provides specific SDK patterns for runtime inference and dedicated Python scripts for formatting and validating training data. This guide explains the end-to-end prompt lifecycle, from generating single-turn responses to preparing multi-turn datasets for supervised fine-tuning.

## Understanding the Agent Platform Prompt Format

The Agent Platform processes prompts using two distinct patterns depending on your use case.

**Message-based prompting** is the preferred format for production workloads. Each prompt is a JSON object containing a `role` (either `user`, `assistant`, or `system`) and a `content` string. When you pass a list of these objects to `generate_content`, the model maintains conversational context across turns and respects system-level instructions.

**Single-turn prompting** accepts a raw text string for straightforward queries. The SDK internally wraps this string as a single user message before sending it to the model endpoint, making it suitable for stateless requests that do not require conversation history.

## Sending Prompts to the Agent Platform

### Single-Turn Inference with the Vertex AI SDK

For simple, one-off requests, initialize the Vertex AI client and call `generate_content` with a plain string. The [`openmaas_vertexai_sdk.py`](https://github.com/google/skills/blob/main/openmaas_vertexai_sdk.py) script in the repository demonstrates this pattern for the OpenMaaS wrapper:

```python
import google.auth
import vertexai
from vertexai.generative_models import GenerativeModel

# Initialize the client (project inferred from ADC)

_, project_id = google.auth.default()
vertexai.init(project=project_id, location="global")

# Build the model reference and send a single-turn prompt

model = GenerativeModel("publishers/zai-org/models/glm-5-maas")
response = model.generate_content("Explain quantum computing.")
print(response.text)

```

This approach leverages the `GenerativeModel` class from `vertexai.generative_models` and automatically handles authentication via Application Default Credentials (ADC).

### Multi-Turn Conversations with Message History

To maintain context across multiple exchanges, construct a messages list and pass it to the `messages` parameter. This pattern, supported by both the Gemini and OpenMaaS SDKs, enables system-prompt injection and conversational memory:

```python
from vertexai.generative_models import GenerativeModel

model = GenerativeModel("publishers/google/models/gemini-1.5-pro")
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the capital of France?"},
]

response = model.generate_content(messages=messages)
print(response.text)  # → "Paris."

```

The platform appends the model's reply as an `assistant` message to the history, allowing you to pass the updated list back for subsequent turns.

## Preparing Prompts for Fine-Tuning and Evaluation

### Converting Tabular Data to JSON-Lines

Fine-tuning jobs require datasets in **JSON-Lines** (`.jsonl`) format where each line represents one training example. The [`prepare_dataset.py`](https://github.com/google/skills/blob/main/prepare_dataset.py) script located at [`skills/cloud/agent-platform-tuning/scripts/prepare_dataset.py`](https://github.com/google/skills/blob/main/skills/cloud/agent-platform-tuning/scripts/prepare_dataset.py) automates conversion from CSV, JSON, or Parquet files.

The script filters out rows with empty or NaN values in the prompt or completion columns, then writes each record using either the `messages` schema or a simple `prompt`/`completion` pair schema:

```bash
python -m skills.cloud.agent_platform_tuning.scripts.prepare_dataset \
  --input my_data.csv \
  --output my_data.jsonl \
  --format_type messages \
  --prompt_col user_prompt \
  --completion_col model_answer

```

Behind the scenes, the `convert_to_jsonl` function uses `datasets.load_dataset` to read the input and produces lines following the messages format:

```json
{
  "messages": [
    {"role": "user", "content": "Explain quantum computing."},
    {"role": "assistant", "content": "Quantum computing ..."}
  ]
}

```

### Validating Dataset Format

Before launching a tuning job, validate your JSON-Lines file using [`validate_dataset.py`](https://github.com/google/skills/blob/main/validate_dataset.py) from the evaluation flywheel ([`skills/cloud/agent-platform-eval-flywheel/scripts/validate_dataset.py`](https://github.com/google/skills/blob/main/skills/cloud/agent-platform-eval-flywheel/scripts/validate_dataset.py)). This utility checks that each line contains the required fields—either a `messages` list or both `prompt` and `completion` keys—and that no values are empty:

```bash
python -m skills.cloud.agent_platform_eval_flywheel.scripts.validate_dataset \
  --input my_data.jsonl \
  --format_type messages

```

The validator outputs a summary indicating the count of valid versus invalid entries, allowing you to catch formatting errors before they cause job failures.

## Running Production Evaluation on Prompts

For batch evaluation of stored prompts against a deployed endpoint, the repository provides [`endpoint_evaluation.py`](https://github.com/google/skills/blob/main/endpoint_evaluation.py) ([`skills/cloud/agent-platform-eval-flywheel/scripts/endpoint_evaluation.py`](https://github.com/google/skills/blob/main/skills/cloud/agent-platform-eval-flywheel/scripts/endpoint_evaluation.py)). This script reads a dataset containing a `prompt` column, calls the endpoint for each entry, and stores responses in a new `response` column:

```python
import pandas as pd
from endpoint_evaluation import run_inference

df = pd.read_json("prompts.jsonl", lines=True)   # expects a 'prompt' column

df["response"] = df["prompt"].apply(
    lambda p: run_inference(p, endpoint_url="https://my-endpoint", token="YOUR_TOKEN")
)
df.to_json("responses.jsonl", orient="records", lines=True)

```

This pattern supports downstream metric calculation (such as BLEU or ROUGE scores) by pairing original prompts with model-generated outputs.

## Summary

- The Agent Platform accepts both raw strings for single-turn requests and structured **messages** lists (role-content pairs) for multi-turn conversations.
- Use `generate_content` from the Vertex AI SDK for runtime inference, passing either a string or a messages list depending on context requirements.
- Fine-tuning requires **JSON-Lines** format; use [`prepare_dataset.py`](https://github.com/google/skills/blob/main/prepare_dataset.py) to convert CSV/Parquet files and [`validate_dataset.py`](https://github.com/google/skills/blob/main/validate_dataset.py) to check for missing fields before job submission.
- For production evaluation, [`endpoint_evaluation.py`](https://github.com/google/skills/blob/main/endpoint_evaluation.py) provides a robust loop for batch-processing prompts against deployed endpoints.

## Frequently Asked Questions

### What format does the Agent Platform expect for prompts?

The platform accepts two formats: a raw string for single-turn requests, which the SDK wraps internally as `{"role": "user", "content": "<prompt>"}`, or a list of message objects with explicit `role` (user, assistant, or system) and `content` fields for multi-turn conversations.

### How do I convert my existing CSV dataset for Agent Platform fine-tuning?

Use the [`prepare_dataset.py`](https://github.com/google/skills/blob/main/prepare_dataset.py) script with `--format_type messages` and specify your column names using `--prompt_col` and `--completion_col`. This utility filters empty rows and outputs a `.jsonl` file where each line contains the properly structured messages array required by the tuning service.

### What validation steps should I run before submitting a tuning job?

Run [`validate_dataset.py`](https://github.com/google/skills/blob/main/validate_dataset.py) against your `.jsonl` file to ensure every line contains either a valid `messages` list or both `prompt` and `completion` fields with non-empty values. This prevents job failures caused by malformed training examples.

### Can I use raw strings instead of the messages format for multi-turn conversations?

While the SDK accepts raw strings for single-turn queries, multi-turn conversations require the messages format to maintain context. Passing a raw string resets the conversation history, whereas the messages list preserves previous turns and system instructions across calls to `generate_content`.