# How to Implement Scorecard Validation for Agent Output Quality

> Implement scorecard validation for agent output quality using the validateScorecard function. Enforce schema compliance and get detailed feedback on validation failures.

- Repository: [Anthropic/cwc-workshops](https://github.com/anthropics/cwc-workshops)
- Tags: how-to-guide
- Published: 2026-07-21

---

**Use the `validateScorecard` function in [`research-desk/src/lib/scorecard.ts`](https://github.com/anthropics/cwc-workshops/blob/main/research-desk/src/lib/scorecard.ts) to enforce strict schema compliance on agent outputs, returning either a sanitized `Scorecard` object or a detailed `problems` array when validation fails.**

The `anthropics/cwc-workshops` repository demonstrates a robust pattern for implementing scorecard validation for agent output quality, ensuring that AI-generated financial analysis sessions emit predictable data structures. This validation layer acts as a critical gate between raw agent outputs and downstream consumers such as CSV exporters, synthesis engines, and analytics dashboards.

## The Core Validation Function

The validation logic lives in **[`research-desk/src/lib/scorecard.ts`](https://github.com/anthropics/cwc-workshops/blob/main/research-desk/src/lib/scorecard.ts)** within the `validateScorecard` function. This utility enforces three fundamental guarantees on incoming payloads:

1. **Known fields only** – The function accepts required fields as mandatory, permits optional fields, and silently strips any extraneous keys.
2. **Typed and bounded values** – Strings truncate to `MAX_SHORT_TEXT` (300 characters), numeric fields must be finite numbers, booleans remain strictly boolean, and list fields cap at `MAX_LIST_ITEMS` (5 entries).
3. **Enumerated constraints** – Specific fields must match predefined allowed values.

According to the source code at lines 75–76, the function first guards against invalid payload types by verifying the input is a non-array object before proceeding with field-level validation.

## Schema Constraints and Type Enforcement

The validation engine performs field-by-field cleaning through a loop over `REQUIRED_FIELDS` and `OPTIONAL_FIELDS` (lines 81–112). This process applies several defensive transformations:

- **String truncation** – Any text field exceeding 300 characters is sliced to prevent storage overflow.
- **List capping** – Arrays containing more than 5 items are truncated, with each remaining entry also subject to length limits.
- **Type coercion protection** – Numeric fields reject non-finite values (`NaN`, `Infinity`), ensuring downstream mathematical operations remain safe.

If a required field remains absent after the cleaning pass, the function immediately aborts and returns `scorecard: null` alongside a descriptive `problems` array (lines 21–23).

## Enumerated Value Validation

The scorecard schema restricts specific categorical fields to enumerated values defined at lines 14–19. The validation logic uses array inclusion checks to enforce these constraints:

- **`guidance_tone`** must be one of: `["positive", "neutral", "cautious", "none"]`
- **`confidence`** must be one of: `["low", "medium", "high"]`

When an agent emits an unrecognized value for either field, `validateScorecard` captures the violation in the `problems` array and rejects the entire payload, preventing invalid categorical data from polluting analytics datasets.

## Handling Validation Results

The function returns a discriminated union pattern that simplifies error handling in TypeScript:

```typescript
const { scorecard, problems } = validateScorecard(rawPayload);

if (scorecard) {
  // Proceed with downstream processing
  forwardToSynthesis(scorecard);
} else {
  // Handle validation failures
  throw new Error(`Scorecard validation failed: ${problems.join("; ")}`);
}

```

This pattern ensures that only conforming data reaches CSV rendering functions like `scorecardsToCsv`, while providing granular feedback for debugging agent prompt engineering issues.

## Testing the Validation Logic

The test suite in **[`research-desk/tests/scorecard.test.ts`](https://github.com/anthropics/cwc-workshops/blob/main/research-desk/tests/scorecard.test.ts)** (lines 28–51) codifies the expected behavior through comprehensive scenarios:

- Accepting fully populated, correct scorecards
- Rejecting payloads with missing required fields
- Flagging invalid enum values and type mismatches
- Enforcing list-length caps and string truncation limits

These tests serve as executable documentation for the validation contract, ensuring that schema changes maintain backward compatibility or explicitly update the test expectations.

## Integration Patterns

### CSV Export Pipeline

Once validated, scorecards integrate seamlessly with export utilities:

```typescript
import { scorecardsToCsv } from "research-desk/src/lib/scorecard";

if (scorecard) {
  const csv = scorecardsToCsv([scorecard]);
  // Returns header row plus single data row
  console.log(csv);
}

```

### Agent Pipeline Error Handling

For production agent orchestration, wrap validation in a retry-aware handler:

```typescript
function processAgentOutput(raw: unknown) {
  const { scorecard, problems } = validateScorecard(raw);
  if (!scorecard) {
    // Surface to orchestrator for retry logic or incident logging
    throw new Error(`Validation failed: ${problems.join("; ")}`);
  }
  return scorecard;
}

```

## Summary

- **Validate early** – Use `validateScorecard` from [`research-desk/src/lib/scorecard.ts`](https://github.com/anthropics/cwc-workshops/blob/main/research-desk/src/lib/scorecard.ts) immediately after agent output generation to catch schema violations before downstream processing.
- **Enforce boundaries** – Leverage built-in constants (`MAX_SHORT_TEXT`, `MAX_LIST_ITEMS`) to prevent data overflow and ensure consistent field sizes.
- **Handle failures explicitly** – Check for `scorecard: null` and inspect the `problems` array to debug agent prompt issues or data quality problems.
- **Test thoroughly** – Reference [`research-desk/tests/scorecard.test.ts`](https://github.com/anthropics/cwc-workshops/blob/main/research-desk/tests/scorecard.test.ts) for validation scenarios covering enums, types, and list constraints.

## Frequently Asked Questions

### What happens if the agent output contains extra fields not defined in the schema?

The `validateScorecard` function ignores any keys not listed in `REQUIRED_FIELDS` or `OPTIONAL_FIELDS`, effectively stripping extraneous data during the cleaning pass at lines 81–112. This prevents schema pollution while maintaining forward compatibility.

### How does the validation handle list fields that exceed the maximum length?

Lists are truncated to `MAX_LIST_ITEMS` (5 entries), with each retained entry also subject to `MAX_SHORT_TEXT` truncation (300 characters). This occurs during the field-by-field loop in [`research-desk/src/lib/scorecard.ts`](https://github.com/anthropics/cwc-workshops/blob/main/research-desk/src/lib/scorecard.ts), ensuring storage and display constraints remain satisfied.

### Can the validation function handle non-object inputs gracefully?

Yes. At lines 75–76, the function checks whether the payload is a non-array object using `typeof` checks. If the input is an array, null, or primitive, validation immediately fails and returns `scorecard: null` with an appropriate error message in the `problems` array.

### What are the valid values for guidance_tone and confidence fields?

According to the constants defined at lines 14–19, `guidance_tone` accepts `"positive"`, `"neutral"`, `"cautious"`, or `"none"`, while `confidence` accepts `"low"`, `"medium"`, or `"high"`. Any other values trigger validation failures captured in the returned `problems` array.