# Adding Evaluation Results to Model Cards Using Model-Index YAML: A Complete Guide

> Easily add evaluation results to Hugging Face model cards with model-index YAML. Automate benchmark scores from READMEs or API data using the evaluation manager CLI.

- Repository: [Hugging Face/skills](https://github.com/huggingface/skills)
- Tags: how-to-guide
- Published: 2026-03-08

---

**You can automatically populate Hugging Face model cards with structured benchmark scores by extracting markdown tables from READMEs or importing Artificial Analysis API data into the `model-index` YAML section using the [`evaluation_manager.py`](https://github.com/huggingface/skills/blob/main/evaluation_manager.py) CLI.**

The `hugging-face-evaluation` skill in the `huggingface/skills` repository automates the tedious process of formatting benchmark results according to the Papers with Code specification. This tool parses evaluation tables, detects their layout structure, and generates the required `model-index` metadata that powers the Hugging Face evaluation widget.

## How the Evaluation Skill Works

The skill is implemented as a standalone Python CLI in [`skills/hugging-face-evaluation/scripts/evaluation_manager.py`](https://github.com/huggingface/skills/blob/main/skills/hugging-face-evaluation/scripts/evaluation_manager.py). It uses `markdown-it-py` for robust table parsing and the `huggingface_hub` library to interact with model repositories.

### Core Components

- **Table Extraction**: The `extract_tables_with_parser()` function pulls raw markdown tables from a model's README using `markdown-it-py`, handling GFM tables, code fences, and links accurately.
- **Layout Classification**: `detect_table_format()`, `is_transposed_table()`, `find_main_model_column()`, and `find_main_model_row()` identify whether data is organized by rows, columns, or transposed, then locate the target model via exact-token matching in `normalize_model_name`.
- **Metric Extraction**: `extract_metrics_from_table()` converts parsed tables into metric dictionaries, supporting optional `--model-column-index` overrides for non-standard layouts.
- **Model-Index Generation**: `extract_evaluations_from_readme()` wraps metrics in the required YAML structure with `task`, `dataset`, `metrics`, and `source` fields.
- **Hub Integration**: `update_model_card_with_evaluations()` merges new results with existing entries (lines 10-21 of the function preserve earlier data) and pushes updates via `ModelCard.push_to_hub()`.

## Extracting Evaluation Results from README Tables

Before modifying any model card, inspect the tables to verify detection accuracy.

### Inspecting Tables Before Extraction

Run the `inspect-tables` command to see how the parser interprets your README:

```bash
uv run scripts/evaluation_manager.py inspect-tables \
  --repo-id "meta-llama/Llama-3.3-70B-Instruct"

```

This outputs each table's format, column indices, and which columns match the model name using the exact-token matching logic in `find_main_model_column()`.

### Dry-Run vs. Apply

Always perform a dry run first to validate the generated YAML structure:

```bash
uv run scripts/evaluation_manager.py extract-readme \
  --repo-id "your-username/your-model-7b" \
  --dry-run

```

The generated `model-index` YAML prints to stdout without modifying the repository. Once validated, apply the changes directly:

```bash
uv run scripts/evaluation_manager.py extract-readme \
  --repo-id "your-username/your-model-7b" \
  --apply

```

### Creating Pull Requests for Community Models

For repositories where you lack direct write access, use the `--create-pr` flag:

```bash
uv run scripts/evaluation_manager.py extract-readme \
  --repo-id "community/awesome-7b" \
  --create-pr

```

The script checks for open PRs via `list_open_prs()` before creating a new one with your evaluation updates.

## Importing from Artificial Analysis API

You can also import benchmarks from the Artificial Analysis API instead of parsing README tables. The `import_aa_evaluations()` function calls `get_aa_model_data()` and transforms the JSON response using `aa_data_to_model_index()`:

```bash
AA_API_KEY="my-aa-key" uv run scripts/evaluation_manager.py import-aa \
  --creator-slug "anthropic" \
  --model-name "claude-sonnet-4" \
  --repo-id "your-username/claude-mirror" \
  --create-pr

```

This bypasses table extraction entirely, converting API responses directly into the `model-index` format.

## Updating Model Cards Programmatically

For CI/CD pipelines or custom workflows, invoke the CLI via Python subprocess:

```python
from pathlib import Path
import subprocess, json, os

repo = "username/model"

# Generate model-index YAML

proc = subprocess.run(
    ["uv", "run", "scripts/evaluation_manager.py", "extract-readme",
     "--repo-id", repo, "--dry-run"],
    capture_output=True, text=True, check=True
)

yaml_output = proc.stdout
print("Generated model-index:\n", yaml_output)

# Push changes to hub

subprocess.run(
    ["uv", "run", "scripts/evaluation_manager.py", "extract-readme",
     "--repo-id", repo, "--apply"],
    check=True
)

```

The script header includes a `/// script` block that automatically installs dependencies (`huggingface_hub`, `markdown-it-py`, `pyyaml`, `requests`, `python-dotenv`), so no manual environment setup is required when using `uv run`.

## Summary

- The [`evaluation_manager.py`](https://github.com/huggingface/skills/blob/main/evaluation_manager.py) CLI in `huggingface/skills` automates adding evaluation results to model cards using the `model-index` YAML format.
- It supports two input sources: **README markdown tables** (parsed via `markdown-it-py`) and the **Artificial Analysis API**.
- Table layouts are automatically classified as rows, columns, or transposed, with exact-token matching locating the correct model entry.
- The merge logic in `update_model_card_with_evaluations()` preserves existing evaluations while appending new results.
- All dependencies are handled automatically via the script header when using `uv run`, enabling immediate execution without setup.

## Frequently Asked Questions

### What is the model-index format?

The `model-index` format is a YAML structure following the Papers with Code specification. It contains a `name` field and a `results` array where each entry specifies the `task`, `dataset`, `metrics` (with type and value), and `source`. The Hugging Face Hub reads this section to render the evaluation widget on model pages.

### How does the skill handle existing evaluations?

The skill never overwrites existing `model-index` entries. In `update_model_card_with_evaluations()` (specifically lines 10-21 of the implementation), new results are merged with existing data, ensuring that previously recorded benchmarks remain intact while additional scores are appended.

### Can I use this without installing dependencies manually?

Yes. The script uses a `/// script` header that declares required packages (`huggingface_hub`, `markdown-it-py`, `pyyaml`, `requests`, `python-dotenv`). When you run the script with `uv run`, these dependencies are installed automatically in an isolated environment without polluting your system Python.

### How does table layout detection work?

The skill uses `detect_table_format()` to classify tables as row-based, column-based, or transposed. It then applies `find_main_model_column()` or `find_main_model_row()` with `normalize_model_name()` to perform exact-token matching against the model identifier. You can override automatic detection using the `--model-column-index` parameter if the parser selects the wrong column.