# Importing Benchmark Scores from Artificial Analysis API to Hugging Face Model Cards

> Automate importing benchmark scores from Artificial Analysis API to Hugging Face model cards. Keep leaderboards updated and showcase your model performance effortlessly with the hugging-face-evaluation skill.

- Repository: [Hugging Face/skills](https://github.com/huggingface/skills)
- Tags: how-to-guide
- Published: 2026-03-08

---

**The hugging-face-evaluation skill automates the process of fetching benchmark results from the Artificial Analysis API and embedding them into your model's [`model-index.yml`](https://github.com/huggingface/skills/blob/main/model-index.yml) metadata, enabling automatic updates to Hugging Face model card leaderboards.**

The huggingface/skills repository includes a dedicated evaluation skill that streamlines importing benchmark scores from Artificial Analysis into model cards. This workflow eliminates manual data entry by programmatically retrieving performance metrics, transforming them into the Hugging Face model-index format, and optionally opening pull requests to update your repository.

## How the Automated Import Works

The import workflow orchestrates data flow from the Artificial Analysis API to your Hugging Face model repository through five distinct stages. All logic resides in [`skills/hugging-face-evaluation/scripts/evaluation_manager.py`](https://github.com/huggingface/skills/blob/main/skills/hugging-face-evaluation/scripts/evaluation_manager.py).

### 1. Authentication Configuration

The system reads your Artificial Analysis API key from the `AA_API_KEY` environment variable. As documented in `examples/.env.example`, you must set this variable before running any import commands. The script passes this key in the `Authorization` header when making HTTP requests to the Artificial Analysis endpoint.

### 2. Data Retrieval from Artificial Analysis

The `get_aa_model_data(creator_slug, model_name)` function constructs the API endpoint `https://api.artificialanalysis.ai/v1/models/{creator_slug}/{model_name}` and uses the **requests** library to fetch the JSON payload containing benchmark scores. This function, located at lines 853-862 of [`evaluation_manager.py`](https://github.com/huggingface/skills/blob/main/evaluation_manager.py), handles network errors, 404 responses, and non-200 status codes by returning `None` and logging clear error messages.

### 3. Schema Transformation

The `aa_data_to_model_index()` function (lines 893-914) converts the raw Artificial Analysis JSON into the Hugging Face model-index schema. It generates a structured dictionary containing `name`, `description`, `datasets`, `metrics`, and `results` fields. The function specifically sets the `dataset_name` to "Artificial Analysis Benchmarks" and injects a `source` entry pointing back to the Artificial Analysis website for traceability.

### 4. Repository Update and PR Creation

The `import_aa_evaluations()` function (lines 943-966) loads any existing [`model-index.yml`](https://github.com/huggingface/skills/blob/main/model-index.yml) from your target repository via `huggingface_hub`, merges the new evaluation entry while preserving existing data, and writes the updated file to a temporary branch. When the `--create-pr` flag is used, the function automatically opens a pull request with a descriptive commit message such as "Add Artificial Analysis evaluations for [model-name]".

### 5. Duplicate Prevention

Before creating any pull request, the skill executes the `get-prs` sub-command to verify that no open PR already contains the same evaluation. This safety check, emphasized in the SKILL documentation, prevents duplicate efforts and repository clutter.

## Core Implementation Functions

Three primary functions in [`evaluation_manager.py`](https://github.com/huggingface/skills/blob/main/evaluation_manager.py) handle the end-to-end integration:

- **`get_aa_model_data`** (lines 853-862): Calls the Artificial Analysis API and returns raw JSON or `None` on failure
- **`aa_data_to_model_index`** (lines 893-914): Transforms API payloads into the Hugging Face model-index format
- **`import_aa_evaluations`** (lines 943-966): Drives the complete import workflow, handles merging, and manages PR creation

## Prerequisites and Dependencies

The skill requires specific Python packages declared in the skill's dependency list:

```bash
uv pip install requests>=2.32.5 huggingface_hub>=0.26.0 python-dotenv

```

You must configure the `AA_API_KEY` environment variable by creating a `.env` file in your working directory:

```bash
echo "AA_API_KEY=your_api_key_here" > .env

```

## Usage Examples

Run the import directly using the evaluation manager CLI:

```bash
uv run scripts/evaluation_manager.py import-aa \
    --repo-id "username/model-name" \
    --creator-slug "username" \
    --model-name "model-name" \
    --create-pr

```

Alternatively, use the provided example script at [`skills/hugging-face-evaluation/examples/artificial_analysis_to_hub.py`](https://github.com/huggingface/skills/blob/main/skills/hugging-face-evaluation/examples/artificial_analysis_to_hub.py):

```bash
uv run skills/hugging-face-evaluation/examples/artificial_analysis_to_hub.py \
    --repo-id "username/model-name" \
    --creator-slug "username" \
    --model-name "model-name"

```

Both methods support the `--create-pr` flag to automatically open pull requests with properly formatted descriptions linking back to the Artificial Analysis dashboard.

## Key Architecture Features

**Modular CLI Design**: The [`evaluation_manager.py`](https://github.com/huggingface/skills/blob/main/evaluation_manager.py) script uses `argparse` to expose sub-commands (`get-prs`, `inspect-tables`, `extract-readme`, `import-aa`), making the Artificial Analysis integration composable with other evaluation workflows like vLLM or lighteval.

**Robust Error Handling**: Network failures, API errors, and missing models are caught and logged without aborting the entire workflow, allowing batch operations to continue processing remaining models.

**Idempotent Operations**: The duplicate PR check ensures repeated runs are safe and do not flood model repositories with redundant pull requests.

## Summary

- The hugging-face-evaluation skill automates importing Artificial Analysis benchmark scores into Hugging Face model cards via the `import-aa` CLI command.
- The workflow requires setting the `AA_API_KEY` environment variable and uses three core functions in [`evaluation_manager.py`](https://github.com/huggingface/skills/blob/main/evaluation_manager.py) to fetch, transform, and merge data.
- Raw API data converts to the Hugging Face model-index format with proper attribution to Artificial Analysis.
- The system prevents duplicate PRs by checking existing open pull requests before creating new ones.
- Both direct CLI usage and example scripts are available in the `skills/hugging-face-evaluation` directory.

## Frequently Asked Questions

### How do I obtain an Artificial Analysis API key?

You must sign up for an account at Artificial Analysis and generate an API key from your dashboard. Store this key in the `AA_API_KEY` environment variable or a `.env` file in your project root, as shown in the `examples/.env.example` file within the huggingface/skills repository.

### What happens if the model doesn't exist in the Artificial Analysis database?

The `get_aa_model_data` function returns `None` and logs an error message when it encounters a 404 response or any non-200 status code. The workflow skips that specific model and continues processing any remaining items in your batch, ensuring network issues don't crash the entire import operation.

### Can I run this import without creating a pull request?

Yes. Omit the `--create-pr` flag when running the `import-aa` command. The script will still fetch and transform the data, displaying the results locally without modifying your Hugging Face repository or opening any pull requests.

### Which file in my repository gets modified by this skill?

The skill updates your repository's [`model-index.yml`](https://github.com/huggingface/skills/blob/main/model-index.yml) file, which stores the structured benchmark metadata that powers Hugging Face model card leaderboards. If this file doesn't exist, the skill creates it; if it exists, it merges new Artificial Analysis entries with your existing evaluation data.