Supported Template Formats for Dataset Creation in Hugging Face Skills

The huggingface/skills repository supports six built-in JSON template formats—chat, classification, qa, completion, tabular, and custom—that define validation schemas, system prompts, and example structures for creating Hugging Face datasets via the CLI or Python API.

The huggingface/skills repository provides a flexible, template-driven workflow for building and managing Hugging Face datasets through the hugging-face-datasets skill. All dataset creation commands rely on JSON template descriptors that specify the logical dataset shape, validation rules, and example structures. Understanding these supported template formats is essential for correctly structuring your data before uploading to the Hub.

What Are Dataset Templates?

Dataset templates are JSON configuration files stored in skills/hugging-face-datasets/templates/ that act as blueprints for dataset creation. Each template defines five core components:

• type – The logical dataset shape (e.g., chat, classification).
• system_prompt – A guiding prompt used when the skill auto-generates examples.
• validation_schema – Required fields, data types, and constraints that the manager validates before uploading.
• example_structure – The expected JSON layout for a single dataset row.
• examples – Pre-built sample rows injected during quick_setup.

The manager loads these templates via load_dataset_template() in skills/hugging-face-datasets/scripts/dataset_manager.py and validates incoming rows against the schema using validate_by_template().

Supported Template Formats

The repository ships with six built-in templates covering the majority of machine learning dataset paradigms. You can also extend functionality by adding new JSON files to the templates directory.

Chat Template

File: templates/chat.json
Type: chat
Use case: Multi-turn conversational data, tool-call interactions, and dialogue systems.

This template structures data as lists of messages with roles (system, user, assistant) and supports optional tool definitions. It is ideal for fine-tuning instruction-following models.

Classification Template

File: templates/classification.json
Type: classification
Use case: Single-sentence or short-text classification tasks such as sentiment analysis, intent detection, or topic labeling.

The validation schema enforces a text field and a label field, with optional support for multi-label configurations.

QA Template

File: templates/qa.json
Type: qa
Use case: Question-answer pairs for reading comprehension, factual QA, or closed-book question answering.

This template expects a question, an answer, and an optional context field for extractive QA scenarios.

Completion Template

File: templates/completion.json
Type: completion
Use case: Prompt-completion pairs for language model pre-training or fine-tuning on code, creative writing, or technical documentation.

The structure is optimized for causal language modeling tasks where the model predicts the continuation of a given prefix.

Tabular Template

File: templates/tabular.json
Type: tabular
Use case: Structured tables for regression, classification, or clustering tasks on numerical or categorical features.

This template supports arbitrary column definitions through a flexible schema that validates data types per column.

Custom Template

File: templates/custom.json
Type: custom
Use case: Arbitrary user-defined schemas when none of the built-in templates match your data structure.

This provides full flexibility, allowing you to define custom validation_schema and example_structure fields while still leveraging the skill's validation and upload infrastructure.

How Templates Work Under the Hood

When you invoke a dataset creation command, the system performs three core operations defined in dataset_manager.py:

1. Template Loading: The load_dataset_template(template_name) function reads the corresponding JSON file from skills/hugging-face-datasets/templates/ and parses the schema.

2. Validation: The validate_by_template(rows, template) function checks each incoming row against the validation_schema defined in the template, ensuring required fields exist and types match before upload.

3. Repository Management: Functions like init_dataset(), define_config(), and add_rows() handle Hugging Face Hub interactions, using the template's system_prompt and examples to initialize the dataset configuration.

You can specify templates via the CLI using --template (for add_rows) or --template_type (for quick_setup).

Using Templates: CLI Examples

List Available Templates

To see all supported template formats and their metadata:

python scripts/dataset_manager.py list_templates

This command scans the templates/ directory and prints each template's name, type, description, and required fields.

Quick Setup with a Template

Create a new dataset pre-populated with template examples:

# Create a new classification dataset

uv run scripts/dataset_manager.py quick_setup \
    --repo_id your-username/my-classification-dataset \
    --template_type classification

Under the hood, quick_setup() loads classification.json, initializes the repository, stores the system prompt in config.json, and uploads the built-in example rows.

Add Rows to an Existing Dataset

Upload custom data validated against a specific template:

# Prepare a JSONL file locally (rows.jsonl)

uv run scripts/dataset_manager.py add_rows \
    --repo_id your-username/my-tabular-dataset \
    --split train \
    --template tabular \
    --rows_json "$(cat rows.jsonl)"

The manager validates each row against the validation_schema in tabular.json before uploading a newline-delimited JSON file to data/train-<timestamp>.jsonl on the Hub.

Programmatic Validation with Python

You can validate data programmatically without uploading using the Python API exposed in dataset_manager.py:

from scripts.dataset_manager import validate_training_data

rows = [
    {"question": "What is 2+2?", "answer": "4", "context": "Simple arithmetic"},
    {"question": "Who wrote '1984'?", "answer": "George Orwell"},
]

# Validate against the QA template

is_valid = validate_training_data(rows, template_name="qa")
print(is_valid)   # → True if schema matches

This function is used internally by add_rows but can be called directly for preprocessing pipelines or CI/CD validation checks.

Summary

• The huggingface/skills repository provides six built-in template formats for dataset creation: chat, classification, qa, completion, tabular, and custom.
• Templates are JSON descriptors stored in skills/hugging-face-datasets/templates/ that define validation schemas, system prompts, and example structures.
• The dataset_manager.py script handles template loading via load_dataset_template() and validation via validate_by_template().
• Use --template or --template_type CLI arguments to specify formats when running quick_setup or add_rows commands.
• Extend supported formats by adding new JSON template files to the templates directory; the CLI automatically discovers them.

Frequently Asked Questions

What is the difference between the chat and completion templates?

The chat template is designed for multi-turn conversational data with structured message roles (system, user, assistant) and optional tool definitions, making it ideal for instruction-following models. The completion template uses a simpler prompt-completion pair structure optimized for causal language modeling where the model predicts text continuations, suitable for code generation or creative writing tasks.

Can I create my own custom template format?

Yes. You can create a custom template by adding a new JSON file to skills/hugging-face-datasets/templates/ following the standard schema structure with type, system_prompt, validation_schema, example_structure, and examples fields. The list_templates command automatically discovers new templates, and the custom template type provides immediate flexibility for arbitrary schemas without modifying source code.

How does template validation prevent upload errors?

The validate_by_template() function in dataset_manager.py checks each row against the validation_schema defined in the template JSON before any Hub upload occurs. It verifies that required fields exist, data types match specifications (strings, numbers, arrays), and structural constraints are met. If validation fails, the CLI returns specific error messages indicating which fields are missing or malformed, preventing corrupted data from reaching your dataset repository.

Where are the template files located in the repository?

All template JSON files are stored in the skills/hugging-face-datasets/templates/ directory within the huggingface/skills repository. This directory contains chat.json, classification.json, qa.json, completion.json, tabular.json, and custom.json. The dataset_manager.py script references this path when executing load_dataset_template() to retrieve schema definitions during dataset creation operations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}