# Handling Authentication for Pushing Models and Datasets to Hugging Face Hub

> Learn to handle Hugging Face Hub authentication for pushing models and datasets. Securely log in using tokens and environment variables for seamless uploads.

- Repository: [Hugging Face/skills](https://github.com/huggingface/skills)
- Tags: how-to-guide
- Published: 2026-03-08

---

**Authenticate with the Hugging Face Hub by retrieving your token from environment variables or cached credentials, then call `huggingface_hub.login()` to establish a persistent session before pushing models or datasets.**

The **huggingface/skills** repository centralizes production-grade patterns for handling authentication when pushing models and datasets to the Hugging Face Hub. These implementations demonstrate secure token management across local development, CI pipelines, and cloud job environments without hardcoding credentials.

## Three Authentication Mechanisms

The Skills repository implements three complementary approaches to Hugging Face Hub authentication, each suited to different execution contexts.

### 1. Explicit Programmatic Login with `huggingface_hub.login`

The most direct method uses `huggingface_hub.login(token=...)` to perform the same OAuth handshake as the CLI, caching credentials locally for subsequent API calls. In [`skills/hugging-face-model-trainer/scripts/unsloth_sft_example.py`](https://github.com/huggingface/skills/blob/main/skills/hugging-face-model-trainer/scripts/unsloth_sft_example.py) (lines 62‑70), the script reads `HF_TOKEN` or `hfjob` from the environment and establishes a session:

```python
from huggingface_hub import login
import os

token = os.getenv("HF_TOKEN") or os.getenv("hfjob")
login(token=token)

```

This creates the `~/.huggingface/token` file that the Hub SDK automatically detects for all future operations.

### 2. Automatic Token Discovery via `get_token()`

For environments where tokens may be pre-authenticated, `huggingface_hub.get_token()` provides robust fallback logic. The script at [`skills/hugging-face-jobs/scripts/generate-responses.py`](https://github.com/huggingface/skills/blob/main/skills/hugging-face-jobs/scripts/generate-responses.py) (lines 27‑39) demonstrates this pattern by checking CLI arguments, environment variables, and finally cached logins:

```python
from huggingface_hub import get_token, login

HF_TOKEN = args.hf_token or os.getenv("HF_TOKEN") or get_token()
login(token=HF_TOKEN)

```

This method reads from stored logins (e.g., from `huggingface-cli login`) or the `~/.huggingface/token` file, enabling seamless execution in CI runners and notebooks.

### 3. Direct Token Passing to Low-Level APIs

When working with private datasets or specialized integrations like DuckDB, tokens must be injected at the API level. The `HFDatasetSQL` class in [`skills/hugging-face-datasets/scripts/sql_manager.py`](https://github.com/huggingface/skills/blob/main/skills/hugging-face-datasets/scripts/sql_manager.py) (lines 54‑95) stores `self.token` initialized from `HF_TOKEN` and creates a DuckDB secret for private dataset access:

```sql
CREATE SECRET hf_token (TYPE HUGGINGFACE, TOKEN '...');

```

When calling `push_to_hub`, the class explicitly forwards the token (lines 71‑81):

```python
dataset.push_to_hub(..., token=self.token)

```

This guarantees authentication even when the global session cache is unavailable.

## Core Authentication Flow

According to the source code in `huggingface/skills`, the authentication logic follows a strict priority hierarchy:

1. **Environment variable priority** – All entry points first check for `HF_TOKEN` (or the legacy `hfjob` variable used by the Jobs platform).
2. **Fallback to stored login** – If no environment variable exists, `huggingface_hub.get_token()` reads the cached token from `~/.huggingface/token`.
3. **Explicit `login()` invocation** – The resolved token is passed to `login(token=...)`, creating a local credential file for automatic SDK detection.
4. **Token propagation** – High-level SDK calls like `Dataset.push_to_hub()` inherit the cached credential, while specialized managers (like the SQL handler) inject tokens directly into DuckDB secrets or API calls.

This layered approach ensures robust authentication across local development, cloud jobs, and containerized evaluations.

## Implementation Examples

### Programmatic Login Before Model Upload

Use this pattern when training and pushing models in scripts like [`unsloth_sft_example.py`](https://github.com/huggingface/skills/blob/main/unsloth_sft_example.py):

```python
import os
from huggingface_hub import login
from transformers import AutoModelForCausalLM, AutoTokenizer

# Resolve token from environment

token = os.getenv("HF_TOKEN") or os.getenv("hfjob")
if not token:
    raise RuntimeError("Set HF_TOKEN before pushing a model")

# Establish cached session

login(token=token)

# Push model and tokenizer

model = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer = AutoTokenizer.from_pretrained("gpt2")
model.push_to_hub("username/my-gpt2", token=token)
tokenizer.push_to_hub("username/my-gpt2", token=token)

```

### Automatic Token Discovery in Job Scripts

For reusable job infrastructure as implemented in [`generate-responses.py`](https://github.com/huggingface/skills/blob/main/generate-responses.py):

```python
import argparse
import os
import sys
from huggingface_hub import get_token, login

parser = argparse.ArgumentParser()
parser.add_argument("--hf-token", help="HF token (optional)")
args = parser.parse_args()

# Resolve token: CLI arg → env var → cached login

HF_TOKEN = args.hf_token or os.getenv("HF_TOKEN") or get_token()
if not HF_TOKEN:
    sys.exit("No HF token found – aborting")

login(token=HF_TOKEN)
print("Authenticated with Hub")

```

### Pushing Datasets with the SQL Manager

For SQL-driven workflows using the `HFDatasetSQL` class from [`sql_manager.py`](https://github.com/huggingface/skills/blob/main/sql_manager.py):

```python
from sql_manager import HFDatasetSQL

# Initialize with automatic HF_TOKEN detection

sql = HFDatasetSQL()

# Transform data via SQL

query = "SELECT subject, COUNT(*) AS cnt FROM data GROUP BY subject"
results = sql.query("cais/mmlu", query, split="train", output_format="df")

# Push to Hub with explicit authentication

sql.push_to_hub(
    dataset_id="cais/mmlu",
    target_repo="my-org/nutrition-subset",
    sql=query,
    private=False,
    commit_message="Create nutrition subset",
)

```

## Key Files and Authentication Patterns

| Path | Role | Implementation Detail |
|------|------|---------------------|
| [`skills/hugging-face-model-trainer/scripts/unsloth_sft_example.py`](https://github.com/huggingface/skills/blob/main/skills/hugging-face-model-trainer/scripts/unsloth_sft_example.py) | Model training authentication | Lines 62‑70: Env-var token retrieval + `login()` |
| [`skills/hugging-face-jobs/scripts/generate-responses.py`](https://github.com/huggingface/skills/blob/main/skills/hugging-face-jobs/scripts/generate-responses.py) | Job infrastructure | Lines 27‑39: `get_token()` fallback logic |
| [`skills/hugging-face-datasets/scripts/sql_manager.py`](https://github.com/huggingface/skills/blob/main/skills/hugging-face-datasets/scripts/sql_manager.py) | Dataset SQL integration | Lines 54‑95: DuckDB secret creation; lines 71‑81: `push_to_hub` with token |
| [`skills/hugging-face-paper-publisher/scripts/paper_manager.py`](https://github.com/huggingface/skills/blob/main/skills/hugging-face-paper-publisher/scripts/paper_manager.py) | Alternative token source | Uses `HfFolder.get_token()` for cached credentials |
| [`skills/hugging-face-evaluation/scripts/run_eval_job.py`](https://github.com/huggingface/skills/blob/main/skills/hugging-face-evaluation/scripts/run_eval_job.py) | Container job security | Propagates `--secrets HF_TOKEN=...` to evaluation containers |

## Summary

- **Store tokens securely** using the `HF_TOKEN` environment variable rather than hardcoding values in scripts.
- **Use `huggingface_hub.login()`** to establish a persistent session that caches credentials at `~/.huggingface/token` for subsequent API calls.
- **Implement fallback chains** with `get_token()` to support both interactive development (cached logins) and automated environments (CI/CD).
- **Pass tokens explicitly** when working with low-level integrations like DuckDB or private dataset operations to ensure authentication bypasses the global cache.
- **Reference the `huggingface/skills` repository** for production patterns that handle edge cases in cloud job platforms and containerized training.

## Frequently Asked Questions

### How do I authenticate without running `huggingface-cli login`?

Set the `HF_TOKEN` environment variable in your shell or CI configuration, then call `huggingface_hub.login(token=os.getenv("HF_TOKEN"))` at the start of your script. This programmatic approach mirrors the CLI authentication flow without requiring interactive terminal input.

### Why does my script fail to push private datasets even after logging in?

Some low-level integrations, such as the DuckDB-based SQL manager in [`sql_manager.py`](https://github.com/huggingface/skills/blob/main/sql_manager.py), require explicit token passing rather than relying on the global cache. Ensure you forward the token directly to `Dataset.push_to_hub(token=...)` or create the appropriate DuckDB secret using `CREATE SECRET hf_token`.

### What is the difference between `get_token()` and `HfFolder.get_token()`?

Both methods read the cached token from `~/.huggingface/token`, but `huggingface_hub.get_token()` is the modern, recommended API that handles additional edge cases and environment variable checks. The `HfFolder` class (used in [`paper_manager.py`](https://github.com/huggingface/skills/blob/main/paper_manager.py)) provides legacy access to the same storage mechanism.

### How do I handle authentication in containerized evaluation jobs?

Pass the token via container secrets as shown in [`run_eval_job.py`](https://github.com/huggingface/skills/blob/main/run_eval_job.py) and [`run_vllm_eval_job.py`](https://github.com/huggingface/skills/blob/main/run_vllm_eval_job.py). Use the `--secrets HF_TOKEN=...` pattern to inject the credential into the job environment, then retrieve it inside the container using `os.getenv("HF_TOKEN")` before calling `login()`.