# What Parameters Does WhichLLM Extract from HuggingFace Models?

> Discover what parameters WhichLLM extracts from HuggingFace models, including technical specs, licensing, popularity, quantization, and benchmarks. Access detailed ModelInfo.

- Repository: [andy/whichllm](https://github.com/Andyyyy64/whichllm)
- Tags: deep-dive
- Published: 2026-06-10

---

**WhichLLM extracts 15+ distinct metadata parameters from every HuggingFace model, including technical specs, licensing, popularity metrics, quantization variants, and benchmark scores, storing them in a structured `ModelInfo` dataclass.**

The open-source **WhichLLM** repository performs deep metadata extraction from the HuggingFace Hub to power its LLM ranking and compatibility engine. Each model fetched through the `fetch_models` API undergoes parsing in [`src/whichllm/models/fetcher.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/models/fetcher.py), where specialized extraction functions resolve technical specifications, MoE configurations, and evaluation data into a strongly-typed `ModelInfo` object defined in [`src/whichllm/models/types.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/models/types.py).

## Core Identification Parameters

WhichLLM establishes model identity through three primary identifiers extracted in the `_parse_model` function (lines 44–45, 60 of [`fetcher.py`](https://github.com/Andyyyy64/whichllm/blob/main/fetcher.py)):

- **Model ID** (`id`): The canonical repository identifier (e.g., `meta-llama/Llama-3.2-8B`)
- **Family ID** (`family_id`): Initially matches the model ID, later grouped by the `grouper` engine
- **Name** (`name`): A short display name derived from the repository path

These identifiers populate the `ModelInfo` dataclass (lines 14–30 of [`types.py`](https://github.com/Andyyyy64/whichllm/blob/main/types.py)), which serves as the central schema for all extracted data.

## Technical Architecture Parameters

### Parameter Count Resolution

The most complex extraction involves determining model size through `_extract_param_count` (lines 50–61), which follows a prioritized cascade:

1. **Authoritative overrides** (hard-coded corrections for known discrepancies)
2. **Safetensors metadata** (from `safetensors` index files)
3. **GGUF metadata** (for quantized variants)
4. **Config estimation** (calculating from architecture config)
5. **Name-hint parsing** (extracting size from model ID strings)
6. **Hard-coded table** (fallback lookup for known models)

Results are stored in `ModelInfo.parameter_count`.

### Active Parameter Counts for MoE Models

For Mixture-of-Experts (MoE) architectures, WhichLLM extracts the **active parameter count**—the number of parameters actually used per token—via `_resolve_moe_active_params` (lines 92–104). This logic checks curated tables (`_KNOWN_MOE_ACTIVE_PARAMS`) or parses the model ID (`_extract_active_size_hint_from_id`) to distinguish between total and active parameters stored in `ModelInfo.parameter_count_active`.

### Architecture and MoE Detection

The `_extract_architecture` function (lines 16–38) normalizes the model architecture by inspecting `config["architectures"]` or falling back to `config["model_type"]`, producing standardized strings like `llama`, `qwen2`, or `mixtral`.

MoE status is determined in `_parse_model` (lines 92–94) by checking config fields or known MoE active-parameter lookups, populating the boolean `is_moe` field.

### Context Length Extraction

Maximum sequence length is resolved through a priority chain:
- Primary source: `config.max_position_embeddings` or `config.max_sequence_length` (lines 45–48)
- Fallback: `gguf` metadata `context_length` field (lines 49–50)

The final value is stored in `ModelInfo.context_length`.

## Distribution and Popularity Metrics

WhichLLM captures engagement metrics directly from the HuggingFace API response:

- **Downloads** (`downloads`): Total download count from `data.get("downloads")` (line 68)
- **Likes** (`likes`): Community endorsement count from `data.get("likes")` (line 69)
- **Published At** (`published_at`): Timestamp extraction via `_extract_published_at` (lines 34–42), preferring `createdAt` with fallback to `lastModified`
- **License** (`license`): SPDX-style license string from `cardData.license` (line 66)

## GGUF Variant Analysis

For quantized deployments, WhichLLM performs deep inspection of GGUF files through a loop over `siblings` (lines 108–138). Each `*.gguf` file generates a `GGUFVariant` entry containing:

- **Filename**: The original file name
- **Quantization type**: Extracted via `_extract_quant_type` (lines 15–31) parsing filename patterns for identifiers like `Q4_K_M` or `FP16`
- **File size**: Reported size or estimates from `_estimate_gguf_size` using `QUANT_BYTES_PER_WEIGHT` constants from [`constants.py`](https://github.com/Andyyyy64/whichllm/blob/main/constants.py)

## Evaluation and Lineage Data

- **Benchmark Scores**: The `_extract_hf_eval_score` function (lines 83–104) filters evaluation results for "general-chat" tasks, computing median scores across benchmarks like MMLU and GSM8K, stored under the `hf_eval` key in `ModelInfo.benchmark_scores`
- **Base Model**: Extracted from `cardData.base_model` (lines 49–55) to identify finetuned variants and their parent architectures

## The Extraction Pipeline Architecture

The complete parameter extraction follows this orchestrated flow in [`src/whichllm/models/fetcher.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/models/fetcher.py):

1. **API Request Construction**: `fetch_models` calls `https://huggingface.co/api/models` with `expand[]` parameters for `config`, `safetensors`, `gguf`, `cardData`, `siblings`, and `evalResults` (lines 84–96)

2. **Model Parsing**: `_parse_model` validates the ID and dispatches to specialized extractors

3. **Parameter Resolution**: The cascade in `_extract_param_count` ensures accurate sizing even for metadata-sparse models

4. **MoE Handling**: Conditional logic detects expert architectures and resolves active parameter counts

5. **Quantization Detection**: Filename pattern matching identifies quantization schemes

6. **Architecture Normalization**: Standardization of architecture strings for consistent categorization

7. **Benchmark Aggregation**: Median calculation across available evaluation results

8. **Dataclass Assembly**: All fields populate the `ModelInfo` instance for JSON serialization and caching

## Code Examples

The following example demonstrates fetching models and accessing extracted parameters:

```python
import asyncio
from whichllm.models.fetcher import fetch_models

async def demo():
    models = await fetch_models(limit=10, include_vision=False)
    for m in models:
        print(f"🧩 {m.id}")
        print(f"  Params: {m.parameter_count:,} ({m.parameter_count_active or 'dense'})")
        print(f"  Arch:  {m.architecture}")
        print(f"  MoE?  {'yes' if m.is_moe else 'no'}")
        print(f"  Context length: {m.context_length or 'unknown'}")
        print(f"  License: {m.license or 'unspecified'}")
        print(f"  Downloads: {m.downloads:,}")
        print(f"  HF Eval score: {m.benchmark_scores.get('hf_eval', 'n/a')}")
        for gguf in m.gguf_variants:
            print(f"    GGUF: {gguf.filename} – {gguf.quant_type} – {gguf.file_size_bytes:,} B")
        print()

asyncio.run(demo())

```

To inspect publication timelines for specific frontier models:

```python
import asyncio
from whichllm.models.fetcher import fetch_model_published_at

async def get_dates():
    dates = await fetch_model_published_at(
        ["meta-llama/Llama-4-Scout-17B-16E-Instruct",
         "deepseek-ai/DeepSeek-V4-Pro"]
    )
    for model_id, ts in dates.items():
        print(f"{model_id} → published/modified at {ts}")

asyncio.run(get_dates())

```

## Summary

- WhichLLM extracts **15+ distinct parameters** from each HuggingFace model, spanning technical specs, licensing, and performance metrics
- **Parameter counts** are resolved through a robust six-tier cascade (authoritative → safetensors → GGUF → config → name-hint → table) in [`src/whichllm/models/fetcher.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/models/fetcher.py)
- **MoE models** receive special handling to distinguish between total and active parameter counts
- **GGUF variants** are fully cataloged with quantization types and file sizes for deployment planning
- All extracted data is stored in the **`ModelInfo` dataclass** defined in [`src/whichllm/models/types.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/models/types.py)

## Frequently Asked Questions

### How does WhichLLM calculate parameter counts for models missing explicit metadata?

WhichLLM employs a prioritized resolution cascade in `_extract_param_count` (lines 50–61). If safetensors metadata is unavailable, it falls back to GGUF headers, then estimates from the config file, parses the model ID for size hints, or consults a hard-coded lookup table for known models. This ensures every model receives a sensible parameter count estimate even when upstream metadata is incomplete.

### What distinguishes active parameter count from total parameter count in MoE models?

For Mixture-of-Experts architectures, the **total parameter count** represents all weights in the model, while the **active parameter count** reflects only the parameters used per token (typically the activated expert pathways). WhichLLM extracts this via `_resolve_moe_active_params` using curated tables or ID parsing, storing the result in `ModelInfo.parameter_count_active` to give users accurate VRAM and latency estimates.

### How are quantization types detected for GGUF variants?

Quantization detection occurs in `_extract_quant_type` (lines 15–31) through filename pattern matching. The function parses GGUF filenames for standard identifiers like `Q4_K_M`, `FP16`, or `Q5_0`, returning a normalized quantization string. When file size metadata is missing, WhichLLM estimates it using byte-per-weight constants from [`constants.py`](https://github.com/Andyyyy64/whichllm/blob/main/constants.py) to ensure complete variant listings.

### Where does WhichLLM store the extracted HuggingFace model parameters?

All extracted parameters populate the **`ModelInfo` dataclass** defined in [`src/whichllm/models/types.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/models/types.py) (lines 14–30). This structured type includes fields for core identifiers (`id`, `name`), technical specs (`parameter_count`, `architecture`, `context_length`), popularity metrics (`downloads`, `likes`), and nested structures for `GGUFVariant` and benchmark scores. The fetcher serializes these instances to JSON for caching and downstream ranking algorithms.