How to Download and Work with ECMWF IFS Model Data Using Herbie

Herbie provides a Python interface to locate, download, and read ECMWF Integrated Forecast System (IFS) data using URL templates built in src/herbie/models/ecmwf.py, with automatic handling of source priorities and GRIB2 subsetting.

Herbie is an open-source Python library that simplifies access to numerical weather prediction data, including the ECMWF IFS model. The package handles the complexity of remote data access by managing URL templates, source failover between cloud providers, and GRIB2 parsing. This guide explains the specific steps for downloading and working with ECMWF IFS model data using Herbie's core API according to the blaylockbk/herbie source code.

Step 1: Create a Herbie Object for IFS Data

The first step is instantiating a Herbie object that points to a specific forecast run. In src/herbie/models/ecmwf.py, the ifs class builds the remote URL template based on the forecast date, resolution, product type, and forecast hour.

When creating the object, specify the model="ifs" parameter to select the IFS template. If you omit the resolution, the template defaults to 0.25° for dates after 2024-02-01 and automatically falls back to the legacy 0.4° product for older archives. The product parameter typically uses "oper" for the high-resolution operational forecast.

from datetime import datetime
from herbie import Herbie

forecast_date = datetime(2024, 2, 28, 0)

H = Herbie(
    forecast_date,
    model="ifs",
    product="oper",
    fxx=0,  # forecast hour

)

Step 2: Download Full Files or Subsets

Once configured, retrieve data using the download() method implemented in src/herbie/core.py. This method handles HTTP requests, retries, and progress reporting across multiple sources.

Herbie checks sources in a specific priority order: Google Cloud Storage → AWS → ECMWF → Azure. You can override this default sequence by passing the priority parameter with any key from the SOURCES attribute.

For targeted retrieval, pass a GRIB message subset regular expression to download only specific fields rather than the full file. This significantly reduces bandwidth and storage requirements.

# Download the complete GRIB2 file

full_path = H.download()

# Download only 2-metre temperature using regex syntax

subset_path = H.download(":2t:")

# Force Azure as the data source

H_azure = Herbie(forecast_date, model="ifs", product="oper", priority="azure")
azure_file = H_azure.download()

Step 3: Load Data into xarray

Convert downloaded GRIB2 data into an analysis-ready format using the xarray() method. This function automatically parses the ECCodes-style index file (configured via self.IDX_STYLE = "eccodes" in the IFS template) and returns a lazy-loaded xarray.Dataset.

You can apply filters during loading to read specific variables without preprocessing the file.

# Load the 2-metre temperature subset directly

ds = H.xarray(":2t:")
print(ds)

# Load multiple variables with regex OR syntax

ds = H.xarray(":TMP:|:10(?:u|v):")

Complete Working Example

This end-to-end example demonstrates configuration, instantiation, downloading, and analysis of ECMWF IFS model data, matching the test patterns found in tests/test_ecmwf.py.

from pathlib import Path
from datetime import datetime
from herbie import Herbie, config

# Configure save directory

save_dir = Path.home() / "herbie-data"
config["default"]["save_dir"] = save_dir

# Define forecast initialization time

forecast_date = datetime(2024, 2, 28, 0)

# Create Herbie instance

H = Herbie(
    forecast_date,
    model="ifs",
    product="oper",
    save_dir=save_dir,
    overwrite=True,
)

# Download full file

full_path = H.download()
print(f"Full file saved to: {full_path}")

# Download specific variable

temp_path = H.download(":2t:")
print(f"Temperature subset saved to: {temp_path}")

# Load into xarray for analysis

ds = H.xarray(":2t:")
print(ds)

Advanced Usage Patterns

Parallel Downloads for Multiple Forecasts

Process multiple forecast hours efficiently by creating multiple Herbie objects and utilizing multi-threaded downloads.

from datetime import timedelta

date = datetime(2024, 2, 28, 0)
objs = [
    Herbie(date + timedelta(hours=h), model="ifs", product="oper")
    for h in range(0, 7, 3)  # 0, 3, 6-hour forecasts

]

for H in objs:
    H.download(max_threads=10)

Preserving Original Files with Subsets

When loading filtered data into xarray, you can retain the original GRIB2 file for later reuse by setting remove_grib=False.

filters = ":TMP:|:10(?:u|v):"
ds = H.xarray(filters, remove_grib=False)

Summary

• URL Template Construction: The IFS model path is built in src/herbie/models/ecmwf.py using date, resolution, product, and forecast hour parameters, defaulting to 0.25° resolution for modern dates.
• Intelligent Sourcing: The download logic in src/herbie/core.py manages source priority across Google, AWS, ECMWF, and Azure endpoints with automatic failover.
• Efficient Subsetting: Use Python regex patterns (e.g., ":2t:") with download() or xarray() to retrieve specific GRIB messages without downloading full files.
• Lazy Loading: The xarray() method leverages ECCodes indexing to provide memory-efficient access to large forecast datasets.

Frequently Asked Questions

What is the default spatial resolution for ECMWF IFS data in Herbie?

According to the template logic in src/herbie/models/ecmwf.py, Herbie defaults to 0.25° resolution for forecast dates on or after February 1, 2024. For older archives, it automatically falls back to the legacy 0.4° product to ensure data availability.

How do I download only specific variables instead of the full GRIB2 file?

Pass a regular expression string to the search parameter in H.download() or the filter parameter in H.xarray(). For example, use ":2t:" for 2-metre temperature or ":TMP:850 mb:" for temperature at 850 hPa. This performs server-side or client-side subsetting based on GRIB message headers.

Can I force Herbie to use a specific data source like Azure or AWS?

Yes. Override the default priority (Google → AWS → ECMWF → Azure) by passing the priority parameter when creating the Herbie object. For example, Herbie(date, model="ifs", priority="azure") forces the downloader to attempt Azure first, as defined in the SOURCES attribute in src/herbie/core.py.

What dependencies are required to read IFS data into xarray?

Herbie requires eccodes bindings to parse the ECMWF index files (configured via self.IDX_STYLE = "eccodes"). The xarray() method uses cfgrib or similar engines under the hood to convert GRIB2 messages into labeled xarray datasets, enabling immediate scientific analysis without manual GRIB decoding.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}