How to Calculate Multiple Aggregates with Pandas Groupby Agg

Use DataFrame.groupby().agg() with a list of functions, a dictionary of column-function pairs, or named aggregation kwargs to compute multiple statistics across groups in a single call.

The pandas groupby agg operation is the primary interface for split-apply-combine workflows in the pandas-dev/pandas repository. This method allows you to compute multiple aggregate functions across rows efficiently, routing your request through optimized Cython kernels or flexible Python fallbacks depending on the complexity of your aggregation.

Understanding the Pandas Groupby Agg Architecture

The agg() method is implemented in pandas/core/groupby/generic.py and serves as the high-level entry point for all aggregation operations. Internally, it detects whether you passed a single function, a list of functions, or a named aggregation dictionary, then routes the call to the appropriate execution engine.

Entry Point and Signature Detection

The method signature and docstring that describe supported call patterns are defined at the start of the aggregate method in pandas/core/groupby/generic.py around line 314. When you pass an iterable of functions—such as ["min", "max"] or [np.mean, "sum"]—the code normalizes the list, injects engine kwargs, and calls the private helper _aggregate_multiple_funcs around lines 361-368.

Named Aggregation Parsing

If you supply named aggregations using kwargs like df.groupby(...).agg(minimum="min", maximum="max"), the method first rewrites these arguments into a column-mapping via validate_func_kwargs (found in pandas/core/apply.py) around lines 345-348 of generic.py. This path then merges with the standard execution flow to build the output frame.

Low-Level Execution Engines

The heavy lifting of actual aggregation occurs in pandas/core/groupby/ops.py (specifically agg_series and _aggregate_multiple_funcs) and in the C-extension pandas/_libs/groupby.c. These implement the Cython kernels used when engine='cython' (the default). For custom Python functions that cannot be handled by Cython, the system falls back to _python_agg_general in generic.py, which uses self._grouper.apply_groupwise to loop over groups in Python.

Pandas Groupby Agg Syntax Patterns

The agg() method supports three primary syntax patterns for specifying multiple aggregate functions, each triggering different internal optimization paths.

List of Functions

Passing a list applies every function to every column in the group. This triggers the fast Cython path for built-in functions.

import pandas as pd
import numpy as np

df = pd.DataFrame({
    "city": ["NY", "NY", "LA", "LA", "LA"],
    "year": [2020, 2021, 2020, 2021, 2022],
    "pop": [8.4, 8.5, 4.0, 4.1, 4.2],
    "area": [784, 784, 503, 503, 503],
})

# Apply min, max, and mean to every numeric column

result = df.groupby("city").agg(["min", "max", "mean"])

Named Aggregation

Using keyword arguments creates custom output column names and allows different functions per column. This uses the validate_func_kwargs path in pandas/core/apply.py.

# Different functions per column with custom names

result = df.groupby("city").agg(
    pop_min=("pop", "min"),
    pop_mean=("pop", "mean"),
    area_sum=("area", "sum"),
)

Dictionary Syntax

You can mix list-based and named aggregation by passing a dictionary to agg(). Dictionary keys specify column names, while values specify functions or lists of functions.

# Mixing list and named aggregation via dictionary unpacking

result = df.groupby("city").agg(
    **{
        "pop": ["min", "max"],
        "area_sum": ("area", "sum"),
    }
)

Practical Examples of Pandas Groupby Agg

The following examples demonstrate real-world usage patterns, from simple statistics to custom calculations with different execution engines.

Example 1: Multiple Built-in Aggregates

Apply standard statistics to every column using a list of strings. This executes via the Cython kernels in pandas/_libs/groupby.c.

out1 = df.groupby("city").agg(["min", "max", "mean"])
print(out1)

Example 2: Named Aggregations for Clarity

Create descriptive column names while applying different functions to different columns. This triggers the relabeling path in pandas/core/groupby/generic.py.

out2 = df.groupby("city").agg(
    pop_min=("pop", "min"),
    pop_mean=("pop", "mean"),
    area_sum=("area", "sum"),
)
print(out2)

Example 3: Combining Syntax Patterns

Mix list-based aggregation for one column with named aggregation for another by using dictionary unpacking.

out3 = df.groupby("city").agg(
    **{
        "pop": ["min", "max"],
        "area_sum": ("area", "sum"),
    }
)
print(out3)

Example 4: Custom Python Functions

When you pass a lambda or custom function, pandas falls back to the pure-Python aggregator _python_agg_general in pandas/core/groupby/generic.py, which loops over groups individually.

out4 = df.groupby("city").agg(
    pop_range=lambda s: s.max() - s.min(),
    area_range=lambda s: s.max() - s.min(),
)
print(out4)

Example 5: Numba-Accelerated Aggregation

For performance-critical custom aggregations, specify engine="numba" to JIT-compile the function. This routes through _aggregate_with_numba and requires the function signature to accept values and idx parameters.

out5 = df.groupby("city").agg(
    pop_range=lambda values, idx: values.max() - values.min(),
    engine="numba",
)
print(out5)

Performance Considerations and Engine Selection

The pandas groupby agg method automatically selects the fastest execution path based on your input, but understanding these paths helps optimize performance.

Cython Engine (Default)

Built-in string aliases like "min", "max", "mean", "sum", and "std" trigger the Cython kernels in pandas/_libs/groupby.c via the agg_series helper in pandas/core/groupby/ops.py. This path is vectorized and avoids Python loops entirely.

Python Fallback

Custom callables that are not recognized built-ins route through _python_agg_general in pandas/core/groupby/generic.py. This method uses self._grouper.apply_groupwise to iterate over groups in Python, which is significantly slower for large datasets but necessary for arbitrary logic.

Numba Engine

When engine="numba" is specified and the function accepts (values, idx) arguments, pandas calls _aggregate_with_numba to JIT-compile the aggregation. This bridges the gap between custom logic and C-speed performance, though it requires Numba installation and restricts the function signature.

Summary

• DataFrame.groupby().agg() is the primary interface for computing multiple statistics across groups in the pandas-dev/pandas repository.
• Syntax flexibility: Pass a list for uniform aggregation, use named kwargs for custom column names, or combine both via dictionary unpacking.
• Execution paths: Built-in functions route through optimized Cython kernels in pandas/_libs/groupby.c, while custom lambdas fall back to _python_agg_general in pandas/core/groupby/generic.py.
• Performance optimization: Use engine="numba" for JIT-compiled custom aggregations via _aggregate_with_numba when working with large datasets.

Frequently Asked Questions

What is the difference between agg and aggregate in pandas?

agg is simply an alias for aggregate. Both names point to the same method implementation in pandas/core/groupby/generic.py, and you can use them interchangeably. The method signature and performance characteristics are identical regardless of which name you choose.

How do I apply different aggregate functions to different columns?

Use named aggregation by passing keyword arguments where each key becomes the output column name and each value is a tuple of (column_name, function). For example: df.groupby("key").agg(total=("col1", "sum"), average=("col2", "mean")). This routes through validate_func_kwargs in pandas/core/apply.py before execution.

Why is my custom lambda function slow in groupby agg?

Custom callables that are not built-in aggregation names bypass the Cython kernels in pandas/_libs/groupby.c and instead execute through _python_agg_general in pandas/core/groupby/generic.py. This method uses self._grouper.apply_groupwise to loop over groups in pure Python, which incurs significant overhead for large datasets. To improve performance, consider using built-in functions where possible or specifying engine="numba" for JIT compilation.

Can I use Numba to accelerate multiple aggregate functions simultaneously?

When using engine="numba", you can only specify a single custom function per aggregation call because the Numba path routes through _aggregate_with_numba, which JIT-compiles one kernel at a time. If you need multiple Numba-accelerated statistics, you must call agg separately for each or combine them into a single custom function that returns multiple values as a tuple.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}