# How Collision Filtering Using Category and Mask Bits Works in Box3D

> Learn how Box3D uses category and mask bits for collision filtering. Control object interactions efficiently with this powerful technique. Optimize your physics engine.

- Repository: [Erin Catto/box3d](https://github.com/erincatto/box3d)
- Tags: how-to-guide
- Published: 2026-08-01

---

**Box3D determines whether two shapes collide by testing reciprocal 64-bit category and mask bitmasks, with an optional group index that can override bitwise logic to force or disable collisions for specific object groups.**

Collision filtering using category and mask bits in the erincatto/box3d repository provides precise control over which physics objects generate contacts. The system evaluates potential collisions early using efficient bitwise operations defined in [`src/shape.h`](https://github.com/erincatto/box3d/blob/main/src/shape.h), preventing expensive narrow-phase detection between objects that should not interact, such as teammates or projectiles and their owners.

## Understanding the b3Filter Structure

The collision filter in Box3D centers on the `b3Filter` structure defined in [`include/box3d/types.h`](https://github.com/erincatto/box3d/blob/main/include/box3d/types.h). This structure contains three fields that govern interaction eligibility:

- **categoryBits**: A 64-bit bitmask declaring which categories the shape belongs to.
- **maskBits**: A 64-bit bitmask declaring which categories this shape will accept collisions from.
- **groupIndex**: A signed integer that overrides category/mask logic when two shapes share the same non-zero value.

When you initialize a shape definition with `b3DefaultShapeDef()`, the filter defaults to `categoryBits = 1` (category 1) and `maskBits = B3_DEFAULT_MASK_BITS` (collides with everything). You override these values by setting `b3ShapeDef.filter` before creating the shape.

## The Collision Decision Logic in src/shape.h

The core collision test resides in the inline function `b3ShouldShapesCollide` within [`src/shape.h`](https://github.com/erincatto/box3d/blob/main/src/shape.h). This function returns a boolean indicating whether two shapes should proceed to narrow-phase contact generation.

### Group Index Overrides

The function first checks for group-based exceptions before evaluating category bits:

```c
if (filterA.groupIndex == filterB.groupIndex && filterA.groupIndex != 0)
{
    return filterA.groupIndex > 0;
}

```

When two shapes share an identical non-zero `groupIndex`, the sign of that integer dictates the result immediately. A positive value forces collision between the shapes regardless of their category settings, while a negative value unconditionally disables it. This mechanism is ideal for implementing team-based collision rules or ensuring that projectiles always hit their targets.

### Category and Mask Bitwise Test

If no group override applies, Box3D performs a reciprocal bitwise AND test:

```c
return (filterA.maskBits & filterB.categoryBits) != 0 &&
       (filterA.categoryBits & filterB.maskBits) != 0;

```

For a collision to occur, **both** conditions must evaluate to true. Shape A must have a category bit that Shape B's mask accepts, and Shape B must have a category bit that Shape A's mask accepts. This bidirectional requirement ensures mutual collision consent.

The engine applies identical logic to shape-versus-query tests (such as ray casts) through `b3ShouldQueryCollide`, allowing spatial queries to respect the same filtering rules as physical collisions.

## Configuring Collision Filters in Practice

You configure collision behavior by populating the `filter` field on `b3ShapeDef` before calling creation functions like `b3CreateBoxShape` or `b3CreateSphereShape`.

### Category-to-Category Filtering

To establish specific collision relationships between object types, set the category and mask bits to target specific bit positions:

```c
b3ShapeDef defA = b3DefaultShapeDef();
defA.filter.categoryBits = 0x01;   // Belongs to category 1
defA.filter.maskBits     = 0x02;   // Accepts collisions from category 2
b3CreateBoxShape(world, bodyA, &defA, &boxA);

b3ShapeDef defB = b3DefaultShapeDef();
defB.filter.categoryBits = 0x02;   // Belongs to category 2
defB.filter.maskBits     = 0x01;   // Accepts collisions from category 1
b3CreateBoxShape(world, bodyB, &defB, &boxB);

```

These shapes will collide because `(0x02 & 0x02) != 0` and `(0x01 & 0x01) != 0`, satisfying both reciprocal bitwise conditions.

### Group-Based Team Collision

Use `groupIndex` to bypass category logic for entire teams or projectile groups:

```c
b3ShapeDef playerDef = b3DefaultShapeDef();
playerDef.filter.groupIndex = -1;  // Same negative group: never collide
b3CreateBoxShape(world, playerBody1, &playerDef, &player1);
b3CreateBoxShape(world, playerBody2, &playerDef, &player2);

b3ShapeDef bulletDef = b3DefaultShapeDef();
bulletDef.filter.groupIndex = 1;   // Positive group: always collide
b3CreateSphereShape(world, bulletBody, &bulletDef, &bullet);

```

In this configuration, the two players pass through each other despite their category settings, while bullets sharing the positive group index will always collide with one another.

### Query Filtering for Ray Casts

Ray casts and overlap queries respect the same filtering mechanism through `b3QueryFilter`:

```c
b3QueryFilter query = {
    .categoryBits = 0x01,  // Query belongs to category 1
    .maskBits = 0xFF       // Accepts collisions with categories 0-7
};
b3RayCast(world, start, direction, maxDistance, &query, callback);

```

This setup ensures the ray only reports hits against shapes whose categories are enabled in the query's mask, and whose masks accept category 1.

## Performance Benefits of Early Rejection

Collision filtering using category and mask bits occurs during the broad phase in [`src/broad_phase.c`](https://github.com/erincatto/box3d/blob/main/src/broad_phase.c) before expensive narrow-phase calculations begin. By inserting objects into the dynamic tree with their `categoryBits`, the engine culls impossible collision pairs early using the trivial bitwise operations in `b3ShouldShapesCollide`. This prevents the generation of contact manifolds and GJK/EPA calculations for objects that are logically unrelated, which is essential for maintaining high performance in simulations with thousands of non-interacting entities.

## Summary

- Box3D collision filtering relies on **64-bit category and mask bitmasks** defined in `b3Filter` and evaluated in [`src/shape.h`](https://github.com/erincatto/box3d/blob/main/src/shape.h).
- The `b3ShouldShapesCollide` function requires **both** `(maskA & categoryB)` and `(categoryA & maskB)` to be non-zero for collision approval.
- **Group index** overrides category/mask logic when two shapes share the same non-zero value: positive forces collision, negative disables it.
- Configure filters via `b3ShapeDef.filter` before calling `b3CreateBoxShape`, `b3CreateSphereShape`, or similar creation functions.
- Query operations use identical filtering through `b3QueryFilter`, ensuring ray casts respect shape collision rules.
- Early filtering in the broad phase prevents unnecessary narrow-phase computations, optimizing simulation performance.

## Frequently Asked Questions

### What happens if two shapes have no overlapping category and mask bits?

If either `(maskA & categoryB)` or `(categoryA & maskB)` equals zero, `b3ShouldShapesCollide` returns false and the potential collision is discarded during the broad phase. The objects will pass through each other without generating contacts or invoking contact listeners.

### Can a single shape belong to multiple collision categories?

Yes. Since `categoryBits` is a 64-bit field, you can assign a shape to multiple categories by OR-ing bit values: `shapeDef.filter.categoryBits = 0x01 | 0x04 | 0x08`. The shape will collide with any query or shape whose `maskBits` has any of those corresponding bits set.

### How does the group index interact with category bits?

When two shapes share an identical non-zero `groupIndex`, the group logic takes precedence and the category/mask bits are ignored entirely. A positive group index forces collision; a negative group index prevents it. This is useful for creating teams or collision exceptions without reconfiguring individual category masks for every shape.

### Where is the collision filter data stored in the Box3D source?

The filter structure `b3Filter` is defined in [`include/box3d/types.h`](https://github.com/erincatto/box3d/blob/main/include/box3d/types.h). The collision decision logic resides in [`src/shape.h`](https://github.com/erincatto/box3d/blob/main/src/shape.h) within the `b3ShouldShapesCollide` and `b3ShouldQueryCollide` functions. Shape creation and default filter application occur in [`src/shape.c`](https://github.com/erincatto/box3d/blob/main/src/shape.c), while the broad phase uses category bits when inserting proxies into the dynamic tree in [`src/broad_phase.c`](https://github.com/erincatto/box3d/blob/main/src/broad_phase.c).