# How Plausible Handles Bot Filtering at the Database Query Level

> Plausible Analytics filters bot traffic before database ingestion. Learn how this efficient bot filtering method eliminates the need for database-level query filtering.

- Repository: [Plausible Analytics/analytics](https://github.com/plausible/analytics)
- Tags: internals
- Published: 2026-05-19

---

**Plausible Analytics filters bot traffic during the ingestion pipeline by dropping detected bot events before they reach ClickHouse, eliminating the need for database-level filtering in analytics queries.**

The `plausible/analytics` repository implements a zero-overhead approach to bot management. Instead of storing bot traffic and filtering it out during query execution, Plausible identifies and discards bot-generated events at the edge of its ingestion pipeline. This architectural decision ensures that the ClickHouse database contains only legitimate visitor data, making all downstream analytics queries inherently bot-free without additional WHERE clauses or performance penalties.

## Why Bot Filtering Happens Before the Database

Plausible’s ingestion architecture prioritizes data cleanliness over storage completeness. Bot detection occurs immediately after request parsing but before any database write operations begin. This early-exit pattern guarantees that malicious or automated traffic never consumes storage resources or complicates analytical queries.

### The Ingestion Pipeline Architecture

When a tracking request arrives, it flows through `Plausible.Ingestion.Event.build_and_buffer/2`, which executes a processing pipeline defined in `pipeline/0`. Each step transforms the event struct until it reaches the persistor. The critical observation is that this pipeline short-circuits upon encountering any validation failure, including bot detection.

### Early Exit Pattern

The pipeline uses `Enum.reduce_while/3` to halt processing immediately when an event is marked as dropped. In [`lib/plausible/ingestion/event.ex`](https://github.com/plausible/analytics/blob/main/lib/plausible/ingestion/event.ex), the `put_user_agent/2` function inspects the User-Agent string and calls `drop(event, :bot)` if it detects automated traffic. Once dropped, the event never reaches the `register_session/2` function that handles ClickHouse persistence, effectively sanitizing the database at the ingestion boundary.

## Bot Detection Implementation in the Ingestion Pipeline

Bot identification relies on **UAInspector**, a library that parses User-Agent strings against known bot signatures. The detection logic resides in the ingestion event handler and distinguishes between legitimate browsers, headless clients, and automated crawlers.

### User-Agent Parsing with UAInspector

In [`lib/plausible/ingestion/event.ex`](https://github.com/plausible/analytics/blob/main/lib/plausible/ingestion/event.ex) (lines 56-64), the `put_user_agent/2` function implements the detection logic:

```elixir
defp put_user_agent(%__MODULE__{} = event, _context) do
  case parse_user_agent(event.request) do
    {:ok, %UAInspector.Result{client: %UAInspector.Result.Client{name: "Headless Chrome"}}} ->
      drop(event, :bot)

    {:ok, %UAInspector.Result.Bot{}} ->
      drop(event, :bot)

    {:ok, %UAInspector.Result{} = user_agent} ->
      update_session_attrs(event, %{user_agent: user_agent})

    _any ->
      event
  end
end

```

This implementation catches two specific bot categories: explicit bot results (`%UAInspector.Result.Bot{}`) and headless Chrome instances often used for automated scraping. Normal visitors proceed to session attribute updates, while detected bots trigger the drop mechanism.

## The Drop Mechanism: Preventing Bots from Reaching ClickHouse

When bot detection fires, Plausible converts the event into a dropped struct rather than persisting it. This transformation happens in the `drop/2` helper function and prevents any subsequent database operations.

### How drop/2 Marks Events as Invalid

The `drop/2` function (lines 78-86 in [`lib/plausible/ingestion/event.ex`](https://github.com/plausible/analytics/blob/main/lib/plausible/ingestion/event.ex)) sets `dropped?: true` and assigns a specific `:bot` reason:

```elixir
defp drop(event, reason) do
  %{
    event
    | dropped?: true,
      drop_reason: reason,
      telemetry: [{:drop, reason} | event.telemetry]
  }
end

```

This struct modification signals to downstream pipeline stages that the event should not proceed to persistence. The function also emits telemetry data, allowing operators to monitor bot traffic volumes through metrics without storing the actual events.

### The Persistor Logic in register_session/2

The final gatekeeper resides in `register_session/2` (lines 145-152), which coordinates event persistence. This function checks the `dropped?` flag before calling the persistor:

```elixir
defp register_session(event, context) do
  if event.dropped? do
    {:error, event.drop_reason}
  else
    Persistor.persist_event(event, context)
  end
end

```

Because `event.dropped?` returns `true` for bot traffic, the persistor never receives the event, and ClickHouse remains free of bot rows. All analytics queries executed against the `events` table therefore return only genuine visitor data by default.

## Database Query Implications

By preventing bot persistence rather than filtering post-hoc, Plausible eliminates the computational overhead typically associated with bot exclusion in analytical databases.

### Zero-Overhead Analytics Queries

Since bots never enter ClickHouse, analytics queries require no additional `WHERE` predicates to exclude automated traffic. Queries like `SELECT count() FROM events WHERE site_id = ?` automatically return accurate visitor counts without filtering clauses such as `AND user_agent NOT LIKE '%bot%'`. This design maintains query simplicity and execution performance, particularly important for high-traffic installations processing millions of events.

## Summary

- **Early detection**: Plausible identifies bots using UAInspector during the ingestion pipeline in [`lib/plausible/ingestion/event.ex`](https://github.com/plausible/analytics/blob/main/lib/plausible/ingestion/event.ex), not during query execution.
- **Drop mechanism**: The `drop/2` function marks events with `dropped?: true` and `drop_reason: :bot`, preventing persistence.
- **Pipeline short-circuit**: `Enum.reduce_while/3` halts processing immediately when bots are detected, before `register_session/2` executes.
- **Clean storage**: Because `Persistor.persist_event/3` receives only non-dropped events, ClickHouse contains no bot traffic, eliminating the need for database-level filtering.
- **Telemetry visibility**: Despite dropping events, Plausible emits `:plausible, :ingest, :event, :dropped` telemetry for monitoring bot volume without storing the data.

## Frequently Asked Questions

### Does Plausible store bot traffic in a separate table?

No. Plausible does not store bot traffic in any table. When the ingestion pipeline detects a bot via `put_user_agent/2`, it immediately marks the event as dropped using `drop/2`. The `register_session/2` function then skips persistence entirely, returning `{:error, :bot}` instead of writing to ClickHouse. This zero-storage approach ensures the database remains free of bot data.

### How does Plausible detect headless browsers?

Plausible detects headless browsers through pattern matching in [`lib/plausible/ingestion/event.ex`](https://github.com/plausible/analytics/blob/main/lib/plausible/ingestion/event.ex). Specifically, the code checks for `UAInspector.Result.Client{name: "Headless Chrome"}` in addition to standard `%UAInspector.Result.Bot{}` structs. This catches automation tools that use headless Chromium but may not identify explicitly as crawlers in standard User-Agent databases.

### Is there any way to include bot traffic in Plausible analytics?

No, there is no configuration option to include bot traffic in the analytics interface. The bot filtering is hardcoded in the ingestion pipeline and operates as a binary gate before storage. Because bot events never reach ClickHouse, they cannot be retrieved through any query interface or API endpoint. Operators wishing to analyze bot traffic must monitor the telemetry events emitted when `drop/2` is invoked.

### What happens to dropped bot events?

Dropped bot events are discarded in memory without ever touching the database. The `drop/2` function sets the `dropped?` flag and records the reason (`:bot`), then the pipeline returns an error tuple that bypasses the persistor. While the events are not stored, Plausible does emit telemetry metrics (`:plausible, :ingest, :event, :dropped`) allowing infrastructure monitoring of bot detection rates without data retention.