# Redis Cache Key Design in Nitter: Namespace Prefixes and Sharded Buckets

> Discover Nitter's Redis cache key design featuring namespace prefixes like p: and pid: and sharded buckets for efficient Twitter data organization.

- Repository: [Zed/nitter](https://github.com/zedeus/nitter)
- Tags: internals
- Published: 2026-09-04

---

**Nitter implements a structured, prefix-based Redis cache key scheme that uses short namespace identifiers (such as `p:`, `pid:`, and `t:`) combined with deterministic construction patterns and hash-mod bucketing to organize cached Twitter data efficiently.**

The open-source Twitter frontend Nitter, maintained in the zedeus/nitter repository, stores virtually every cached object in Redis under a carefully designed key hierarchy. This Redis cache key design ensures fast lookups, avoids namespace collisions, and keeps related data grouped together while supporting high-throughput caching of user profiles, tweets, lists, and RSS feeds.

## Core Principles of the Cache Key Architecture

Nitter’s cache organization follows three fundamental principles: namespace isolation, deterministic key generation, and sharded storage for high-cardinality data sets.

### Namespace Prefix Isolation

Every object type in Nitter receives a unique, short prefix that partitions the Redis keyspace. As implemented in `src/redis_cache.nim` (lines 66-70), these prefixes prevent collisions between different data types:

- **`p:`** — User profiles (`userKey`)
- **`pid:`** — User ID buckets (`uidKey`)
- **`t:`** — Individual tweets (`tweetKey`)
- **`l:`** — Twitter lists (`listKey`)
- **`rss:`** — RSS feed outputs
- **`bc:`** — Broadcast metadata
- **`sp:`** — Audio spaces
- **`ai:`** — Account information

This convention allows administrators to inspect cache contents easily using pattern matching (for example, `KEYS p:*` to list all cached profiles) while ensuring that a username cannot collide with a tweet ID or list identifier.

### Deterministic Key Construction

Keys are built from stable identifiers through template functions that concatenate components with colons. In `src/redis_cache.nim`, the system uses templates like `uidKey`, `userKey`, and `listKey` as the single source of truth for naming.

For RSS feeds, the `redisKey` procedure in `src/routes/rss.nim` (lines 13-17) constructs keys by combining the page type, identifier, and cursor:

```nim
proc redisKey(page, name, cursor: string): string =
  page & ":" & name & (if cursor.len > 0: ":" & cursor else: "")

```

This deterministic approach ensures that identical requests always map to the same Redis entry, maximizing cache hit rates across paginated timelines and search results.

### Sharded User-ID Buckets

To handle the high volume of username-to-ID mappings without creating millions of individual hash keys, Nitter implements a sharded bucket system. The `uidKey` template in `src/redis_cache.nim` (line 66) calculates the bucket using a hash modulo operation:

```nim
template uidKey(name: string): string =
  "pid:" & $(hash(name) div 1_000_000)

```

Each bucket (for example, `pid:123`) stores a Redis hash where the field is the lowercased username and the value is the numeric Twitter user ID. This keeps individual hash sizes manageable while maintaining O(1) lookup performance for ID resolution.

## Serialization and Storage Efficiency

Before writing to Redis, Nitter serializes objects using the **flatty** library and compresses the result with **supersnappy**. This `compress(toFlatty(data))` pattern, visible in `src/redis_cache.nim` (lines 86-100), reduces memory footprint and network transfer costs.

The code follows this pattern for all cached entities:

```nim
await r.setEx(key, baseCacheTime, compress(toFlatty(userObject)))

```

## Key Patterns by Object Type

### User Profiles

Full user objects are stored under the `p:` prefix using the `userKey` template:

```nim
let profileKey = userKey("someUser")  # Results in "p:someUser"

await r.setEx(profileKey, baseCacheTime, compressedData)

```

### Tweets and Lists

- **Tweets**: Stored under `t:<tweetId>` using the `tweetKey` template
- **Lists**: Stored under `l:<listId>` with list members cached separately under `cm:` and `cmm:` prefixes

### RSS Feed Caching

RSS endpoints generate keys through the `redisKey` function and prepend the `rss:` namespace. In `src/routes/rss.nim` (lines 49-55), pagination cursors become part of the key structure:

```nim
let key = "rss:" & query  # Where query contains page, hash, and cursor

await cacheRss(key, rssData)

```

This design caches each page of results independently, allowing users to paginate through RSS feeds without redundant API calls to Twitter.

### Ephemeral and Long-Term Data

- **Broadcasts (`bc:`)**: Live video metadata with standard TTL
- **Audio Spaces (`sp:`)**: Running streams with shorter TTLs
- **Account Info (`ai:`)**: Cached for 24 hours using `setEx("ai:" & toLower(name), ttl, data)`

## TTL and Expiration Strategy

Nitter applies **per-object TTLs** based on data volatility. The `baseCacheTime` (approximately one hour) applies to most objects, while list-related data uses the configurable `listCacheTime` for longer retention. The `setEx` command is used throughout `src/redis_cache.nim` (lines 75-78) to ensure automatic expiration and prevent stale data accumulation.

## Summary

- Nitter uses **single-character prefixes** (`p:`, `t:`, `l:`, etc.) to isolate object types in Redis and prevent key collisions.
- **User ID resolution** employs sharded buckets (`pid:<hash_mod_1M>`) that store username-to-ID mappings in Redis hashes for scalable lookups.
- All cached objects undergo **flatty serialization** followed by **supersnappy compression** before storage to optimize memory usage.
- **RSS pagination** implements cursor-based key construction (`rss:<page>:<hash>:<cursor>`) to cache paginated results independently.
- Helper templates in `src/redis_cache.nim` (`uidKey`, `userKey`, `tweetKey`) provide a single source of truth for key generation across the codebase.

## Frequently Asked Questions

### How does Nitter prevent cache key collisions between usernames and tweet IDs?

Nitter enforces **strict namespace prefixes** through the template functions defined in `src/redis_cache.nim`. User profiles always use the `p:` prefix (e.g., `p:elonmusk`), while tweets use `t:` (e.g., `t:1234567890`). This prefix isolation ensures that even if a username matches a numeric ID string, they occupy separate Redis keyspaces.

### Why does Nitter use hash bucketing for user ID storage instead of direct string keys?

The **sharded bucket design** (`pid:<bucket_number>`) prevents the creation of millions of individual Redis keys for username-to-ID mappings. By hashing the username and dividing by 1,000,000, Nitter groups thousands of user mappings into a manageable number of Redis hashes (approximately 1,000 buckets), reducing memory overhead while maintaining constant-time lookup performance through `HGET` operations.

### How does Nitter cache paginated RSS feeds?

RSS cache keys incorporate the **pagination cursor** into the key name using the `redisKey` function in `src/routes/rss.nim`. The format `rss:<page>:<identifier>:<cursor>` means that page one and page two of the same timeline receive distinct Redis keys. This allows Nitter to serve cached RSS pages instantly without re-fetching upstream data when users paginate through feeds.

### What compression algorithm does Nitter use for cached objects?

Nitter uses **supersnappy** compression on top of **flatty** serialization. As implemented in `src/redis_cache.nim`, the code calls `compress(toFlatty(data))` before writing to Redis via `setEx`. This combination provides fast compression/decompression speeds suitable for high-throughput caching while significantly reducing the bytes stored in Redis compared to raw JSON or MessagePack formats.