# What Compression and Serialization Formats Does Nitter Use for Redis Caching?

> Discover Nitter's Redis caching strategy learn about its flatty binary serialization and supersnappy compression for efficient data storage and retrieval.

- Repository: [Zed/nitter](https://github.com/zedeus/nitter)
- Tags: internals
- Published: 2026-08-29

---

**Nitter uses the `flatty` library for binary serialization and `supersnappy` for fast Snappy-compatible compression when caching data in Redis.**

Nitter, the privacy-focused alternative Twitter front-end written in Nim, implements a high-performance Redis caching layer to reduce API calls and improve response times. According to the source code in `src/redis_cache.nim`, the application serializes Nim objects into compact binary representation before applying compression to minimize network overhead and memory usage.

## Serialization with Flatty

**Flatty** is a Nim-native binary serialization format that efficiently converts complex objects into byte streams. In Nitter's implementation, data structures such as tweets, user profiles, and timelines are encoded using `flatty` before storage.

The library provides direct encoding of Nim types without schema definitions, making it ideal for caching structured data. As implemented in `zedeus/nitter`, the serialization occurs through the `flatty.encode()` function:

```nim
let serialized = flatty.encode(tweet)  # Convert Tweet object to bytes

```

This produces a compact byte representation of the data structures defined in `src/types.nim`, including the `Tweet` and `User` types.

## Compression with Supersnappy

After serialization, Nitter applies **Supersnappy**, a high-performance Snappy-compatible compression library optimized for Nim. Supersnappy provides fast compression and decompression with minimal CPU overhead, making it suitable for high-throughput caching scenarios.

The compression reduces the payload size transmitted to Redis and stored in memory. The workflow uses `supersnappy.compress()` to pack the serialized bytes:

```nim
let compressed = supersnappy.compress(serialized)  # Compress byte payload

```

Supersnappy maintains compatibility with the standard Snappy format while incorporating performance tweaks specific to Nim's memory model.

## Implementation in src/redis_cache.nim

The core caching logic resides in `src/redis_cache.nim`, which imports both libraries alongside the Redis client:

```nim
import redis, redpool, flatty, supersnappy

```

### Writing Data to Redis

When storing cached content, Nitter executes a three-step pipeline: serialization, compression, and storage. The following pattern appears throughout the codebase when writing tweets or user data:

```nim
proc cacheTweet(key: string, tweet: Tweet) {.async.} =
  let raw = flatty.encode(tweet)          # 1. Serialize Nim object

  let packed = supersnappy.compress(raw)  # 2. Compress bytes

  await redisPool.set(key, packed)        # 3. Store in Redis

```

This approach minimizes the memory footprint of cached items while maintaining the complete object structure.

### Reading Data from Redis

Cache retrieval reverses the pipeline using decompression followed by deserialization. The implementation handles optional values to account for cache misses:

```nim
proc getCachedTweet(key: string): Future[Option[Tweet]] {.async.} =
  let packed = await redisPool.get(key)      # Retrieve compressed data

  if packed.isNone: 
    return none(Tweet)

  let raw = supersnappy.decompress(packed.get)   # Decompress bytes

  let tweet = flatty.decode(raw, Tweet)          # Deserialize to object

  return some(tweet)

```

The `flatty.decode()` function requires the type parameter (`Tweet`) to reconstruct the Nim object from the byte stream correctly.

## Data Structures and Types

The objects being cached are defined in `src/types.nim`, which contains the complete schema for tweets, users, timelines, and search results. These type definitions enable `flatty` to perform zero-overhead serialization without additional mapping layers or reflection.

When Nitter caches a timeline or user profile, it serializes the entire object graph defined in these types, preserving all nested fields and metadata required for rendering.

## Configuration

Redis connection parameters for the caching layer are managed in `src/config.nim`, which handles host, port, and authentication settings. The combination of `flatty` serialization and `supersnappy` compression operates transparently once the Redis pool is initialized, requiring no additional configuration beyond standard Redis setup.

## Summary

- **Flatty** provides binary serialization of Nim objects into compact byte representations before storage.
- **Supersnappy** applies Snappy-compatible compression to reduce Redis memory usage and network transfer.
- The `src/redis_cache.nim` module orchestrates the encode-compress-store workflow for writes and the retrieve-decompress-decode sequence for reads.
- Data structures from `src/types.nim` are serialized without schema overhead, enabling efficient caching of complex Twitter data models.

## Frequently Asked Questions

### How does Nitter handle cache misses?

When a key does not exist in Redis, `redisPool.get()` returns `none`, and the application proceeds to fetch fresh data from Twitter's API. After retrieval, the new data follows the standard serialization and compression workflow before being written back to the cache with a configurable expiration time.

### Why does Nitter use Supersnappy instead of standard compression?

Supersnappy offers superior decompression speeds compared to general-purpose algorithms like gzip or zlib, making it ideal for high-throughput caching where read latency matters more than compression ratio. The library maintains binary compatibility with Google's Snappy format while optimizing for Nim's concurrency model.

### Can the Redis caching be disabled?

Yes. Nitter can operate without Redis by using an in-memory cache or no caching at all, though this increases API rate limit consumption. Configuration options in `src/config.nim` allow administrators to specify Redis connection details or omit them entirely to disable the distributed caching layer.

### What data structures does Nitter cache in Redis?

Nitter caches deserialized API responses including `Tweet` objects, `User` profiles, timeline aggregations, and search results. These complex nested structures benefit from `flatty`'s ability to serialize arbitrary Nim objects without manual schema mapping or JSON parsing overhead.