# How to Read Files Efficiently with mulle-buffer: 5 High-Performance Techniques

> Discover 5 high-performance techniques to read files efficiently with mulle-buffer. Learn zero-copy file access and eliminate double-copying for faster data handling.

- Repository: [mulle-c/mulle-buffer](https://github.com/mulle-c/mulle-buffer)
- Tags: performance
- Published: 2026-03-07

---

**Use `mulle_buffer_guarantee` to reserve writable space for file chunks, read directly into the buffer with POSIX `read(2)`, and commit bytes with `mulle_buffer_advance` to eliminate double-copying, or wrap memory-mapped regions using `MULLE_BUFFER_STATIC_DATA` for true zero-copy file access.**

The mulle-c/mulle-buffer library provides a lightweight, growable-memory abstraction designed as a building block for high-performance I/O. While it does not expose a dedicated "read file" API, the headers [`src/mulle-buffer.h`](https://github.com/mulle-c/mulle-buffer/blob/main/src/mulle-buffer.h) and [`src/mulle-flushablebuffer.h`](https://github.com/mulle-c/mulle-buffer/blob/main/src/mulle-flushablebuffer.h) contain all primitives needed to implement efficient file readers with minimal overhead.

## Core Primitives for Zero-Copy and Streaming Reads

### Dynamic Growth with Guaranteed Capacity

When file sizes are unknown, **dynamic growth** prevents over-allocation while ensuring contiguous storage. In [`src/mulle-buffer.h`](https://github.com/mulle-c/mulle-buffer/blob/main/src/mulle-buffer.h), the `mulle_buffer_guarantee` function reserves capacity without advancing the write pointer, allowing direct I/O into uninitialized memory.

Call `mulle_buffer_guarantee(buf, n)` to obtain a writable pointer for the next *n* bytes, fill it with `read(2)` or `fread`, then commit the actual byte count with `mulle_buffer_advance`. This approach eliminates temporary buffers and reduces system call overhead by reading in large chunks (e.g., 8 KB).

### Static Storage for Memory-Mapped Files

For scenarios requiring true zero-copy access, **in-place static storage** wraps existing memory blocks without allocation. The macro `MULLE_BUFFER_STATIC_DATA` (or the function `mulle_buffer_init_with_static_bytes`) in [`src/mulle-buffer.h`](https://github.com/mulle-c/mulle-buffer/blob/main/src/mulle-buffer.h) initializes a buffer from an external memory region.

Use this with `mmap(2)` on POSIX or `CreateFileMapping` / `MapViewOfFile` on Windows. The buffer references the mapped region directly—no memcpy occurs during initialization.

### Read-Only Safety and Zero-Copy Extraction

After loading completes, **read-only mode** prevents accidental modifications and enables safe concurrent access. The function `mulle_buffer_set_readonly` (declared in [`src/mulle-buffer.h`](https://github.com/mulle-c/mulle-buffer/blob/main/src/mulle-buffer.h)) locks the buffer state.

For parsing without memory duplication, `mulle_buffer_reference_bytes` returns a direct pointer to internal storage. Feed this pointer to tokenizers, UTF-8 validators, or other byte-oriented parsers without extracting or copying data.

### Streaming with Flushable Buffers

The **flushable buffer** variant in [`src/mulle-flushablebuffer.h`](https://github.com/mulle-c/mulle-buffer/blob/main/src/mulle-flushablebuffer.h) supports custom read callbacks through the `mulle_flushablebuffer_flusher_t` hook. Implement a flusher that reads from `FILE*` handles or network sockets, appending data to the underlying buffer without intermediate temporaries.

This pattern enables streaming reads where the buffer grows on-demand as new chunks arrive, ideal for processing files larger than physical memory.

### Byte-Level Navigation

For parsers requiring sequential scanning, [`src/mulle-buffer.h`](https://github.com/mulle-c/mulle-buffer/blob/main/src/mulle-buffer.h) exposes **bounded read operations**: `mulle_buffer_next_byte`, `mulle_buffer_peek_byte`, and `mulle_buffer_seek_byte`. These work on any buffer state (including read-only) and provide safe, bounds-checked byte access for delimiter searching or protocol parsing.

## Step-by-Step Implementation Patterns

### Chunked Reading for Unknown File Sizes

When file statistics are unavailable or unreliable, read iteratively using guaranteed capacity:

```c
#include <fcntl.h>
#include <unistd.h>
#include "mulle-buffer.h"

struct mulle_buffer buf = MULLE_BUFFER_DATA(NULL);  // start with default allocator

int fd = open("data.bin", O_RDONLY);
if (fd < 0) return -1;

while (1) {
    void *dst = mulle_buffer_guarantee(&buf, 8192);  // reserve 8KB
    if (!dst) break;  // allocation failure
    
    ssize_t n = read(fd, dst, 8192);
    if (n < 0) { /* handle error */ break; }
    if (n == 0) break;  // EOF
    
    mulle_buffer_advance(&buf, (size_t)n);  // commit actual bytes read
}

mulle_buffer_set_readonly(&buf);  // protect completed data
close(fd);

```

This pattern minimizes reallocations by growing the buffer exponentially while reading fixed-size chunks.

### Zero-Copy Memory Mapping

When the entire file fits in addressable memory, map it directly:

```c
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include "mulle-buffer.h"

int fd = open("largefile.bin", O_RDONLY);
struct stat st;
fstat(fd, &st);

void *map = mmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
struct mulle_buffer buf = MULLE_BUFFER_STATIC_DATA(map, st.st_size, NULL);

mulle_buffer_set_readonly(&buf);  // protect the mapping

// Direct pointer access for parsing
const void *data = mulle_buffer_reference_bytes(&buf, mulle_buffer_get_length(&buf));
// process data...

munmap(map, st.st_size);
close(fd);

```

This approach bypasses the heap entirely for the file content, though the mapping itself consumes virtual address space.

## Key Source Files and Implementation Details

According to the mulle-c/mulle-buffer source code, three headers provide the complete toolkit:

- **[`src/mulle-buffer.h`](https://github.com/mulle-c/mulle-buffer/blob/main/src/mulle-buffer.h)**: Public API containing `mulle_buffer_guarantee`, `mulle_buffer_advance`, `mulle_buffer_set_readonly`, and `mulle_buffer_reference_bytes`. Contains inline wrappers for high-performance operations.

- **[`src/mulle--buffer.h`](https://github.com/mulle-c/mulle-buffer/blob/main/src/mulle--buffer.h)**: Low-level implementation exposing internal `_mulle__buffer_*` functions used by the public inline wrappers. Useful for understanding allocation strategies.

- **[`src/mulle-flushablebuffer.h`](https://github.com/mulle-c/mulle-buffer/blob/main/src/mulle-flushablebuffer.h)**: Defines `struct mulle_flushablebuffer` and the `mulle_flushablebuffer_flusher_t` callback type for implementing custom streaming readers.

## Summary

- **Use `mulle_buffer_guarantee`** to reserve writable capacity before system calls, eliminating temporary buffers.
- **Commit bytes with `mulle_buffer_advance`** after `read(2)` returns to maintain accurate length tracking.
- **Wrap `mmap` regions** using `MULLE_BUFFER_STATIC_DATA` for true zero-copy file access without heap allocation.
- **Lock completed buffers** with `mulle_buffer_set_readonly` to prevent accidental writes and enable safe sharing.
- **Extract raw pointers** via `mulle_buffer_reference_bytes` for parser integration without data duplication.
- **Implement streaming** through `mulle_flushablebuffer` in [`src/mulle-flushablebuffer.h`](https://github.com/mulle-c/mulle-buffer/blob/main/src/mulle-flushablebuffer.h) when processing files larger than available RAM.

## Frequently Asked Questions

### Does mulle-buffer provide a built-in function to read files directly?

No. The library intentionally operates as a low-level abstraction in [`src/mulle-buffer.h`](https://github.com/mulle-c/mulle-buffer/blob/main/src/mulle-buffer.h). You implement efficient file reading by combining `mulle_buffer_guarantee` with POSIX `read(2)` or `fread`, or by wrapping memory-mapped regions with `MULLE_BUFFER_STATIC_DATA`. This design keeps the core library agnostic to I/O mechanisms while supporting optimal performance.

### How do I prevent buffer reallocations while reading large files?

Call `mulle_buffer_guarantee` with your desired chunk size (e.g., 64 KB) before each read operation. If the existing capacity suffices, the function returns the current write pointer without reallocating. Only when the internal storage fills does the buffer grow, typically doubling in size to maintain amortized O(1) append complexity.

### Can I use mulle-buffer with memory-mapped files safely?

Yes. Initialize the buffer with `MULLE_BUFFER_STATIC_DATA` pointing to your `mmap` region, then immediately call `mulle_buffer_set_readonly`. This prevents write attempts that would trigger segmentation faults on read-only mappings. The buffer provides full read API access—including `mulle_buffer_next_byte` and `mulle_buffer_seek_byte`—without copying data from the mapped region.

### What is the difference between mulle_buffer and mulle_flushablebuffer?

`struct mulle_buffer` in [`src/mulle-buffer.h`](https://github.com/mulle-c/mulle-buffer/blob/main/src/mulle-buffer.h) is the base growable-memory container. `struct mulle_flushablebuffer` in [`src/mulle-flushablebuffer.h`](https://github.com/mulle-c/mulle-buffer/blob/main/src/mulle-flushablebuffer.h) extends this with a callback hook (`mulle_flushablebuffer_flusher_t`) that triggers when the buffer fills. Wire this flusher to `fread` or custom read functions to implement streaming readers that process files chunk-by-chunk without loading the entire contents into memory.