How to Implement an NSMutableData-like Structure with mulle-buffer

You can implement an NSMutableData-like resizable byte buffer in C using mulle-buffer, which provides struct mulle_buffer with automatic growth, custom allocator support, and direct memory access patterns equivalent to Apple's NSMutableData.

The mulle-c/mulle-buffer repository offers a lightweight, zero-overhead C library for dynamic memory management. By leveraging the flexible buffer API, you can replicate Foundation's NSMutableData behaviors—including dynamic appending, length manipulation, and ownership transfer—while maintaining full control over allocation strategies through custom allocators.

Core Architecture: Mapping NSMutableData Concepts

mulle-buffer provides a direct conceptual mapping to NSMutableData through two primary structures. The public ** struct mulle_buffer ** (defined in src/mulle-buffer.h) serves as the user-facing handle containing the allocator pointer, while ** struct mulle__buffer ** (defined in src/mulle--buffer.h) manages the underlying storage layout with _curr (cursor), _sentinel (capacity boundary), and storage pointers.

Buffers initialize as either flexible or inflexible. Flexible buffers created with MULLE_BUFFER_FLEXIBLE_DATA or mulle_buffer_init_default automatically grow via _mulle__buffer_grow, which doubles the allocation size when capacity is exceeded. Inflexible buffers initialized with MULLE_BUFFER_INFLEXIBLE_DATA or mulle_buffer_init_inflexible_with_static_bytes maintain fixed capacity and reject overflow operations. Every buffer carries a ** struct mulle_allocator * **; when NULL, the global mulle_default_allocator provides standard malloc/free semantics.

The API surface directly corresponds to NSMutableData operations:

• initWithCapacity: → mulle_buffer_init(buffer, capacity, NULL)
• appendBytes:length: → mulle_buffer_add_bytes(buffer, ptr, len)
• mutableBytes → mulle_buffer_get_bytes(buffer) (direct pointer access)
• setLength: → mulle_buffer_set_length(buffer, new_len, options)
• copy (transfer ownership) → mulle_buffer_extract_data(buffer) returning struct mulle_data

All functions are implemented as inline wrappers in src/mulle-buffer.h to eliminate call overhead, while heavy logic resides in src/mulle--buffer.c.

Implementation Examples

Basic Dynamic Buffer with Automatic Growth

Create a heap-backed flexible buffer that grows automatically as you append data, equivalent to [[NSMutableData alloc] init].

#include "mulle-buffer.h"

int main(void)
{
    struct mulle_buffer buf;

    /* Initialize with default capacity (~128 bytes) and flexible growth */
    mulle_buffer_init_default(&buf);

    /* Append raw bytes */
    const char payload[] = "Hello, mulle-buffer!";
    mulle_buffer_add_bytes(&buf, (void *)payload, sizeof(payload) - 1);

    /* Append integer in network order */
    mulle_buffer_add_uint32(&buf, 0xDEADBEEFU);

    /* Access as C-string (auto-null-terminated) */
    char *cstr = mulle_buffer_get_string(&buf);
    printf("Content: %s\n", cstr);

    /* Truncate length (equivalent to setLength:) */
    mulle_buffer_set_length(&buf, 5, MULLE_BUFFER_NO_SHRINK_OR_ZEROFILL);
    printf("Truncated: %s\n", mulle_buffer_get_string(&buf));

    /* Extract ownership of underlying memory */
    struct mulle_data data = mulle_buffer_extract_data(&buf);
    printf("Extracted %zu bytes, buffer now empty\n", data.length);

    /* Release extracted memory using stored allocator */
    mulle_allocator_free(data.allocator, data.bytes);
    mulle_buffer_destroy(&buf);
    return 0;
}

Stack-Based Static Buffer

Use pre-allocated stack memory to avoid heap allocation entirely, falling back to the allocator only if growth exceeds static capacity.

#include "mulle-buffer.h"

#define STACK_CAPACITY 256

int main(void)
{
    unsigned char stack[STACK_CAPACITY];
    struct mulle_buffer buf;

    /* Initialize with static storage; no malloc occurs until overflow */
    mulle_buffer_init_with_static_bytes(&buf, stack, STACK_CAPACITY, NULL);

    mulle_buffer_add_string(&buf, "Stack-based buffer");
    printf("%s\n", mulle_buffer_get_string(&buf));

    /* Cleanup required even for static storage if growth occurred */
    mulle_buffer_destroy(&buf);
    return 0;
}

Fixed-Size Inflexible Buffer

Create a read-only-capable buffer that rejects writes beyond its fixed capacity, useful for parsing or constrained embedded contexts.

#include "mulle-buffer.h"

int main(void)
{
    unsigned char fixed[64];
    struct mulle_buffer buf;

    /* Inflexible initialization prevents any reallocation */
    mulle_buffer_init_inflexible_with_static_bytes(&buf, fixed, sizeof(fixed), NULL);

    mulle_buffer_add_bytes(&buf, "Fixed size", 10);
    
    printf("Length=%zu, Capacity=%zu\n", 
           mulle_buffer_get_length(&buf),
           mulle_buffer_get_capacity(&buf));

    mulle_buffer_destroy(&buf);
    return 0;
}

Direct Memory Access with Guarantee

Reserve space and write directly into the buffer's memory region, mirroring NSMutableData's mutableBytes pattern for zero-copy operations.

#include "mulle-buffer.h"
#include <string.h>

int main(void)
{
    struct mulle_buffer buf;
    mulle_buffer_init_default(&buf);

    /* Reserve 32 bytes and obtain direct pointer */
    void *dst = mulle_buffer_guarantee(&buf, 32);
    if (dst)
        memcpy(dst, "Direct write into reserved area", 30);

    /* Advance cursor to reflect written bytes */
    mulle_buffer_advance(&buf, 30);

    printf("Result: %s\n", mulle_buffer_get_string(&buf));
    mulle_buffer_destroy(&buf);
    return 0;
}

Key Source Files

Understanding the repository layout helps you navigate the implementation details when debugging or extending functionality:

• src/mulle-buffer.h – Public API with inline wrappers, creation macros (MULLE_BUFFER_DATA, MULLE_BUFFER_FLEXIBLE_DATA), and high-level operations like mulle_buffer_add_string and mulle_buffer_extract_data.

• src/mulle--buffer.h – Internal structure definitions (struct mulle__buffer) and low-level growth logic including _mulle__buffer_grow which implements the doubling allocation strategy.

• src/mulle--buffer.c – Non-inline implementations for buffer lifecycle management, destruction, and complex reallocation scenarios.

• src/mulle-flushablebuffer.h and src/mulle-flushablebuffer.c – Extended buffer variant that streams data to file descriptors, useful for implementing output streams similar to NSOutputStream.

• test/buffer/example.c – Reference implementation demonstrating typical creation, appending, and extraction patterns.

Summary

• ** struct mulle_buffer ** provides the public API for NSMutableData-like behavior, while struct mulle__buffer handles internal storage layout in src/mulle--buffer.h.

• Flexible buffers automatically double their capacity when full via _mulle__buffer_grow, while inflexible buffers maintain fixed size and reject overflow.

• Use mulle_buffer_init_default for heap-backed dynamic growth, mulle_buffer_init_with_static_bytes for stack optimization, and mulle_buffer_init_inflexible_with_static_bytes for fixed-size constraints.

• ** mulle_buffer_extract_data ** transfers ownership of the underlying memory block to the caller, leaving the buffer empty but initialized.

• All operations support custom allocators through struct mulle_allocator *, enabling integration with memory pools, arenas, or embedded system allocators.

Frequently Asked Questions

How does mulle-buffer handle memory growth when appending data?

When an append operation would exceed current capacity, the library invokes _mulle__buffer_grow (defined in src/mulle--buffer.h), which doubles the allocated memory size and updates internal pointers. This exponential growth strategy ensures amortized O(1) time complexity for append operations, identical to NSMutableData's standard behavior.

Can I use mulle-buffer without calling malloc for embedded systems?

Yes. Initialize buffers with mulle_buffer_init_with_static_bytes providing a pre-allocated array (stack or global). The buffer operates entirely within that fixed space until exhausted; only if you append beyond the static capacity does it fall back to the allocator. For completely allocation-free operation, use mulle_buffer_init_inflexible_with_static_bytes which rejects growth beyond the provided buffer.

What is the difference between mulle_buffer and mulle__buffer in the source code?

struct mulle__buffer (double underscore) is the internal representation containing storage pointers, cursor, and sentinel, implemented in src/mulle--buffer.h. struct mulle_buffer (single underscore) wraps this structure and adds the allocator field, providing the public API in src/mulle-buffer.h. This separation keeps allocator configuration in the public layer while hiding storage mechanics.

How do I extract ownership of the underlying bytes from a mulle_buffer?

Call mulle_buffer_extract_data, which returns a struct mulle_data containing the bytes pointer, length, and allocator. The buffer resets to empty state (length zero, capacity zero) but remains initialized. You must later free the extracted memory using mulle_allocator_free(data.allocator, data.bytes) to match the buffer's original allocation strategy.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}