How the Fiber Channel API Enables Inter-Fiber Communication in libfiber

The fiber channel API enables inter-fiber communication by providing buffered or unbuffered channels where fibers can send and receive data through a cooperative scheduler that automatically pairs blocking operations using an alt/select mechanism.

The libfiber library from iQIYI implements lightweight, cooperative threads that exchange data without kernel context switches. The fiber channel (ACL_CHANNEL) serves as the primary synchronization primitive, allowing multiple fibers to concurrently wait to send or receive objects while remaining in user space. This article explores the internal architecture, blocking semantics, and practical patterns for building high-performance pipelines using the channel API.

Core Architecture of the Fiber Channel API

The fiber channel implementation centers on three cooperating structures that manage pending operations and data transfer. In c/src/sync/channel.c, the ACL_CHANNEL struct maintains a circular buffer and two dynamic arrays (asend and arecv) that queue waiting fibers.

The ACL_CHANNEL Structure

According to c/src/sync/channel.c#L33-L42, the channel structure encapsulates:

typedef struct ACL_CHANNEL {
    unsigned int    elemsize;      // Size of each transferred element
    unsigned int    bufsize;       // Buffer capacity (0 = unbuffered)
    unsigned int    sendx;         // Send index in circular buffer
    unsigned int    recvx;         // Receive index in circular buffer
    FIBER_ALT_ARRAY asend;         // Fibers waiting to send
    FIBER_ALT_ARRAY arecv;         // Fibers waiting to receive
    unsigned char   buf[0];        // Flexible array member for data
} ACL_CHANNEL;

The FIBER_ALT_ARRAY structures (defined at c/src/sync/channel.c#L27-L31) act as dynamic vectors storing FIBER_ALT entries. These arrays grow automatically as more fibers block on the same channel.

The Alt Mechanism and FIBER_ALT

Every blocking operation creates a temporary FIBER_ALT structure representing the pending action. As defined at c/src/sync/channel.c#L19-L25, this structure tracks:

• The target channel (c)
• The operation type (CHANSND, CHANRCV, CHANNOBLK, or CHANEND)
• The data pointer (v)
• A link to the next alt in the queue

The channel_alt() function (implemented at c/src/sync/channel.c#L262-L352) serves as the central scheduler. It checks if any operation can execute immediately via alt_can_exec(), which verifies buffer space or peer queue availability. When multiple operations are ready, the implementation randomizes selection to ensure fairness, then performs the data copy through alt_copy() and transitions the partner fiber to FIBER_READY state.

Blocking and Non-Blocking Operations

The fiber channel API exposes symmetric primitives for data transfer, available in both blocking and non-blocking variants. The public interface resides in c/include/fiber/fiber_channel.h.

Channel Creation

Channels are instantiated through acl_channel_create() (found at c/src/sync/channel.c#L44-L53):

ACL_CHANNEL *acl_channel_create(int elemsize, int bufsize);

The bufsize parameter determines synchronization semantics. A value of 0 creates an unbuffered channel where senders and receivers must rendezvous directly. Positive values allocate a circular buffer of that many elements, allowing producers to proceed without immediate consumers.

Send and Receive Primitives

The API provides type-safe and specialized variants for common use cases:

Blocking operations (defined at c/src/sync/channel.c#L369-L381):

• acl_channel_send(ACL_CHANNEL *c, void *v) – Copies elemsize bytes from v into the channel or queues the fiber until a receiver arrives
• acl_channel_recv(ACL_CHANNEL *c, void *v) – Copies data from the channel into v, blocking if empty

Non-blocking operations (defined at c/src/sync/channel.c#L374-L386):

• acl_channel_send_nb(ACL_CHANNEL *c, void *v) – Returns -1 immediately if buffer full or no waiting receiver
• acl_channel_recv_nb(ACL_CHANNEL *c, void *v) – Returns -1 immediately if no data available

Specialized variants avoid copying large structures:

• acl_channel_send_p() / acl_channel_recv_p() – Transfer void* pointers
• acl_channel_send_ul() / acl_channel_recv_ul() – Transfer unsigned long values

Inter-Fiber Communication Patterns

The fiber channel API supports common concurrency patterns through its blocking and alt mechanisms. These patterns enable lock-free pipelines that scale with the lightweight fiber model.

Producer-Consumer with Buffered Channels

Buffered channels decouple producers and consumers through a bounded queue. When the buffer reaches capacity, acl_channel_send transparently blocks the producer, yielding control to the scheduler until the consumer creates space.

#include "fiber/libfiber.h"
#include "fiber/fiber_channel.h"

#define BUF_ELEM_SIZE   sizeof(int)
#define CHANNEL_BUF     4

static void producer(ACL_FIBER *fb, void *ctx)
{
    ACL_CHANNEL *ch = (ACL_CHANNEL *)ctx;
    for (int i = 0; i < 10; ++i) {
        acl_channel_send(ch, &i);  // Blocks if buffer full
    }
    acl_fiber_exit(0);
}

static void consumer(ACL_FIBER *fb, void *ctx)
{
    ACL_CHANNEL *ch = (ACL_CHANNEL *)ctx;
    int value;
    for (int i = 0; i < 10; ++i) {
        acl_channel_recv(ch, &value);  // Blocks if buffer empty
    }
    acl_fiber_exit(0);
}

int main(void)
{
    ACL_CHANNEL *ch = acl_channel_create(BUF_ELEM_SIZE, CHANNEL_BUF);
    acl_fiber_create(producer, ch, 128000);
    acl_fiber_create(consumer, ch, 128000);
    acl_fiber_schedule();  // Cooperative scheduler loop
    acl_channel_free(ch);
    return 0;
}

Non-Blocking Communication Patterns

For scenarios requiring back-pressure or event-driven logic, the *_nb() variants enable fibers to probe channel state without yielding:

int try_send(ACL_CHANNEL *ch, int val)
{
    if (acl_channel_send_nb(ch, &val) == 0) {
        return 0;  // Successfully queued
    }
    return -1;  // Would block, handle overflow
}

int try_recv(ACL_CHANNEL *ch, int *val)
{
    return acl_channel_recv_nb(ch, val);  // 0 = success, -1 = empty
}

These primitives allow fibers to implement timeout logic, drop excess messages, or redirect traffic when downstream consumers lag.

Multi-Channel Select Operations

The internal channel_alt() function enables fibers to wait on the first ready operation across multiple channels, similar to Go's select statement. When a fiber initiates multiple operations, the scheduler builds a FIBER_ALT array:

FIBER_ALT a[3];
a[0].c = ch1; a[0].op = CHANSND; a[0].v = &data;
a[1].c = ch2; a[1].op = CHANRCV; a[1].v = &buffer;
a[2].op = CHANEND;  // Terminator for blocking wait

channel_alt(a);  // Returns when either ch1 accepts send or ch2 provides data

This mechanism, orchestrated within c/src/sync/channel.c, guarantees that exactly one operation completes atomically while others are canceled, preventing race conditions during complex synchronization scenarios.

Summary

The fiber channel API in libfiber transforms cooperative multitasking into a message-passing architecture:

• Channels combine circular buffers with FIBER_ALT_ARRAY queues to manage pending operations
• Blocking calls enqueue fibers in asend or arecv arrays, allowing the scheduler to match complementary operations through channel_alt()
• Non-blocking calls test channel state via alt_can_exec() and return immediately if unavailable
• Alt/select logic provides fairness through randomized selection among ready operations, supporting complex synchronization across multiple channels

By keeping communication within the user-space scheduler, the API eliminates kernel context switches and lock contention typical of thread-based concurrency.

Frequently Asked Questions

What is the difference between buffered and unbuffered fiber channels?

Buffered channels (created with bufsize > 0 in acl_channel_create()) store up to bufsize elements in a circular buffer, allowing producers to send multiple messages before blocking. Unbuffered channels (bufsize == 0) enforce strict rendezvous semantics where acl_channel_send blocks until a concurrent acl_channel_recv is ready to receive the data directly.

How does the fiber channel API handle back-pressure?

Back-pressure emerges naturally through blocking semantics. When a buffered channel fills, subsequent acl_channel_send calls block the producer fiber until the consumer creates space. Applications can avoid blocking by using acl_channel_send_nb() to detect saturation and implement custom overflow handling, such as dropping messages or spawning additional consumers.

Can multiple fibers wait on the same channel simultaneously?

Yes. The FIBER_ALT_ARRAY structures (asend and arecv in ACL_CHANNEL) dynamically expand to accommodate multiple waiting fibers. When data becomes available, channel_alt() selects one waiting fiber randomly to prevent starvation, waking it via FIBER_READY state transition while others remain queued.

Is the fiber channel API thread-safe for use with OS threads?

No. The fiber channel API is designed exclusively for communication between fibers managed by the same libfiber scheduler instance. The implementation does not use mutexes or atomic operations for the internal FIBER_ALT queues, as it assumes cooperative scheduling. For inter-thread communication, standard pthread primitives or the library's thread-aware synchronization mechanisms should be used instead.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}