How to Use fiber_event for Efficient Fiber Synchronization in libfiber

Use acl_fiber_event_create() to initialize the synchronization primitive, acl_fiber_event_wait() to acquire ownership, and acl_fiber_event_notify() to release it and wake the next waiter in FIFO order, selecting FIBER_FLAG_USE_MUTEX only when contending with hundreds of threads.

The fiber_event API in the iqiyi/libfiber repository provides a lightweight, lock-free synchronization mechanism that works seamlessly between fibers and native threads. Exposed via c/include/fiber/fiber_event.h and implemented in c/src/sync/fiber_event.c, this primitive allows high-concurrency applications to coordinate access to shared resources without the overhead of traditional OS mutexes. Understanding how to use fiber_event correctly is essential for building scalable fiber-based architectures.

What is fiber_event?

fiber_event is a hybrid synchronization object defined by the ACL_FIBER_EVENT structure in fiber_event.h. Unlike standard POSIX mutexes, it combines an atomic flag for fast uncontended paths with an optional pthread_mutex_t for high-contention scenarios, while maintaining a FIFO wait queue through a ring buffer of FIBER_BASE objects.

The structure tracks ownership through two fields: owner (pointer to the holding fiber) and tid (OS thread ID). When contention occurs, waiters insert themselves into the waiters ring and block on their own event objects, eliminating the thundering herd problem by waking only the next eligible owner.

Creating and Configuring a fiber_event

Initialize the event using acl_fiber_event_create(), which accepts configuration flags to tune performance characteristics:

#include "fiber/libfiber.h"

/* High-contention scenario: use real mutex */
ACL_FIBER_EVENT *event = acl_fiber_event_create(FIBER_FLAG_USE_MUTEX);

/* Low-contention fiber-only scenario: use atomic mode */
ACL_FIBER_EVENT *event_atomic = acl_fiber_event_create(0);

The implementation in c/src/sync/fiber_event.c allocates two atomic variables during creation: one for the main lock state (event->atomic) and a secondary lock (event->lock.atomic.alock) when operating in atomic-only mode. When FIBER_FLAG_USE_MUTEX is specified, the union switches to event->lock.tlock, using a real pthread_mutex_t to protect internal fields.

Acquiring and Releasing Locks

Blocking Acquisition with acl_fiber_event_wait

The acl_fiber_event_wait() function implements a two-phase lock strategy defined in fiber_event.c:

1. Fast path: Performs an atomic compare-and-swap on event->atomic (transitioning 0 → 1). If successful, the caller immediately becomes the owner and the function returns 0.
2. Slow path: If the lock is held, the function creates a FIBER_BASE waiter, prepends it to the waiters ring via ring_prepend(), and blocks via fbase_event_wait(). When acl_fiber_event_notify() is called, the first waiter is popped from the ring and awakened.

/* Acquire ownership - blocks if necessary */
if (acl_fiber_event_wait(event) == -1) {
    /* Handle error or process aborts if FIBER_FLAG_USE_FATAL is set */
}

Non-blocking Try-Lock with acl_fiber_event_trywait

For scenarios requiring immediate feedback, acl_fiber_event_trywait() attempts a single atomic CAS operation. It returns 0 if ownership is acquired immediately, or -1 if the event is already held, without inserting the caller into the wait queue.

/* Attempt to acquire without blocking */
if (acl_fiber_event_trywait(event) == 0) {
    /* Critical section executed */
    acl_fiber_event_notify(event);
}

Signaling Waiters with acl_fiber_event_notify

Release ownership and wake the next waiter using acl_fiber_event_notify(). According to the implementation in fiber_event.c, this function:

1. Validates that the caller matches the stored owner/tid to prevent illegal releases.
2. Pops the head waiter from the waiters ring using ring_pop_head().
3. Atomically resets the lock flag (1 → 0).
4. Wakes the next waiter via fbase_event_wakeup() if one exists.

/* Release ownership and wake next waiter */
if (acl_fiber_event_notify(event) == -1) {
    /* Handle error: wrong owner or corruption detected */
}

Practical Implementation Example

The following example from samples/c/event/main.c demonstrates cross-thread synchronization using fiber_event to protect a shared counter:

#include "fiber/libfiber.h"
#include <pthread.h>
#include <stdio.h>

static long long counter = 0;

static void *thread_main(void *arg) {
    ACL_FIBER_EVENT *ev = (ACL_FIBER_EVENT *)arg;
    
    for (int i = 0; i < 1000000; ++i) {
        if (acl_fiber_event_wait(ev) == -1) abort();
        counter++;  /* Critical section */
        if (acl_fiber_event_notify(ev) == -1) abort();
    }
    return NULL;
}

int main(int argc, char *argv[]) {
    /* Create event with mutex mode for high thread contention */
    ACL_FIBER_EVENT *event = acl_fiber_event_create(FIBER_FLAG_USE_MUTEX);
    
    const int NTHREADS = 4;
    pthread_t tids[NTHREADS];
    
    for (int i = 0; i < NTHREADS; ++i) {
        pthread_create(&tids[i], NULL, thread_main, event);
    }
    
    for (int i = 0; i < NTHREADS; ++i) {
        pthread_join(tids[i], NULL);
    }
    
    printf("final counter = %lld\n", counter);
    acl_fiber_event_free(event);  /* Cleanup */
    return 0;
}

Compile with: gcc -I./c/include -L./c/src -lfiber -pthread -o event_demo main.c

Choosing Between Atomic and Mutex Modes

The fiber_event implementation offers two distinct locking strategies selected via the flag parameter:

• Atomic mode (flag = 0): Uses lock-free atomics (atomic_int64_cas) for both the main lock and internal protection. This mode delivers optimal performance when fewer than 100 threads contend, eliminating kernel context switches during acquisition.
• Mutex mode (FIBER_FLAG_USE_MUTEX): Employs pthread_mutex_t for internal field protection. According to the source in fiber_event.c, this prevents the thundering herd problem when hundreds of threads compete, as the POSIX mutex queuing semantics integrate with the fiber wait queue.

Select atomic mode for fiber-to-fiber coordination within a single scheduler, and mutex mode when synchronizing across numerous native threads.

Summary

• Create events using acl_fiber_event_create() with FIBER_FLAG_USE_MUTEX for high-contention thread pools or 0 for lightweight fiber coordination.
• Acquire locks via acl_fiber_event_wait() for blocking semantics or acl_fiber_event_trywait() for non-blocking attempts.
• Release always with acl_fiber_event_notify() to transfer ownership to the next FIFO waiter and reset the atomic flag.
• Cleanup resources using acl_fiber_event_free() to release atomics and destroy internal mutexes.
• Thread safety is guaranteed across both native threads and fibers, with ownership validation preventing erroneous releases.

Frequently Asked Questions

What is the difference between atomic mode and mutex mode in fiber_event?

Atomic mode uses lock-free compare-and-swap operations on event->atomic and a secondary atomic lock, providing zero kernel overhead for uncontended cases. Mutex mode incorporates a pthread_mutex_t within the ACL_FIBER_EVENT union to serialize access to the waiter ring and owner fields when hundreds of threads contend, preventing cache thrashing and thundering herds.

Can fiber_event be used across native threads and fibers simultaneously?

Yes. The implementation in c/src/sync/fiber_event.c tracks both fiber ownership (owner pointer) and thread ownership (tid field), allowing seamless synchronization between native pthreads and scheduled fibers. The API automatically detects the execution context and uses the appropriate FIBER_BASE representation for blocking.

How does fiber_event prevent the thundering herd problem?

Unlike POSIX condition variables that wake all waiters simultaneously, acl_fiber_event_notify() pops exactly one waiter from the waiters ring (managed via ring_pop_head()) and signals only that specific FIBER_BASE through fbase_event_wakeup(). This owner-based wake-up ensures FIFO ordering and eliminates spurious wake-ups for threads that cannot yet acquire the lock.

What happens if I forget to call acl_fiber_event_notify after wait?

Failing to call acl_fiber_event_notify() leaves the atomic flag set to 1 and the owner field populated, causing permanent deadlock. Subsequent calls to acl_fiber_event_wait() will enter the slow path and block indefinitely because the lock appears held. Always pair wait operations with notify operations, even when using FIBER_FLAG_USE_FATAL to force process termination on errors.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}