How to Use fiber_mutex for Cross-Fiber and Cross-Thread Synchronization in libfiber

The fiber_mutex primitive in iqiyi/libfiber combines a POSIX pthread_mutex_t with a fiber-aware private lock to enable safe synchronization between cooperative fibers within the same thread and preemptive OS threads across different cores.

The fiber_mutex API in the iqiyi/libfiber library provides a unified synchronization primitive that bridges cooperative multitasking (fibers) and preemptive OS threads. By using fiber_mutex for cross-fiber and cross-thread synchronization, developers can protect critical sections accessed by both execution models without maintaining separate locking mechanisms.

Understanding the fiber_mutex Architecture

Dual-Lock Design

The implementation in c/src/sync/fiber_mutex.c employs two distinct locks to handle mixed contexts. The thread_lock is a standard POSIX pthread_mutex_t initialized at line 55 in acl_fiber_mutex_create, which serializes access among native threads. The second component, the private lock (accessed via LOCK(m) and UNLOCK(m) macros at lines 76-86), protects internal bookkeeping such as the waiter list and owner field while fibers are being scheduled.

Owner Tracking for Mixed Contexts

The mutex tracks ownership through a single owner field that distinguishes between fiber and thread contexts. A positive value indicates a fiber ID, while a negative value represents a thread ID. This dual representation, implemented in the locking logic at line 567 of acl_fiber_mutex_lock, allows the same mutex to synchronize both fibers and native threads safely.

Creating and Configuring fiber_mutex

Initialization Flags

When calling acl_fiber_mutex_create, you can combine flags defined in c/include/fiber/fiber_mutex.h (lines 12-14) to control behavior:

• FIBER_MUTEX_F_LOCK_TRY: Uses the try-lock algorithm, allowing fibers to yield if the mutex is busy (default behavior).
• FIBER_MUTEX_F_LOCK_ONCE: Forces the lock to be taken only once per holder without retry logic.
• FIBER_MUTEX_F_CHECK_DEADLOCK: Enables deadlock detection by tracking thread waiters in mutex->waiting_threads.

The implementation normalizes these flags at lines 44-48 of fiber_mutex.c to ensure either TRY or ONCE is always set.

Example creation with deadlock detection:

#include "fiber/fiber_mutex.h"

ACL_FIBER_MUTEX *mx = acl_fiber_mutex_create(
    FIBER_MUTEX_F_LOCK_TRY | FIBER_MUTEX_F_CHECK_DEADLOCK);

Locking and Unlocking Operations

Blocking Acquisition

The acl_fiber_mutex_lock function at line 567 in c/src/sync/fiber_mutex.c implements a two-tier approach:

1. Thread-level fast path: Attempts pthread_mutex_trylock on the thread_lock. If successful, the calling thread gains exclusive ownership immediately.
2. Fiber-level path: If the context is a fiber (determined by var_hook_sys_api), the function registers the fiber in mutex->waiters, yields to the scheduler, and resumes when the mutex becomes available.

Non-Blocking Attempts

For scenarios requiring immediate feedback, acl_fiber_mutex_trylock (line 590) attempts to acquire the thread_lock without blocking. It returns immediately with success or failure status, bypassing the fiber wait queue entirely.

Basic usage example:

static ACL_FIBER_MUTEX *mx;

static void *worker(void *arg)
{
    (void) arg;
    acl_fiber_mutex_lock(mx);          // block until we own the mutex
    printf("Fiber %u has the lock\n", acl_fiber_self());
    /* critical section */
    acl_fiber_mutex_unlock(mx);
    return NULL;
}

int main(void)
{
    mx = acl_fiber_mutex_create(FIBER_MUTEX_F_LOCK_TRY);
    acl_fiber_create(worker, NULL);
    acl_fiber_create(worker, NULL);
    acl_fiber_schedule();              // start the scheduler
    acl_fiber_mutex_free(mx);
    return 0;
}

Cross-Thread and Cross-Fiber Synchronization

Same Thread, Different Fibers

When multiple fibers within the same OS thread contend for a fiber_mutex, the thread_lock is already held by that thread. The implementation falls back to the fiber-level queue in mutex->waiters, managed by the private lock macros (LOCK/UNLOCK). Fibers that cannot obtain the mutex yield immediately, allowing the scheduler to run other fibers until the owner releases the lock via acl_fiber_mutex_unlock.

Different OS Threads

For contention across different OS threads, each thread attempts to acquire the thread_lock (POSIX mutex) first. If successful, the thread becomes the owner (recorded as a negative value) and enters the critical section without fiber-specific overhead. If the thread_lock is busy, threads block on the POSIX mutex, while fibers yield to their scheduler. This dual behavior ensures that fiber_mutex operates efficiently in both cooperative and preemptive contexts.

Example mixing fibers and pthreads:

#include "fiber/fiber_mutex.h"
#include <pthread.h>

static ACL_FIBER_MUTEX *mx;

void *native_thread(void *arg)
{
    (void) arg;
    if (acl_fiber_mutex_lock(mx) == 0) {
        printf("Native thread %lu got the lock\n", pthread_self());
        /* critical section */
        acl_fiber_mutex_unlock(mx);
    }
    return NULL;
}

static void *fiber_task(void *arg)
{
    (void) arg;
    acl_fiber_mutex_lock(mx);
    printf("Fiber %u got the lock\n", acl_fiber_self());
    /* critical section */
    acl_fiber_mutex_unlock(mx);
    return NULL;
}

int main(void)
{
    mx = acl_fiber_mutex_create(FIBER_MUTEX_F_LOCK_TRY |
                                FIBER_MUTEX_F_CHECK_DEADLOCK);
    pthread_t th;
    pthread_create(&th, NULL, native_thread, NULL);
    acl_fiber_create(fiber_task, NULL);
    acl_fiber_schedule();          // runs the fiber side
    pthread_join(th, NULL);
    acl_fiber_mutex_free(mx);
    return 0;
}

Deadlock Detection

Enabling FIBER_MUTEX_F_CHECK_DEADLOCK

To enable deadlock detection, pass the FIBER_MUTEX_F_CHECK_DEADLOCK flag to acl_fiber_mutex_create. When this flag is set, the implementation records blocking threads in mutex->waiting_threads using thread_waiter_add and thread_waiter_remove (found in the lock/unlock paths).

Detecting and Diagnosing Deadlocks

The acl_fiber_mutex_deadlock function (line 317 in c/src/sync/fiber_mutex.c) walks the global list __locks and constructs a wait-for graph of owners and waiting mutexes. If a cycle is detected, it returns an ACL_FIBER_MUTEX_STATS structure containing the deadlock information. Use acl_fiber_mutex_stats_show to print human-readable diagnostics, and acl_fiber_mutex_stats_free to release the structure.

Example deadlock detection:

/* Assume two mutexes m1, m2 with deadlock detection enabled */
ACL_FIBER_MUTEX *m1 = acl_fiber_mutex_create(FIBER_MUTEX_F_LOCK_TRY |
                                            FIBER_MUTEX_F_CHECK_DEADLOCK);
ACL_FIBER_MUTEX *m2 = acl_fiber_mutex_create(FIBER_MUTEX_F_LOCK_TRY |
                                            FIBER_MUTEX_F_CHECK_DEADLOCK);

/* Thread A locks m1 then tries m2 */
pthread_create(&t1, NULL, thread_func_a, NULL);
/* Thread B locks m2 then tries m1 */
pthread_create(&t2, NULL, thread_func_b, NULL);

/* After some time we ask libfiber to dump the state */
sleep(1);
ACL_FIBER_MUTEX_STATS *stats = acl_fiber_mutex_deadlock();
if (stats) {
    acl_fiber_mutex_stats_show(stats);
    acl_fiber_mutex_stats_free(stats);
}

Summary

• Dual-lock architecture: fiber_mutex combines a POSIX pthread_mutex_t (thread_lock) for thread-level serialization with a private lock for fiber wait queues.
• Universal context support: The owner field distinguishes fiber IDs (positive) from thread IDs (negative), enabling the same mutex to synchronize both fibers and native threads.
• Flexible locking modes: Flags FIBER_MUTEX_F_LOCK_TRY and FIBER_MUTEX_F_LOCK_ONCE control whether fibers yield or spin when contending for the lock.
• Built-in deadlock detection: The FIBER_MUTEX_F_CHECK_DEADLOCK flag enables runtime cycle detection via acl_fiber_mutex_deadlock and diagnostic output via acl_fiber_mutex_stats_show.
• Key source files: Implementation resides in c/src/sync/fiber_mutex.c with public API declarations in c/include/fiber/fiber_mutex.h.

Frequently Asked Questions

Can fiber_mutex be used from standard pthreads without any fiber context?

Yes. The fiber_mutex implementation in c/src/sync/fiber_mutex.c explicitly handles native threads. When var_hook_sys_api is false (indicating a standard pthread context), the code path falls back to the POSIX thread_lock without fiber-specific yielding. The owner field records the thread ID as a negative value, ensuring proper ownership tracking even in pure pthread environments.

What is the performance cost of using fiber_mutex compared to a standard pthread_mutex?

For native thread contention, the cost is nearly identical because fiber_mutex delegates to the underlying pthread_mutex_t (thread_lock). For fiber-only contention within the same thread, the overhead involves the private lock macros (LOCK/UNLOCK) and fiber wait queue management in mutex->waiters, which is lighter than a kernel context switch but heavier than a simple spinlock. The hybrid design optimizes for the common case where fibers cooperate within a thread while remaining safe across threads.

How does fiber_mutex handle priority inversion or starvation between fibers and threads?

The current implementation in c/src/sync/fiber_mutex.c does not implement explicit priority inheritance or priority ceiling protocols. Fibers waiting on a mutex are queued in mutex->waiters and resumed in FIFO order when the owner calls acl_fiber_mutex_unlock. Native threads block on the POSIX mutex and are scheduled by the kernel. To avoid starvation, design your application such that critical sections are brief and consider using FIBER_MUTEX_F_LOCK_TRY to allow fibers to perform other work if the mutex is contended.

Can I recursively lock the same fiber_mutex from the same fiber or thread?

No, fiber_mutex does not support recursive locking. The implementation tracks a single owner (either fiber ID or thread ID) and treats any subsequent lock attempt from the same owner as a contention case. If a fiber or thread attempts to re-acquire a mutex it already holds, it will deadlock itself (or be detected by the deadlock checker if FIBER_MUTEX_F_CHECK_DEADLOCK is enabled). Design your code to release the mutex before re-entering the critical section or use a separate recursive locking primitive if needed.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}