How libfiber's Scheduler Works Internally: Cooperative Fiber Scheduling in C

Libfiber implements a cooperative, per-thread scheduler that drives user-level fibers through a thread-local state machine, FIFO ready queues, and low-level assembly context switches.

The iqiyi/libfiber library provides high-performance user-space concurrency for C and C++ applications. Understanding how libfiber's scheduler works internally reveals a cooperative multitasking design where each OS thread maintains isolated fiber queues and manually swaps execution contexts via platform-specific assembly routines.

Thread-Local Scheduler State

Every OS thread using libfiber owns a private THREAD object stored in the thread-local variable __thread_fiber. Defined in c/src/fiber.c at line 24, this structure maintains two critical FIFO rings: a ready queue for fibers eligible to run and a dead queue for fibers awaiting cleanup. The structure also tracks a global fibers[] array containing all created fiber objects and bookkeeping flags for scheduler state.

When you create a fiber via acl_fiber_create2, the implementation calls fiber_alloc to initialize the object. The new fiber enters the fibers[] array and immediately joins the ready ring through the FIBER_READY macro, making it eligible for scheduling on the next loop iteration.

Automatic Scheduling Initialization

The acl_fiber_schedule_init function (line 997 in c/src/fiber.c) configures the thread-local __schedule_auto flag. When enabled, creating the first fiber automatically triggers an implicit call to acl_fiber_schedule(), starting the scheduler without explicit user intervention.

The Main Scheduling Loop

The acl_fiber_schedule function (line 200 in c/src/fiber.c) serves as the engine of libfiber's cooperative multitasking. The implementation follows a strict seven-phase protocol:

1. Re-entrancy Guard: Checks __scheduled to prevent nested scheduler calls on the same thread.
2. State Transition: Marks the thread as actively scheduled.
3. Fiber Selection: Repeatedly pops the head of the ready ring using ring_pop_head.
4. Status Assignment: Marks the popped fiber as FIBER_STATUS_READY and assigns it to __thread_fiber->running.
5. Context Activation: Calls fiber_swap(original, fiber) to transfer CPU control to the target fiber's stack.
6. Continuation Loop: After the fiber yields or exits, execution returns to step 3 to process the next ready fiber.
7. Termination Cleanup: When the ready queue empties, all fibers in the dead queue are freed, I/O buffers are cleared, and the __scheduled flag is reset.

Context Switching and Fiber Lifecycle

Switching Execution Contexts

The fiber_swap function (line 445 in c/src/fiber.c) handles the mechanical transfer of execution between fibers. When the current fiber is exiting, fiber_swap moves it to the dead ring for later reuse or deallocation. The target fiber's status transitions to FIBER_STATUS_RUNNING before the low-level assembly routine fiber_real_swap saves the caller's registers and jumps to the target's saved stack pointer.

Cooperative Yielding

Fibers voluntarily surrender control through acl_fiber_yield (line 455 in c/src/fiber.c). This function moves the currently running fiber back to the ready ring via FIBER_READY, then invokes acl_fiber_switch() (line 665) to select and activate the next eligible fiber. The acl_fiber_switch implementation serves identical logic for internal yield points, such as after I/O completion events.

Event-Driven I/O Integration

Libfiber's scheduler abstracts underlying event mechanisms through acl_fiber_schedule_set_event and acl_fiber_schedule_with. The implementation supports poll, select, Windows message loops, and io_uring backends. The selected event type stores in a global variable consulted by the I/O subsystem (event_set), allowing fibers to block on network operations without blocking the underlying OS thread. When I/O completes, the waiting fiber re-enters the ready ring for rescheduling.

Stopping the Scheduler

To gracefully halt scheduling, acl_fiber_schedule_stop (line 324 in c/src/fiber.c) clears the __scheduled flag. The current scheduling loop iteration completes normally, processing any remaining ready fibers and dead-ring cleanup before returning control to normal thread execution.

C++ API Wrapper

The C++ interface in cpp/src/fiber.cpp provides object-oriented syntactic sugar over the C core. The fiber::schedule() method forwards to acl_fiber_schedule_with(type), while fiber::schedule_gui() configures Windows message pump integration by calling acl_fiber_schedule_init(1), acl_fiber_schedule_set_event(FIBER_EVENT_WMSG), and WinAPI hooks. These wrappers preserve the underlying cooperative semantics while exposing a modern C++ class interface.

Code Examples

The following C++ example demonstrates three cooperative fibers yielding control in a round-robin fashion:

// example.cpp – simple cooperative fibers
#include <fiber/fiber.hpp>

class PrintFiber : public acl::fiber {
protected:
    void run() override {
        for (int i = 0; i < 5; ++i) {
            printf("Fiber %u iteration %d\n", self(), i);
            // Give other fibers a chance to run
            acl::fiber::yield();
        }
    }
};

int main() {
    // Create three fibers
    PrintFiber f1, f2, f3;
    f1.start();   // default stack, non‑shared
    f2.start();
    f3.start();

    // Start the scheduler (uses poll by default)
    acl::fiber::schedule();   // equivalent to acl_fiber_schedule()
    return 0;
}

The equivalent C implementation creates fibers directly through the core API:

/* example.c – same idea in pure C */
#include "fiber.h"

static void worker(ACL_FIBER *f, void *arg) {
    int id = acl_fiber_self();
    for (int i = 0; i < 5; ++i) {
        printf("C fiber %d iteration %d\n", id, i);
        acl_fiber_yield();
    }
}

int main(void) {
    acl_fiber_create(worker, NULL, 64000);
    acl_fiber_create(worker, NULL, 64000);
    acl_fiber_create(worker, NULL, 64000);

    // Implicit scheduling because __schedule_auto defaults to off;
    // we kick it manually.
    acl_fiber_schedule();
    return 0;
}

Summary

• Per-thread isolation: Each OS thread maintains private THREAD state in __thread_fiber with separate ready and dead rings.
• FIFO scheduling: acl_fiber_schedule processes fibers in round-robin order from the ready ring head.
• Assembly context switches: fiber_swap uses fiber_real_swap to save registers and switch stacks at the CPU level.
• Explicit yielding: Fibers call acl_fiber_yield to re-enter the ready queue and transfer control cooperatively.
• Pluggable I/O: The scheduler integrates with poll, select, io_uring, or Windows message pumps via acl_fiber_schedule_set_event.
• C++ facade: cpp/src/fiber.cpp maps object-oriented methods to the procedural C implementation in c/src/fiber.c.

Frequently Asked Questions

How does libfiber prevent stack overflow when switching between fibers?

Each fiber allocates a dedicated stack segment during creation (defaulting to 64KB in the examples). The fiber_swap routine in c/src/fiber.c (line 445) saves the current stack pointer and registers before jumping to the target fiber's pre-allocated stack, ensuring isolated memory regions per fiber.

Can multiple OS threads run libfiber schedulers simultaneously?

Yes. Because the scheduler state lives in the thread-local __thread_fiber variable defined in c/src/fiber.c, each pthread or Windows thread can independently call acl_fiber_schedule() without locking. Fibers never migrate between OS threads; they remain bound to their creator's scheduler instance.

What happens when a fiber exits without yielding?

When a fiber's entry function returns, the internal fiber_exit path moves the fiber to the dead ring via fiber_swap (line 445). The scheduler loop detects this during the next iteration, cleans up the dead fiber's resources, and continues with the next ready fiber. No explicit yield is required for termination.

How does libfiber handle blocking I/O without blocking the entire thread?

The scheduler supports event-driven backends (poll, select, io_uring) configured through acl_fiber_schedule_with. When a fiber calls a blocking I/O operation, the library parks the fiber in a wait state and yields execution via acl_fiber_switch. The event loop polls descriptors until data arrives, then re-enqueues the fiber into the ready ring for rescheduling.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}