How the Libfiber Fiber Scheduler Handles Multiple Threads: Per-Thread Architecture Explained

Libfiber implements a per-thread fiber scheduler where each OS thread maintains its own isolated scheduler instance through thread-local storage, ensuring fibers never migrate between threads.

The iqiyi/libfiber library provides cooperative multitasking through user-space fibers, but understanding how the fiber scheduler handles multiple threads requires examining its thread-local architecture. Unlike work-stealing schedulers, libfiber assigns each native OS thread its own scheduler state, ready queue, and fiber context, guaranteeing thread affinity throughout a fiber's lifecycle.

Thread-Local Scheduler Architecture

Core Thread-Local Variables

In c/src/fiber.c (lines 52-55), the scheduler stores all per-thread state in the __thread_fiber variable, a thread-local THREAD structure containing the ready queue, dead queue, and currently running fiber reference. A companion flag, __scheduled (lines 53-55), acts as a reentrancy guard to prevent nested scheduler invocation on the same thread.

Lazy Initialization with thread_init()

When a thread first calls any fiber API, the thread_init() function (lines 30-45 in c/src/fiber.c) automatically allocates the THREAD structure, configures the shared-stack if enabled, and registers the thread-local key. This lazy initialization ensures zero overhead for threads that never create fibers while guaranteeing immediate scheduler readiness for those that do.

The Scheduling Loop in acl_fiber_schedule()

The core scheduling routine acl_fiber_schedule() (C API) or fiber::schedule() (C++) implements the execution engine. Located in c/src/fiber.c (lines 25-53), the function first checks __scheduled to avoid recursive entry (lines 25-27).

Once confirmed, the scheduler enters a continuous loop that:

1. Pops the next ready fiber from __thread_fiber->ready
2. Updates the fiber status to READY and stores it in __thread_fiber->running
3. Invokes fiber_swap() to context-switch from the OS thread to the fiber's execution context
4. Returns to the loop when the fiber yields or terminates, processing the next ready fiber

When the ready queue empties, the scheduler cleans dead fibers, clears I/O hooks, and resets __scheduled to 0.

Creating Fibers Across Multiple Threads

Automatic Per-Thread Isolation

Every fiber created via fiber_create or the C++ go helper remains bound to its originating thread. Because __thread_fiber is thread-local, fibers queued in thread A's ready queue can never execute on thread B's scheduler. This design eliminates cross-thread synchronization overhead for fiber scheduling decisions.

Cross-Thread Fiber Creation with go_wait_thread

For explicit multi-threading, the C++ API provides go_wait_thread in cpp/include/fiber/go_fiber.hpp (lines 66-73). This helper overloads operator<< to launch a lambda in a new std::thread, where the per-thread scheduler initializes lazily and runs the fiber independently from the parent thread's scheduler.

Fiber Pools and Thread Confinement

The fiber_pool class (implemented in cpp/src/fiber_pool.cpp, lines 64-71) creates multiple fibers within a single OS thread. To achieve parallelism, instantiate one pool per thread. The pool's internal fiber_create calls use the same thread-local scheduler mechanism, ensuring all pooled fibers execute within their creator's thread context.

Practical Example: Running Schedulers in Parallel

#include <acl/lib_fiber.hpp>
#include <iostream>
#include <thread>

void fiber_task(const char* name) {
    std::cout << "Fiber " << name << " running on thread "
              << std::this_thread::get_id() << "\n";
    for (int i = 0; i < 3; ++i) {
        std::cout << "Fiber " << name << " step " << i << "\n";
        acl::fiber::yield();
    }
}

void thread_worker(const char* label) {
    acl::fiber::schedule_init(true);
    go[&] { fiber_task(label); };
    go[&] { fiber_task(label); };
    acl::fiber::schedule();
}

int main() {
    std::thread t1(thread_worker, "A");
    std::thread t2(thread_worker, "B");
    t1.join();
    t2.join();
    return 0;
}

Each thread_worker initializes auto-scheduling, creates fibers in the local ready queue, and invokes acl::fiber::schedule() to run the independent scheduling loop. The two OS threads execute their fiber schedulers in parallel without sharing state.

Summary

• Per-thread isolation: Each OS thread maintains a private THREAD structure via __thread_fiber, ensuring complete scheduler separation as implemented in c/src/fiber.c.
• Lazy initialization: The thread_init() function allocates scheduler state only when a thread first creates a fiber, minimizing resource overhead.
• Non-migratory fibers: Fibers created in a specific thread remain in that thread's ready queue permanently, eliminating cross-thread migration complexity.
• Manual scheduling control: Call acl_fiber_schedule() or fiber::schedule() explicitly, or use go_wait_thread to spawn new OS threads with embedded schedulers.

Frequently Asked Questions

Can fibers migrate between threads in libfiber?

No. According to the source code in c/src/fiber.c, fibers are permanently bound to their creating thread through the thread-local __thread_fiber structure. The ready queue is thread-specific, and fiber_swap() never transfers execution contexts across OS threads.

How do I run fibers on multiple CPU cores?

Create multiple std::thread instances and invoke acl::fiber::schedule() in each one. Use go_wait_thread (lines 66-73 in cpp/include/fiber/go_fiber.hpp) to launch fiber workloads in new threads automatically, or manually manage one fiber_pool per thread as shown in cpp/src/fiber_pool.cpp (lines 64-71).

What happens if I call acl_fiber_schedule() twice on the same thread?

The scheduler returns immediately. The __scheduled flag (lines 25-27 in c/src/fiber.c) prevents recursive entry, protecting the scheduling loop from reentrant calls while fibers are already being dispatched.

Is there a global scheduler or master thread?

No. Libfiber explicitly avoids global scheduler state. Every thread that creates fibers gets an independent scheduler instance through the thread_init() mechanism in c/src/fiber.c, allowing true parallel execution without central bottlenecks.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}