# How the Libfiber Fiber Scheduler Handles Multiple Threads: Per-Thread Architecture Explained

> Discover how the libfiber fiber scheduler utilizes a per-thread architecture for efficient multi-threaded execution. Learn how isolated schedulers prevent fiber migration.

- Repository: [iQIYI/libfiber](https://github.com/iqiyi/libfiber)
- Tags: internals
- Published: 2026-03-04

---

**Libfiber implements a per-thread fiber scheduler where each OS thread maintains its own isolated scheduler instance through thread-local storage, ensuring fibers never migrate between threads.**

The iqiyi/libfiber library provides cooperative multitasking through user-space fibers, but understanding how the fiber scheduler handles multiple threads requires examining its thread-local architecture. Unlike work-stealing schedulers, libfiber assigns each native OS thread its own scheduler state, ready queue, and fiber context, guaranteeing thread affinity throughout a fiber's lifecycle.

## Thread-Local Scheduler Architecture

### Core Thread-Local Variables

In [`c/src/fiber.c`](https://github.com/iqiyi/libfiber/blob/main/c/src/fiber.c) (lines 52-55), the scheduler stores all per-thread state in the `__thread_fiber` variable, a thread-local `THREAD` structure containing the **ready queue**, **dead queue**, and currently running fiber reference. A companion flag, `__scheduled` (lines 53-55), acts as a reentrancy guard to prevent nested scheduler invocation on the same thread.

### Lazy Initialization with thread_init()

When a thread first calls any fiber API, the `thread_init()` function (lines 30-45 in [`c/src/fiber.c`](https://github.com/iqiyi/libfiber/blob/main/c/src/fiber.c)) automatically allocates the `THREAD` structure, configures the shared-stack if enabled, and registers the thread-local key. This lazy initialization ensures zero overhead for threads that never create fibers while guaranteeing immediate scheduler readiness for those that do.

## The Scheduling Loop in acl_fiber_schedule()

The core scheduling routine `acl_fiber_schedule()` (C API) or `fiber::schedule()` (C++) implements the execution engine. Located in [`c/src/fiber.c`](https://github.com/iqiyi/libfiber/blob/main/c/src/fiber.c) (lines 25-53), the function first checks `__scheduled` to avoid recursive entry (lines 25-27).

Once confirmed, the scheduler enters a continuous loop that:

1. Pops the next ready fiber from `__thread_fiber->ready`
2. Updates the fiber status to **READY** and stores it in `__thread_fiber->running`
3. Invokes `fiber_swap()` to context-switch from the OS thread to the fiber's execution context
4. Returns to the loop when the fiber yields or terminates, processing the next ready fiber

When the ready queue empties, the scheduler cleans dead fibers, clears I/O hooks, and resets `__scheduled` to 0.

## Creating Fibers Across Multiple Threads

### Automatic Per-Thread Isolation

Every fiber created via `fiber_create` or the C++ `go` helper remains bound to its originating thread. Because `__thread_fiber` is thread-local, fibers queued in thread A's ready queue can never execute on thread B's scheduler. This design eliminates cross-thread synchronization overhead for fiber scheduling decisions.

### Cross-Thread Fiber Creation with go_wait_thread

For explicit multi-threading, the C++ API provides `go_wait_thread` in [`cpp/include/fiber/go_fiber.hpp`](https://github.com/iqiyi/libfiber/blob/main/cpp/include/fiber/go_fiber.hpp) (lines 66-73). This helper overloads `operator<<` to launch a lambda in a new `std::thread`, where the per-thread scheduler initializes lazily and runs the fiber independently from the parent thread's scheduler.

### Fiber Pools and Thread Confinement

The `fiber_pool` class (implemented in [`cpp/src/fiber_pool.cpp`](https://github.com/iqiyi/libfiber/blob/main/cpp/src/fiber_pool.cpp), lines 64-71) creates multiple fibers within a single OS thread. To achieve parallelism, instantiate one pool per thread. The pool's internal `fiber_create` calls use the same thread-local scheduler mechanism, ensuring all pooled fibers execute within their creator's thread context.

## Practical Example: Running Schedulers in Parallel

```cpp
#include <acl/lib_fiber.hpp>
#include <iostream>
#include <thread>

void fiber_task(const char* name) {
    std::cout << "Fiber " << name << " running on thread "
              << std::this_thread::get_id() << "\n";
    for (int i = 0; i < 3; ++i) {
        std::cout << "Fiber " << name << " step " << i << "\n";
        acl::fiber::yield();
    }
}

void thread_worker(const char* label) {
    acl::fiber::schedule_init(true);
    go[&] { fiber_task(label); };
    go[&] { fiber_task(label); };
    acl::fiber::schedule();
}

int main() {
    std::thread t1(thread_worker, "A");
    std::thread t2(thread_worker, "B");
    t1.join();
    t2.join();
    return 0;
}

```

Each `thread_worker` initializes auto-scheduling, creates fibers in the local ready queue, and invokes `acl::fiber::schedule()` to run the independent scheduling loop. The two OS threads execute their fiber schedulers in parallel without sharing state.

## Summary

- **Per-thread isolation**: Each OS thread maintains a private `THREAD` structure via `__thread_fiber`, ensuring complete scheduler separation as implemented in [`c/src/fiber.c`](https://github.com/iqiyi/libfiber/blob/main/c/src/fiber.c).
- **Lazy initialization**: The `thread_init()` function allocates scheduler state only when a thread first creates a fiber, minimizing resource overhead.
- **Non-migratory fibers**: Fibers created in a specific thread remain in that thread's ready queue permanently, eliminating cross-thread migration complexity.
- **Manual scheduling control**: Call `acl_fiber_schedule()` or `fiber::schedule()` explicitly, or use `go_wait_thread` to spawn new OS threads with embedded schedulers.

## Frequently Asked Questions

### Can fibers migrate between threads in libfiber?

No. According to the source code in [`c/src/fiber.c`](https://github.com/iqiyi/libfiber/blob/main/c/src/fiber.c), fibers are permanently bound to their creating thread through the thread-local `__thread_fiber` structure. The ready queue is thread-specific, and `fiber_swap()` never transfers execution contexts across OS threads.

### How do I run fibers on multiple CPU cores?

Create multiple `std::thread` instances and invoke `acl::fiber::schedule()` in each one. Use `go_wait_thread` (lines 66-73 in [`cpp/include/fiber/go_fiber.hpp`](https://github.com/iqiyi/libfiber/blob/main/cpp/include/fiber/go_fiber.hpp)) to launch fiber workloads in new threads automatically, or manually manage one `fiber_pool` per thread as shown in [`cpp/src/fiber_pool.cpp`](https://github.com/iqiyi/libfiber/blob/main/cpp/src/fiber_pool.cpp) (lines 64-71).

### What happens if I call acl_fiber_schedule() twice on the same thread?

The scheduler returns immediately. The `__scheduled` flag (lines 25-27 in [`c/src/fiber.c`](https://github.com/iqiyi/libfiber/blob/main/c/src/fiber.c)) prevents recursive entry, protecting the scheduling loop from reentrant calls while fibers are already being dispatched.

### Is there a global scheduler or master thread?

No. Libfiber explicitly avoids global scheduler state. Every thread that creates fibers gets an independent scheduler instance through the `thread_init()` mechanism in [`c/src/fiber.c`](https://github.com/iqiyi/libfiber/blob/main/c/src/fiber.c), allowing true parallel execution without central bottlenecks.