How the libfiber API Hook Mechanism Replaces Standard System Calls on Linux/BSD

The libfiber API hook mechanism replaces standard system calls on Linux and BSD systems by using dynamic linker interposition to cache original libc function pointers via dlsym(RTLD_NEXT, …), then wrapping them with fiber-aware implementations that integrate with the library's cooperative scheduling engine.

The iqiyi/libfiber library implements a transparent API hook mechanism that intercepts standard POSIX networking and I/O functions without requiring application source code changes. By leveraging dynamic linker interposition techniques on Linux and BSD platforms, the library replaces blocking system calls with fiber-aware alternatives that yield control to the library's scheduler instead of blocking the underlying thread.

Core Architecture of the API Hook Mechanism

Function Pointer Abstraction in hook.c

In c/src/hook/hook.c, the library defines dual function pointer variables for each system call it intercepts. A static internal pointer (e.g., __sys_socket) caches the original libc implementation, while a public pointer (sys_socket) allows external access to the current implementation.

static socket_fn __sys_socket = NULL;
socket_fn *sys_socket = NULL;

Symbol Resolution via dlsym and RTLD_NEXT

The LOAD_FN macro in c/src/hook/hook.c uses dlsym(RTLD_NEXT, …) to locate the next occurrence of a symbol in the dynamic link chain—specifically, the real libc implementation. This technique prevents infinite recursion by ensuring the wrapper calls the original function, not itself.

#define LOAD_FN(name, type, fn, fp, fatal) do {           \
    (fn) = (type) dlsym(RTLD_NEXT, name);               \
    if ((fn) == NULL) {                                 \
        const char* e = dlerror();                      \
        printf("%s(%d): name=%s not found: %s\r\n",      \
            __FUNCTION__, __LINE__, name, e ? e:"unknown"); \
        assert((fatal) != 1);                           \
    }                                                   \
    (fp) = &(fn);                                       \
} while (0)

The hook_api() function enumerates all intercepted calls—including socket, close, read, write, poll, epoll, and others—and invokes LOAD_FN for each to populate the function pointer table.

Thread-Safe Initialization with pthread_once

To ensure the hook initialization occurs exactly once regardless of concurrent access, hook_once() in c/src/hook/hook.c uses pthread_once with a static control variable __once_control. Every wrapper function (such as acl_fiber_socket in c/src/hook/socket.c) checks if sys_socket is NULL and invokes hook_once() to guarantee initialization completes before first use.

Runtime Control and Mode Switching

The var_hook_sys_api Flag

Located in c/src/fiber.c, the thread-local variable var_hook_sys_api acts as a runtime switch for the API hook mechanism:

• When set to 0, wrapper functions bypass libfiber's logic and directly invoke the cached original system call via (*sys_socket)(…).
• When set to 1, the wrapper executes fiber-aware logic—such as setting non-blocking mode and registering with the event loop—before delegating to the original implementation.

Public API for Custom Implementations

The library exposes setter functions including set_socket_fn(), set_close_fn(), and similar methods that allow external code to replace the function pointers with custom implementations (useful for testing or specialized instrumentation). These setters directly assign the global pointer:

void WINAPI set_socket_fn(socket_fn *fn) {
    sys_socket = fn;
}

Deployment via LD_PRELOAD

The API hook mechanism activates transparently when libfiber is compiled as a shared library and loaded via the LD_PRELOAD environment variable. The dynamic linker resolves symbols like socket() to the wrapper in libfiber.so first. The wrapper then forwards to the cached original function (__sys_socket) via the function pointer, achieving interception without modifying application source code.

Practical Example: Hooked socket() Workflow

Consider a standard client application that calls socket():

/* client.c – runs with libfiber pre-loaded */
#include <stdio.h>
#include <unistd.h>
#include <sys/socket.h>
#include <arpa/inet.h>

int main(void) {
    int fd = socket(AF_INET, SOCK_STREAM, 0);   /* acl_fiber_socket called */
    if (fd < 0) {
        perror("socket");
        return 1;
    }
    printf("socket created: %d\n", fd);
    close(fd);                                 /* acl_fiber_close called */
    return 0;
}

Execution flow when running with LD_PRELOAD=./libfiber.so ./client:

1. The dynamic linker resolves socket() to acl_fiber_socket in libfiber.so.
2. acl_fiber_socket detects sys_socket == NULL and invokes hook_once().
3. hook_once() executes hook_api(), which uses dlsym(RTLD_NEXT, "socket") to load the real socket from libc into __sys_socket and sets sys_socket = &__sys_socket.
4. If var_hook_sys_api is enabled, the wrapper sets the socket to non-blocking and registers it with the fiber scheduler before calling (*sys_socket)(AF_INET, SOCK_STREAM, 0).
5. If var_hook_sys_api is disabled, the wrapper directly calls (*sys_socket)(…), effectively bypassing fiber-aware logic.

Key Source Files Reference

File	Role
[c/src/hook/hook.c](https://github.com/iqiyi/libfiber/blob/master/c/src/hook/hook.c)	Core interposition logic: defines static/public pointers, LOAD_FN macro, hook_api(), and hook_once().
[c/src/hook/socket.c](https://github.com/iqiyi/libfiber/blob/master/c/src/hook/socket.c)	Wrapper implementations for socket(), listen(), accept(), etc.; demonstrates hook_once() guard and var_hook_sys_api logic.
[c/src/hook/select.c](https://github.com/iqiyi/libfiber/blob/master/c/src/hook/select.c)	Wrapper for select() multiplexing calls.
[c/src/hook/poll.c](https://github.com/iqiyi/libfiber/blob/master/c/src/hook/poll.c)	Wrapper for poll() with fiber-specific timer handling.
[c/src/fiber.c](https://github.com/iqiyi/libfiber/blob/master/c/src/fiber.c)	Contains thread-local var_hook_sys_api flag and hook_once() invocation logic.
[c/src/hook/getaddrinfo.c](https://github.com/iqiyi/libfiber/blob/master/c/src/hook/getaddrinfo.c)	DNS resolution interposition using the same hook pattern.
[c/src/hook/epoll.c](https://github.com/iqiyi/libfiber/blob/master/c/src/hook/epoll.c)	epoll wrapper integrating with libfiber's I/O scheduler.

Summary

• Dynamic linker interposition via dlsym(RTLD_NEXT, …) in c/src/hook/hook.c captures original libc function pointers without source code modification.
• Thread-safe initialization using pthread_once ensures hook setup occurs exactly once, even under concurrent access.
• Runtime mode switching through the thread-local var_hook_sys_api flag allows transparent fallback to native syscalls or fiber-aware interception.
• LD_PRELOAD deployment enables transparent interception of networking and I/O calls in unmodified applications.
• Modular wrapper architecture separates concerns across socket.c, poll.c, epoll.c, and other files, each implementing specific syscall families while sharing the common hook infrastructure.

Frequently Asked Questions

What is the difference between __sys_socket and sys_socket in libfiber?

__sys_socket is a static internal pointer that caches the original libc socket implementation loaded via dlsym, while sys_socket is a public global pointer that external code and wrapper functions dereference to invoke the current implementation. This dual-pointer design allows the library to swap implementations at runtime while maintaining a stable reference to the original system call.

How does libfiber prevent infinite recursion when hooking system calls?

The library prevents recursion by using dlsym(RTLD_NEXT, …) to obtain a pointer to the next symbol in the dynamic link chain—specifically the real libc implementation—rather than calling the wrapper function again. By storing this pointer in __sys_socket (and similar variables) during initialization and invoking (*sys_socket)(…) instead of socket(), the wrapper delegates to the original function, avoiding self-reference.

Can I disable the API hook mechanism at runtime without recompiling?

Yes, the API hook mechanism can be toggled at runtime using the thread-local flag var_hook_sys_api defined in c/src/fiber.c. When this flag is set to 0, wrapper functions bypass libfiber's fiber-aware logic and directly invoke the cached original system calls. External code can manipulate this flag through the library's public API to temporarily disable interception for specific threads or code sections.

Why does libfiber use pthread_once for initialization instead of a simple static initializer?

pthread_once guarantees that hook_api() executes exactly once across all threads, even if multiple threads concurrently invoke wrapped system calls before initialization completes. A simple static initializer cannot atomically perform the complex multi-step setup required—loading dozens of symbols via dlsym, assigning function pointers, and handling potential errors—without risking race conditions or double-initialization. The pthread_once pattern ensures thread-safe, lazy initialization of the hook infrastructure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}