How to Use fiber_sem to Limit Concurrent Database Connections in libfiber

Use fiber_sem to create a counting semaphore initialized with your maximum connection count, then call wait() before opening a database connection and post() after closing it, or use fiber_sem_guard for automatic RAII-based release.

The iqiyi/libfiber library provides a lightweight, fiber-aware synchronization primitive that allows you to cap resource usage without blocking OS threads. When building high-concurrency database applications, you can leverage fiber_sem to enforce hard limits on simultaneous connections, preventing pool exhaustion and server overload. This guide explains the semaphore mechanics and implementation patterns based on the source code in cpp/include/fiber/fiber_sem.hpp and cpp/src/fiber_sem.cpp.

Understanding fiber_sem Semaphore Mechanics

The fiber_sem class implements a counting semaphore optimized for fiber-based concurrency. Unlike OS semaphores that block threads, this primitive suspends individual fibers, allowing the scheduler to multiplex thousands of concurrent operations onto a small thread pool.

Construction and Initialization

Create a global or shared semaphore by constructing fiber_sem with a maximum count and optional attributes:

#include "fiber/fiber_sem.hpp"

// Allow at most 5 concurrent DB connections
static acl::fiber_sem g_db_sem(5);

The constructor signature is fiber_sem(int max, int attr = fiber_sem_t_async). The default fiber_sem_t_async attribute creates an asynchronous semaphore: when wait() is called and no slots are available, the current fiber suspends immediately and yields control back to the scheduler rather than consuming a thread.

Acquiring and Releasing Resources

The semaphore exposes two primary operations for resource management:

• wait(int ms = -1) – Decrements the internal counter. If the counter is zero, the fiber suspends until another fiber calls post() or the specified timeout (in milliseconds) elapses, returning -1 on timeout.
• post() – Increments the counter and wakes exactly one suspended fiber if any are queued.

Because these operations are fiber-aware, context switches occur in user space without kernel thread manipulation, making them suitable for high-frequency database operations.

Implementing Database Connection Limits with fiber_sem

To enforce a connection cap, initialize the semaphore with your pool size, then wrap every database operation between wait() and post() calls. The following example demonstrates limiting concurrent queries to five, even when spawning many worker fibers:

#include <iostream>
#include <acl_cpp/thread/thread.hpp>          // fiber creation utilities
#include "fiber/fiber_sem.hpp"                // fiber_sem definition

// Global semaphore limiting concurrent DB connections
static acl::fiber_sem g_db_sem(5);

void db_query(const std::string& sql)
{
    // Simulate network/IO latency with fiber-friendly sleep
    acl::fiber::sleep(100);
    std::cout << "executed: " << sql << std::endl;
}

void worker_fiber(void* arg)
{
    const char* sql = static_cast<const char*>(arg);

    // Manual acquire/release pattern
    g_db_sem.wait();           // Block until a slot is free
    db_query(sql);             // Perform DB work
    g_db_sem.post();           // Release slot for next fiber
}

int main()
{
    const char* queries[] = {
        "SELECT * FROM users",
        "UPDATE orders SET status='ok' WHERE id=1",
        "INSERT INTO logs(msg) VALUES('test')"
    };

    // Create a fiber for each query
    for (size_t i = 0; i < sizeof(queries)/sizeof(queries[0]); ++i) {
        acl::fiber::start(worker_fiber, (void*)queries[i]);
    }

    // Enter scheduler loop and wait for completion
    acl::fiber::schedule();
    return 0;
}

In this implementation, acl::fiber::sleep() yields the fiber without blocking the underlying thread, preserving scheduler efficiency while the semaphore gates access to the simulated database layer.

RAII Pattern with fiber_sem_guard

For exception safety and cleaner resource management, use the fiber_sem_guard helper defined in fiber_sem.hpp. This RAII wrapper automatically acquires the semaphore on construction and releases it on destruction, guaranteeing that post() is called even if exceptions escape the scope.

void worker_fiber(void* arg)
{
    const char* sql = static_cast<const char*>(arg);

    {
        acl::fiber_sem_guard guard(g_db_sem);  // Constructor calls wait()
        db_query(sql);                          // Safe DB work here
    }  // Destructor calls post() automatically, even on exception
}

Using fiber_sem_guard prevents connection pool leaks that would occur if manual post() calls were skipped due to early returns or thrown exceptions.

Key Source Files and Implementation Details

The semaphore logic and usage patterns are distributed across the following files in the iqiyi/libfiber repository:

• cpp/include/fiber/fiber_sem.hpp – Defines the public API for fiber_sem, fiber_sem_guard, and the fiber_sbox utility that internally leverages the semaphore.
• cpp/src/fiber_sem.cpp – Implements the core operations including wait(), trywait(), post(), and num() for inspecting the current counter value.
• cpp/include/fiber/fiber_pool.hpp – Demonstrates pooling patterns where fiber_sem bounds resource usage across worker fiber groups.
• samples/cxx/waiter/main.cpp – Provides a runnable example of fiber synchronization using semaphores, adaptable for database limiting scenarios.

Summary

• fiber_sem is a fiber-aware counting semaphore that suspends fibers (not OS threads) when resources are unavailable, defined in cpp/include/fiber/fiber_sem.hpp.
• Initialize the semaphore with your maximum connection count using fiber_sem(max), defaulting to asynchronous behavior.
• Call wait() before acquiring a database connection and post() after releasing it to enforce the concurrency cap.
• Use fiber_sem_guard for exception-safe, automatic resource management that prevents pool leakage.
• The implementation in cpp/src/fiber_sem.cpp provides efficient, user-space context switching suitable for high-throughput database clients.

Frequently Asked Questions

What is the difference between fiber_sem_t_async and other semaphore attributes?

The default fiber_sem_t_async attribute creates an asynchronous semaphore where wait() suspends the calling fiber and yields to the scheduler. This is the standard mode for fiber-based applications and provides optimal performance for limiting database connections. Alternative attributes (if available in specific versions) may modify wake-up ordering, but the asynchronous mode is recommended for general use.

How does fiber_sem handle timeouts when waiting for a connection?

The wait(int ms = -1) method accepts an optional timeout in milliseconds. Passing a positive value (e.g., wait(5000)) causes the method to return -1 if no slot becomes available within that duration, allowing your fiber to implement retry logic or error handling instead of waiting indefinitely.

Can I use fiber_sem with any database driver?

Yes. Because fiber_sem is a generic synchronization primitive independent of specific I/O libraries, you can wrap any synchronous or asynchronous database client. Simply place the connection acquisition and query execution between the wait() and post() calls (or within a fiber_sem_guard scope) to enforce the concurrency limit regardless of the underlying driver implementation.

What happens if an exception occurs while holding the semaphore?

The fiber_sem_guard class guarantees that post() is invoked during stack unwinding if an exception escapes the guard's scope, preventing semaphore leakage. However, if you use manual wait()/post() pairs without a guard, an exception thrown before post() executes will permanently reduce your available connection pool by one slot until the application restarts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}