ASIO use_awaitable with Custom Executors and Thread Pools: A Complete Guide

asio::use_awaitable is a completion token that suspends C++20 coroutines until asynchronous operations finish, and it integrates seamlessly with custom executors like asio::thread_pool to control exactly where coroutines resume execution.

The chriskohlhoff/asio library provides robust abstractions for asynchronous I/O using C++20 coroutines. When building high-performance applications, you often need to separate I/O operations from CPU-bound work using custom executors and thread pools. Understanding how use_awaitable interacts with these executors allows you to build responsive systems where coroutines resume on specific threads under your control.

How use_awaitable Integrates with Executors

The use_awaitable token serves as the bridge between coroutine suspension and executor dispatch. When you pass this token to an asynchronous operation, Asio automatically creates a handler that suspends the coroutine and schedules its resumption on the associated executor.

The use_awaitable Token Implementation

In include/asio/use_awaitable.hpp, the library defines the primary completion token:

// include/asio/use_awaitable.hpp
constexpr use_awaitable_t<> use_awaitable(0,0,0);

This token represents the currently executing coroutine. When used with async operations like socket.async_read_some(), the operation suspends the coroutine until completion, then resumes it using the executor associated with the I/O object.

Executor Adaptation with executor_with_default

The use_awaitable_t class provides a nested executor_with_default adaptor that allows you to make use_awaitable the default completion token for any executor type. According to the source in include/asio/use_awaitable.hpp:

template<class InnerExecutor>
struct executor_with_default : InnerExecutor {
    using default_completion_token_type = use_awaitable_t;
};

This adaptation is crucial when working with asio::thread_pool because it allows coroutines to implicitly use use_awaitable without explicitly passing the token to every operation.

Setting Up Thread Pools for Coroutines

The asio::thread_pool class, defined in include/asio/thread_pool.hpp, provides an executor that runs work across a pool of threads. To use this with coroutines, you combine the pool's executor with executor_with_default to ensure that co_await operations suspend and resume correctly on the pool's threads.

When you spawn a coroutine using asio::co_spawn with a thread pool executor, the entire coroutine runs on that pool. Asynchronous operations that use use_awaitable will suspend the coroutine, and the completion handler will be dispatched back to the thread pool for resumption.

Practical Implementation Examples

Example 1: Echo Server Running on a Thread Pool

This example from src/examples/cpp20/coroutines/echo_server_with_deferred.cpp demonstrates a TCP echo server where the listener and all connection handlers run on a thread pool:

#include <asio.hpp>

using asio::awaitable;
using asio::use_awaitable;
using asio::detached;
using asio::ip::tcp;

awaitable<void> echo(tcp::socket socket)
{
    char data[1024];
    std::size_t n = co_await socket.async_read_some(
        asio::buffer(data), use_awaitable);
    co_await asio::async_write(socket,
        asio::buffer(data, n), use_awaitable);
}

awaitable<void> listener(asio::thread_pool& pool,
                         unsigned short port)
{
    tcp::acceptor acceptor(
        co_await asio::this_coro::executor,
        tcp::endpoint(tcp::v4(), port));

    for (;;) {
        tcp::socket sock = co_await acceptor.async_accept(use_awaitable);
        asio::co_spawn(pool.get_executor(),
                       echo(std::move(sock)),
                       detached);
    }
}

int main()
{
    asio::thread_pool pool{4};
    asio::co_spawn(pool.get_executor(),
                   listener(pool, 12345),
                   detached);
    pool.join();
}

Key implementation details:

• listener is started on pool.get_executor(), ensuring the coroutine runs on the thread pool.
• acceptor.async_accept(use_awaitable) suspends the coroutine until a connection arrives, then resumes on the pool.
• Each accepted socket spawns a new echo coroutine on the same thread pool executor.

Example 2: Custom Executor with Default Token

You can create a reusable adaptor that makes use_awaitable the default token for any executor, eliminating the need to pass the token explicitly:

#include <asio.hpp>

using asio::awaitable;
using asio::detached;

template<class Exec>
auto make_awaitable_executor(Exec ex)
{
    return typename asio::use_awaitable_t<>::template executor_with_default<Exec>(ex);
}

awaitable<void> timer_example()
{
    // No explicit token needed due to executor_with_default
    co_await asio::steady_timer(
        co_await asio::this_coro::executor, 
        std::chrono::seconds(1))
        .async_wait();
}

int main()
{
    asio::thread_pool pool{2};
    auto adapted = make_awaitable_executor(pool.get_executor());

    asio::co_spawn(adapted, timer_example(), detached);
    pool.join();
}

Key implementation details:

• make_awaitable_executor wraps the thread pool executor with executor_with_default.
• Inside timer_example, async_wait() is called without an explicit token because the executor provides use_awaitable_t as the default_completion_token_type.

Example 3: Hybrid I/O and Thread Pool Execution

For applications requiring high-performance I/O, you can keep sockets on an io_context while moving CPU-intensive work to a thread pool:

#include <asio.hpp>

using asio::awaitable;
using asio::use_awaitable;
using asio::detached;
using asio::ip::tcp;

awaitable<void> process_data(tcp::socket sock,
                             asio::thread_pool& pool)
{
    char data[512];
    
    // Perform I/O on the socket's executor (typically io_context)
    std::size_t n = co_await sock.async_read_some(
        asio::buffer(data), use_awaitable);
    
    // Switch to thread pool for CPU-intensive processing
    co_await asio::post(pool.get_executor(), use_awaitable);
    
    // Execute heavy computation here on pool threads
    
    // Return to I/O executor for writing
    co_await asio::async_write(sock,
        asio::buffer(data, n), use_awaitable);
}

Key implementation details:

• co_await asio::post(pool.get_executor(), use_awaitable) explicitly transitions the coroutine to the thread pool.
• Subsequent operations resume on the pool until another executor switch occurs.
• This pattern prevents blocking the I/O context with compute work.

Key Source Files and Implementation Details

Understanding the implementation requires examining these specific files in the chriskohlhoff/asio repository:

• include/asio/use_awaitable.hpp: Defines use_awaitable_t, the use_awaitable constant, and the executor_with_default adaptor.
• include/asio/thread_pool.hpp: Implements the thread pool executor type compatible with executor_with_default.
• src/examples/cpp20/coroutines/echo_server_with_deferred.cpp: Production-ready example of coroutine-based servers using thread pools.
• src/tests/unit/experimental/coro/executor.cpp: Unit tests demonstrating executor adaptation with custom thread pools.

Summary

• use_awaitable is a completion token that suspends coroutines until async operations complete, with automatic resumption on the associated executor.
• executor_with_default adapts any executor (including asio::thread_pool) to use use_awaitable as the default completion token.
• asio::co_spawn initiates coroutines on specific executors, determining where the coroutine starts and where it resumes after suspension.
• You can switch executors mid-coroutine using asio::post() with an explicit executor argument, enabling hybrid I/O and compute patterns.
• Thread pools combined with use_awaitable allow you to perform CPU-intensive work without blocking I/O contexts.

Frequently Asked Questions

What is the difference between use_awaitable and use_future?

use_awaitable is designed specifically for C++20 coroutines, suspending the coroutine and resuming it on the executor associated with the operation. use_future returns a std::future object, which is useful for traditional async programming but lacks the executor control and suspension efficiency of coroutines. According to the Asio source code, use_awaitable avoids the overhead of future synchronization and provides zero-cost abstraction for coroutine-based async operations.

How do I make a thread pool use use_awaitable by default?

Wrap the thread pool executor with use_awaitable_t::executor_with_default. As implemented in include/asio/use_awaitable.hpp, you create an adapted executor that declares use_awaitable_t as its default_completion_token_type. When you pass this adapted executor to asio::co_spawn, all async operations within that coroutine will implicitly use use_awaitable unless you specify a different token.

Can I switch executors in the middle of a coroutine?

Yes, using co_await asio::post(new_executor, use_awaitable). This pattern suspends the coroutine and schedules its resumption on the specified executor. According to the implementation in include/asio/post.hpp, this effectively transfers the coroutine's execution context from one executor (such as an I/O context) to another (such as a thread pool), allowing you to optimize thread usage throughout the coroutine's lifetime.

Does use_awaitable work with strands?

Yes, use_awaitable is fully compatible with asio::strand. When you wrap a strand around an executor and adapt it with executor_with_default, the coroutine will resume on the strand, ensuring that handlers execute sequentially without explicit locking. This is particularly important when using thread pools with shared state, as it prevents race conditions while maintaining the benefits of coroutine-based async programming.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}