How to Use bpftime with Kernel eBPF: Hybrid Execution and Verification Guide

Set BPFTIME_RUN_WITH_KERNEL=true when loading eBPF programs to enable kernel verification while maintaining userspace execution, allowing bpftime to leverage the Linux kernel verifier for safety guarantees without sacrificing userspace performance benefits.

The eunomia-bpf/bpftime project provides a high-performance userspace eBPF runtime that can operate independently or integrate with the Linux kernel eBPF subsystem. When you configure bpftime with kernel eBPF support, you enter a hybrid execution mode where programs undergo mandatory kernel verification while still running in userspace, combining the safety guarantees of the kernel verifier with the flexibility and performance of userspace execution.

Understanding the bpftime Kernel eBPF Hybrid Mode

bpftime can execute eBPF programs entirely in userspace or in a hybrid mode where the program is loaded into the Linux kernel for verification while execution remains in userspace. This architecture, documented in usage.md【/cache/repos/github.com/eunomia-bpf/bpftime/master/usage.md#L57-L64】, provides three primary advantages:

• Kernel verifier compliance: The program must pass the kernel's eBPF verifier, ensuring memory safety and bounded execution.
• Map sharing: Reuse kernel-side eBPF maps (such as BPF_MAP_TYPE_HASH or BPF_MAP_TYPE_PERCPU_ARRAY) to share state with kernel programs like kprobes or XDP filters.
• Agent persistence: The bpftime agent remains attached to the target process, providing userspace helpers, ufuncs, and shared-memory map infrastructure.

Environment Variables for Kernel Integration

The hybrid workflow is controlled through environment variables parsed into the bpftime::agent_config structure defined in runtime/include/bpftime_config.hpp【/cache/repos/github.com/eunomia-bpf/bpftime/master/runtime/include/bpftime_config.hpp#L53-L66】:

• BPFTIME_RUN_WITH_KERNEL: When set to true, bpftime invokes the kernel's bpf() system call to load the eBPF object and run the kernel verifier. The program is not executed by the kernel; it remains in userspace for execution by the bpftime VM.
• BPFTIME_NOT_LOAD_PATTERN: A regular expression matching program names that should not be loaded into the kernel even when BPFTIME_RUN_WITH_KERNEL is true. This is essential for userspace-only programs utilizing helpers unsupported by the kernel.

Loading eBPF Programs with Kernel Verification

To load a program with kernel verification, use the bpftime CLI (tools/cli/main.cpp), which triggers bpf_prog_load() via bpftime_helper_group::get_kernel_utils_helper_group() when the environment variable is set.

The following example demonstrates the malloc tracing demo from example/malloc/:

# Build the example program

make -C example/malloc

# Enable kernel verification and load the program

BPFTIME_RUN_WITH_KERNEL=true \
LD_PRELOAD=~/.bpftime/libbpftime-syscall-server.so \
bpftime load ./example/malloc/malloc

Then attach to the target process:

LD_PRELOAD=~/.bpftime/libbpftime-agent.so \
bpftime start ./example/malloc/victim

You should observe output similar to:

pid=30415  malloc calls: 1079
pid=30393  malloc calls: 203

The kernel verifier logs its output to dmesg (or console if BPFTIME_LOG_OUTPUT=console), confirming successful verification without kernel-side execution.

Sharing Kernel Maps with Userspace Execution

When running in hybrid mode, kernel-created maps are represented by bpftime::map_handle objects that point to a shared-memory region defined in runtime/include/bpftime_shm.hpp. The map data resides in a POSIX shared memory segment accessible by both kernel helpers and the userspace VM.

Define a kernel-compatible map in your eBPF program:

// In your libbpf program (e.g., malloc.c)
struct {
    __uint(type, BPF_MAP_TYPE_HASH);
    __type(key, __u32);
    __type(value, __u64);
    __uint(max_entries, 1024);
} malloc_counts SEC(".maps");

When loaded with BPFTIME_RUN_WITH_KERNEL=true, this map becomes accessible to:

• Kernel-side helpers: Other kprobes or XDP filters can increment the same map.
• bpftime userspace VM: The program executes in userspace but reads/writes the same shared map instance.

No additional configuration is required; the runtime automatically creates the bpftime_shm map entry that maps the kernel-allocated file descriptor to the shared region.

Selective Kernel Loading with Pattern Matching

For projects containing both kernel-compatible and userspace-only eBPF programs, use BPFTIME_NOT_LOAD_PATTERN to exclude specific programs from kernel loading:

# Exclude programs with "userspace_only" in their name

BPFTIME_RUN_WITH_KERNEL=true \
BPFTIME_NOT_LOAD_PATTERN=userspace_only \
LD_PRELOAD=~/.bpftime/libbpftime-syscall-server.so \
bpftime load ./userspace_only.bpf.o

In this scenario:

1. The program is initially loaded into the kernel to run the verifier.
2. If the name matches the pattern, the kernel-loaded copy is immediately released.
3. Execution continues via the LLVM-JIT VM (vm/vm-core) using userspace-only helpers (such as custom ffi helpers) that the kernel does not implement.

Summary

• Hybrid execution combines kernel verification with userspace performance by setting BPFTIME_RUN_WITH_KERNEL=true.
• Environment variables defined in runtime/include/bpftime_config.hpp control whether programs are loaded into the kernel and which programs are excluded via regex patterns.
• Kernel map sharing occurs automatically through POSIX shared memory (runtime/include/bpftime_shm.hpp), allowing seamless data exchange between kernel eBPF components and the bpftime userspace VM.
• Selective loading via BPFTIME_NOT_LOAD_PATTERN enables mixed workloads where some programs use kernel verification while others rely solely on userspace helpers.

Frequently Asked Questions

What is the difference between bpftime's userspace-only mode and kernel hybrid mode?

In userspace-only mode, bpftime loads and executes eBPF programs entirely within the userspace VM without interacting with the kernel eBPF subsystem. In kernel hybrid mode, activated by BPFTIME_RUN_WITH_KERNEL=true, the program is first loaded into the kernel via bpf_prog_load() to undergo the kernel verifier's safety checks, but execution remains in userspace. This provides kernel-level safety guarantees while retaining the performance benefits and helper flexibility of userspace execution.

How does bpftime share map data between kernel and userspace components?

bpftime utilizes a shared-memory architecture defined in runtime/include/bpftime_shm.hpp. When a program is loaded with kernel support, kernel-created maps are backed by POSIX shared memory segments. The bpftime::map_handle objects bridge the kernel file descriptors to these shared regions, allowing both kernel-side eBPF helpers (like kprobes) and the bpftime userspace VM to read and write the same map instances concurrently without data copying.

Can I use kernel-specific eBPF helpers when running bpftime with kernel eBPF?

Yes, when operating in hybrid mode with BPFTIME_RUN_WITH_KERNEL=true, you can use standard kernel eBPF helpers that are supported by the kernel verifier. However, if you need to use custom userspace helpers (such as ffi helpers) that the kernel does not implement, you should set BPFTIME_NOT_LOAD_PATTERN to exclude those specific programs from kernel loading. This allows the programs to run in pure userspace mode while other programs in the same project benefit from kernel verification.

What happens if the kernel verifier rejects my program when using BPFTIME_RUN_WITH_KERNEL?

If the kernel verifier rejects your program during the bpf_prog_load() call initiated by the bpftime CLI (tools/cli/main.cpp), the loading process will fail and the program will not execute. The verifier's error log is typically available via dmesg or console output (when BPFTIME_LOG_OUTPUT=console). You must modify your eBPF code to satisfy the kernel verifier's safety constraints—such as ensuring bounded loops, valid pointer dereferences, and appropriate map access—before the program can be loaded in hybrid mode.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}