# Core C FFI Functions in Needle: The Complete Python-to-C Interface

> Discover the four core C FFI functions in Needle: engine_load, engine_forward, tokenizer_encode, and tokenizer_decode for high-performance Python-to-C inference.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: api-reference
- Published: 2026-08-24

---

**Needle exposes exactly four low-level C functions—`engine_load`, `engine_forward`, `tokenizer_encode`, and `tokenizer_decode`—through a lightweight CFFI layer in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) to enable high-performance on-device inference.**

Needle is a minimal inference engine from Cactus Compute that delivers a 28 MiB runtime for running compiled *.cact* models. The repository implements a thin Python wrapper that binds to optimized C code using CFFI, exposing only the essential **C FFI functions** required for model execution and tokenization.

## The Four Core C FFI Functions Bound in needle/model/run.py

The entire Python-to-C interface lives in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) and exposes four critical operations. The wrapper initializes these bindings at import time using `cffi.FFI()` and maps them to user-friendly Python functions.

### engine_load – Initializing the Inference Context

The first **C FFI function**, `engine_load`, handles model initialization. According to the source code in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py), this function accepts a file path to a *.cact* checkpoint and returns a pointer to the engine context.

### engine_forward – Executing the Inference Step

The `engine_forward` function performs the actual computation. It accepts token IDs, the KV cache pointer, and output buffers for logits. This **C FFI function** updates the cache in-place and returns prediction scores without Python overhead during the forward pass.

### tokenizer_encode and tokenizer_decode – Text Processing

Text conversion happens through two dedicated **C FFI functions**. `tokenizer_encode` converts UTF-8 strings to integer token arrays, while `tokenizer_decode` reverses the process. Both operate directly on C strings to avoid Python string copying overhead.

## How the FFI Layer Initializes the Shared Library

The binding mechanism uses CFFI's ABI mode. When [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) loads, it executes:

```python
_ffi = cffi.FFI()
_ffi.cdef("""
    void* engine_load(const char* path);
    void  engine_forward(void* ctx,
                         const int32_t* tokens, size_t n_tokens,
                         float* logits, size_t vocab_size,
                         void* kv_cache);
    int32_t* tokenizer_encode(const char* text, size_t* out_len);
    char*   tokenizer_decode(const int32_t* tokens, size_t n);
""")
_lib = _ffi.dlopen(_LIB_PATH)

```

This declares the C function signatures once, then dynamically loads the compiled shared library at `_LIB_PATH`. The Python wrapper functions convert Python objects to raw C pointers before delegating to these bound symbols.

## Complete Workflow Example

The following code demonstrates the four **core C FFI functions** in action:

```python
import needle

# 1️⃣ Load a tuned model (the .cact file contains the compiled engine)

engine = needle.load_engine("my_needle.cact")

# 2️⃣ Encode a prompt – uses the C tokenizer

tokens = needle.encode("Translate English to French: Hello world.")

# 3️⃣ Run a forward pass – calls the C inference routine

logits, new_state = needle.run_engine(engine, tokens)

# 4️⃣ Decode the generated tokens back to text

response = needle.decode(logits.argmax(axis=-1))
print(response)     # → "Bonjour le monde."

```

Each public function—`load_engine`, `run_engine`, `encode`, and `decode`—forwards to its corresponding C implementation with minimal marshalling overhead.

## Architecture of the C FFI Surface

The **C FFI functions** are supported by four key files that constitute the minimal interface:

- **[`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py)**: Contains the CFFI declarations and the Python wrappers that bind the core engine functions.
- **[`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py)**: Exports PyTorch checkpoints to the single-file *.cact* binary that `engine_load` reads.
- **[`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py)**: Defines the neural network layout that the compiled C engine implements.
- **[`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)**: Exposes public entry points that forward to the FFI layer in [`run.py`](https://github.com/cactus-compute/needle/blob/main/run.py).

## Summary

- Needle exposes exactly four **C FFI functions**: `engine_load`, `engine_forward`, `tokenizer_encode`, and `tokenizer_decode`.
- The bindings are defined in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) using CFFI and loaded via `_ffi.dlopen()`.
- Each function handles a critical path: model loading, inference execution, and text tokenization.
- The deliberately tiny C API keeps the runtime footprint at 28 MiB while eliminating Python overhead during the forward pass.

## Frequently Asked Questions

### What are the exact C function signatures exposed by Needle's FFI?

Needle declares four functions in its CFFI header: `void* engine_load(const char* path)`, `void engine_forward(void* ctx, const int32_t* tokens, size_t n_tokens, float* logits, size_t vocab_size, void* kv_cache)`, `int32_t* tokenizer_encode(const char* text, size_t* out_len)`, and `char* tokenizer_decode(const int32_t* tokens, size_t n)`. These signatures are declared once in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) using `_ffi.cdef()`.

### How does Needle minimize overhead when calling C FFI functions from Python?

The wrapper in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) uses CFFI's ABI mode to declare signatures once at import time, then calls `_ffi.dlopen()` to load the shared library. Python wrapper functions convert objects to raw pointers before crossing the language boundary, eliminating data copying during the inference hot path.

### Where is the C library actually loaded in the Needle codebase?

The shared library is loaded in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) through the line `_lib = _ffi.dlopen(_LIB_PATH)`, which executes when the module is first imported. This binds the Python names to the actual C symbols for the duration of the process.

### Why does Needle use CFFI instead of ctypes or Cython for its Python bindings?

CFFI provides a minimal, explicit declaration of C function signatures that matches Needle's design philosophy of a tiny runtime footprint. The `_ffi.cdef()` approach generates efficient calling code without requiring Cython compilation steps or ctypes' runtime signature lookup overhead.