How Nelson's Foreign Function Interface (FFI) Loads C and Fortran Code

Nelson's FFI dynamically links native libraries at runtime using dlopen on POSIX or LoadLibrary on Windows, resolves symbols via dlsym or GetProcAddress, and wraps C/Fortran functions in callable handles that automatically marshal data between Nelson and native code.

The nelson-lang/nelson interpreter provides a Foreign Function Interface (FFI) that enables direct execution of compiled C, C++, and Fortran code from Nelson scripts without static linking or recompilation. This system leverages platform-specific dynamic linking APIs to load shared libraries and create callable wrappers for native functions.

The Five-Step FFI Workflow

Nelson's FFI implementation follows a structured pipeline for integrating native code. The core logic resides in DynamicLinkLibraryObject within modules/dynamic_link/src/cpp/DynamicLinkLibraryObject.cpp.

1. Open the Native Library

The process begins when dlopen (or loadlibrary on Windows) loads a shared object file (.so, .dylib, or .dll). The builtin function dlopen creates a DynamicLinkLibraryObject instance by calling load_dynamic_library, which wraps the system-specific library loading mechanism.

On POSIX systems, this uses dlopen from <dlfcn.h>. On Windows, it uses LoadLibraryW. The abstraction layer in modules/commons/src/cpp/DynamicLibrary.cpp handles platform detection and API selection at compile time.

2. Discover Function Symbols

Once loaded, the library handle resolves function addresses using dlsym (POSIX) or GetProcAddress (Windows). The method DynamicLinkLibraryObject::getFunctionPointer performs this lookup, returning a raw function pointer for the requested symbol name.

If the symbol does not exist, the method returns an empty pointer, which Nelson converts into a runtime error when attempting to create the function handle.

3. Create the Callable Wrapper

The dlsym builtin constructs a Nelson function_handle that encapsulates the native function pointer along with its prototype signature. This wrapper stores the return type and argument types (e.g., double, int32Ptr, doublePtr) to enable automatic type marshaling.

The conversion utilities in the dynamic-link gateway marshal Nelson data types to native C types before invocation and convert results back to Nelson objects afterward.

4. Execute the Native Function

When the user invokes the function handle, Nelson executes the native code directly through the stored pointer. The wrapper manages the stack frame, passes marshaled arguments, and captures the return value.

Errors occurring during native execution are intercepted using dlerror (POSIX) or GetLastError (Windows), then wrapped into Nelson exceptions with descriptive messages.

5. Close the Library Handle

When the dllib object is destroyed or explicitly deleted via delete(lib), the destructor invokes close_dynamic_library. This calls dlclose on POSIX or FreeLibrary on Windows, releasing the shared library from memory and cleaning up associated resources.

Platform-Specific Implementation Details

Nelson's FFI abstracts operating system differences through a common interface in modules/commons/src/cpp/DynamicLibrary.cpp.

On POSIX systems (Linux, macOS), the implementation relies on:

• dlopen to load shared objects
• dlsym to resolve symbols
• dlclose to unload libraries
• dlerror to retrieve error messages

On Windows, the corresponding APIs are:

• LoadLibraryW for loading DLLs
• GetProcAddress for symbol resolution
• FreeLibrary for cleanup
• GetLastError for diagnostic information

The DynamicLinkLibraryObject class defined in modules/dynamic_link/src/include/DynamicLinkLibraryObject.hpp provides the cross-platform object model that encapsulates these system-specific handles.

Calling C Functions: Practical Example

The following example demonstrates loading the standard C math library and calling the cos function:

% Load the shared library (Linux example)
lib = dlopen('libm.so');

% Resolve the symbol with type signature: double cos(double)
cosHandle = dlsym(lib, "cos", "double", {"double"});

% Call the native function
result = dlcall(cosHandle, 0.5);
disp(result);  % Output: 0.87758256189

% Clean up resources
delete(cosHandle);
delete(lib);

This workflow uses dlopenBuiltin.cpp to create the library object, DynamicLinkLibraryObject::getFunctionPointer to resolve cos, and the dynamic-link gateway to marshal the double argument and return value.

Calling Fortran Code: BLAS Integration Example

Nelson treats Fortran symbols identically to C symbols, though Fortran typically appends an underscore to symbol names. The official example modules/dynamic_link/examples/call_fortran.m demonstrates calling the BLAS dasum routine:

function call_fortran()
    % Detect installed BLAS library
    blasLib = detect_blas();
    lib = dlopen(blasLib);  % Load the shared library
    
    % Fortran symbol name (trailing underscore convention)
    DASUM_SYMBOL = "dasum";
    
    % Prepare test data
    V = [-2, 1, 3, -5, 4, 0, -1, -3];
    N = length(V);
    
    % Create function handle with Fortran signature
    % return: double, args: int32*, double*, int32*
    f = dlsym(lib, DASUM_SYMBOL, 'double', {'int32Ptr','doublePtr','int32Ptr'});
    
    % Call it: dasum(N, V, inc) -> L1 norm of V
    r = dlcall(f, int32(N), V, int32(1));
    assert(r == 19);  % Verify L1 norm result
    
    % Release resources
    delete(f);
    delete(lib);
end

This example leverages DynamicLinkLibraryObject to manage the BLAS library lifecycle and uses pointer type specifiers (int32Ptr, doublePtr) to match Fortran's pass-by-reference semantics.

Error Handling and Symbol Inspection

Nelson provides mechanisms for debugging FFI interactions and handling native errors gracefully.

Runtime Error Handling

When library loading or symbol resolution fails, Nelson captures OS-specific error messages and converts them into standard exceptions. On POSIX systems, it retrieves detailed error descriptions via dlerror(), while on Windows it uses GetLastError() combined with FormatMessageW. These diagnostics are wrapped in Nelson's Error function and presented to the user with messages like "Cannot load library: [details]".

Symbol Enumeration

For debugging purposes, DynamicLinkLibraryObject provides getAvailableSymbols(), which returns a cell array of all exported symbols in a loaded library:

lib = dlopen('libm.so');
symbols = lib.getAvailableSymbols();  % Returns cell array of strings
disp(symbols(1:10));  % Display first 10 exported functions
delete(lib);

On Linux and macOS, this method executes nm -D via runCommandCaptureOutput to parse the dynamic symbol table, while on Windows it reads the PE export directory directly.

Key Source Files in the Nelson Repository

The FFI implementation spans several modules with clear separation of concerns:

File	Role
modules/dynamic_link/builtin/cpp/dlopenBuiltin.cpp	Implements the dlopen builtin that instantiates DynamicLinkLibraryObject handles.
modules/dynamic_link/src/include/DynamicLinkLibraryObject.hpp	Header defining the DynamicLinkLibraryObject class interface for library management.
modules/dynamic_link/src/cpp/DynamicLinkLibraryObject.cpp	Core implementation of library loading (load_dynamic_library), symbol resolution (getFunctionPointer), and cleanup (close_dynamic_library).
modules/commons/src/cpp/DynamicLibrary.cpp	OS-agnostic wrapper around dlopen/LoadLibrary and related calls.
modules/dynamic_link/examples/call_fortran.m	End-to-end example showing detection, loading, and calling a Fortran BLAS routine.
modules/dynamic_link/examples/call_c.m	Reference implementation demonstrating C library interaction patterns.

These files together constitute Nelson's Foreign Function Interface, allowing seamless interoperation with compiled C/Fortran code without needing a separate compilation step.

Summary

Nelson's Foreign Function Interface provides a robust mechanism for calling native C and Fortran code through dynamic linking:

• Dynamic loading uses platform-specific APIs (dlopen on POSIX, LoadLibrary on Windows) via the DynamicLinkLibraryObject class in modules/dynamic_link/src/cpp/DynamicLinkLibraryObject.cpp.
• Symbol resolution maps function names to executable pointers using dlsym or GetProcAddress, wrapped by getFunctionPointer.
• Type marshaling automatically converts Nelson data types to native C types (e.g., double, int32Ptr) when creating function handles with dlsym and invoking them with dlcall.
• Resource management ensures proper cleanup through destructors that invoke dlclose or FreeLibrary when library handles are deleted.
• Cross-platform support abstracts OS differences through modules/commons/src/cpp/DynamicLibrary.cpp, providing a unified interface for Linux, macOS, and Windows.

Frequently Asked Questions

How does Nelson handle library loading errors when using dlopen?

Nelson captures OS-specific error messages and converts them into standard exceptions. On POSIX systems, it retrieves detailed error descriptions via dlerror(), while on Windows it uses GetLastError() combined with FormatMessageW. These diagnostics are wrapped in Nelson's Error function and presented to the user with messages like "Cannot load library: [details]".

Can Nelson call Fortran functions that use pass-by-reference semantics?

Yes, Nelson handles Fortran's pass-by-reference convention through pointer type specifiers. When creating a function handle with dlsym, you declare arguments as int32Ptr or doublePtr rather than value types. This matches Fortran's expectation of receiving memory addresses rather than immediate values, allowing seamless integration with libraries like OpenBLAS without modification.

What is the difference between dlsym and dlcall in Nelson's FFI?

dlsym creates a callable function handle by combining a library reference, symbol name, return type, and argument type list. It returns a persistent object that knows how to marshal data but does not yet execute the native code. dlcall actually invokes the native function through the handle created by dlsym, passing the marshaled arguments and returning the result to Nelson. This separation allows you to resolve symbols once and call them multiple times efficiently.

How can I inspect what functions are available in a loaded library?

Use the getAvailableSymbols() method on a library handle returned by dlopen. This method returns a cell array of all exported symbol names. On Linux and macOS, it executes nm -D via runCommandCaptureOutput to parse the dynamic symbol table, while on Windows it reads the PE export directory directly. This is useful for debugging linkage issues or discovering available APIs in unfamiliar libraries.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}