Nelson Engine API: How to Embed Nelson as a Computational Backend in C Applications

The Nelson Engine API is a MATLAB-compatible MEX interface that allows C programs to start a Nelson process, evaluate commands, and exchange data via shared memory IPC.

The Nelson Engine API, implemented in the nelson-lang/nelson repository, provides a direct path to embed Nelson’s numerical computing capabilities into external C or C++ applications. By mirroring the MATLAB MEX Engine API, it enables existing MATLAB-centric codebases to migrate or integrate with Nelson while maintaining familiar function signatures and workflows.

What Is the Nelson Engine API?

The Nelson Engine API is a C interface defined in modules/mex/src/include/engine.h and implemented in modules/mex/src/cpp/Engine.cpp. It exposes a set of functions that allow host applications to control a Nelson interpreter running in a separate process. The API is designed to be compatible with the MATLAB MEX Engine API, meaning functions like engOpen, engEvalString, and engPutVariable behave identically to their MATLAB counterparts.

The engine operates by launching a Nelson GUI or headless process and establishing communication through inter-process shared memory, named mutexes, and a lightweight IPC protocol. The host application receives an opaque Engine* handle, which it uses for all subsequent interactions.

Architecture and IPC Mechanism

Process Management and Shared Memory

When engOpen is invoked, the API performs several steps to establish the connection:

1. Creates a Nelson-interprocess receiver for the parent PID using Nelson::createNelsonInterprocessReceiver, which sets up the shared-memory segment for commands and data.
2. Checks for an existing GUI-enabled Nelson instance via Nelson::getLatestPidWithModeInSharedMemory(NELSON_ENGINE_MODE::GUI).
3. If no instance exists, spawns a new Nelson GUI process using start_child (platform-specific: CreateProcessW on Windows or fork/exec on POSIX).
4. Waits for the child to signal readiness through the NelsonReadyNamedMutex (waitUntilNelsonIsReady).

The resulting Engine struct stores a pointer to the PROCESS_CHILD object and an isSingleUse flag that determines whether engClose should terminate the child process.

Thread Safety and Reference Counting

All engine interactions are thread-safe. The implementation maintains a reference count (countEngine) to manage shared resources. When the last engine handle is closed via engClose, the API cleans up shared memory segments and mutexes, ensuring no resource leaks occur between multiple embedded instances.

Core Engine API Functions

The Nelson Engine API provides the following functions for controlling the interpreter:

Function	Purpose
engOpen / engOpenSingleUse	Start a Nelson process (shared or single-use) and return an Engine* handle
engClose	Close the engine connection and optionally terminate the child process
engEvalString	Execute a Nelson command string in the remote workspace
engPutVariable	Transfer a variable from C (mxArray) to Nelson's workspace
engGetVariable	Retrieve a variable from Nelson's workspace into a C mxArray
engOutputBuffer	Attach a character buffer to capture text output from the engine
engSetVisible / engGetVisible	Control or query the visibility of the Nelson GUI window

Data marshalling between Nelson's internal ArrayOf type and MATLAB-compatible mxArray structures is handled by ArrayOfToMxArray and MxArrayToArrayOf, defined in modules/mex/src/include/mxArrayOf.hpp.

Step-by-Step Implementation Guide

Opening the Engine Connection

To begin embedding Nelson, include the engine header and open a connection:

#include "engine.h"

Engine *ep = engOpen("");
if (ep == NULL) {
    fprintf(stderr, "Unable to start Nelson engine\n");
    return 1;
}

Passing an empty string to engOpen attempts to connect to an existing Nelson GUI instance or launches a new one. Use engOpenSingleUse to force a dedicated process that terminates when closed.

Evaluating Commands and Exchanging Data

Once connected, execute Nelson commands and transfer data:

/* Execute a command in Nelson's workspace */
engEvalString(ep, "x = 1:10; y = x.^2;");

/* Create a double array in C and send it to Nelson */
double data[] = {3.14, 2.71, 1.41};
mxArray *mx = mxCreateDoubleMatrix(1, 3, mxREAL);
memcpy(mxGetPr(mx), data, sizeof(data));
engPutVariable(ep, "constants", mx);
mxDestroyArray(mx);

/* Retrieve a variable from Nelson */
mxArray *result = engGetVariable(ep, "y");
if (result != NULL) {
    double *pr = mxGetPr(result);
    printf("First element of y: %f\n", pr[0]);
    mxDestroyArray(result);
}

Closing the Engine Session

When finished, close the connection to release resources:

engClose(ep);

If the engine was opened in shared mode (engOpen), the Nelson GUI process remains running for other applications to use. If opened with engOpenSingleUse, engClose terminates the dedicated process.

Complete C Example: Embedding Nelson

The following complete example demonstrates starting the engine, evaluating expressions, transferring variables, and shutting down:

#include <stdio.h>
#include "engine.h"        /* Nelson Engine API header */

int main(void)
{
    /* 1. Open a shared Nelson engine (reuse existing GUI if present) */
    Engine *ep = engOpen("");
    if (ep == NULL) {
        fprintf(stderr, "Failed to start Nelson engine.\n");
        return 1;
    }

    /* 2. Evaluate a command */
    if (engEvalString(ep, "a = 42;") != 0) {
        fprintf(stderr, "Eval failed.\n");
        engClose(ep);
        return 1;
    }

    /* 3. Create a numeric mxArray on the C side */
    double data[3] = {1.0, 2.0, 3.0};
    mxArray *mx = mxCreateDoubleMatrix(1, 3, mxREAL);
    memcpy(mxGetPr(mx), data, sizeof(data));

    /* 4. Put the array into Nelson's workspace as variable 'b' */
    if (engPutVariable(ep, "b", mx) != 0) {
        fprintf(stderr, "PutVariable failed.\n");
        mxDestroyArray(mx);
        engClose(ep);
        return 1;
    }
    mxDestroyArray(mx);  /* safe to free after transfer */

    /* 5. Retrieve variable 'a' from Nelson */
    mxArray *retr = engGetVariable(ep, "a");
    if (retr) {
        double a_val = *mxGetPr(retr);
        printf("Nelson returned a = %f\n", a_val);
        mxDestroyArray(retr);
    } else {
        fprintf(stderr, "GetVariable failed.\n");
    }

    /* 6. Close the engine (will not terminate the GUI because we used shared mode) */
    engClose(ep);
    return 0;
}

Explanation of workflow

Step	API call	Purpose
1	engOpen("")	Attaches to existing Nelson GUI or launches new process; returns Engine* handle
2	engEvalString	Executes a = 42; in the remote workspace
3‑4	mxCreateDoubleMatrix + engPutVariable	Constructs C array, sends to Nelson as variable b
5	engGetVariable	Retrieves a from Nelson into C mxArray
6	engClose	Releases handle; shared GUI remains running

Compile with the Nelson library (e.g., -lnlsMex on Linux or link libnlsMex.dll on Windows) and ensure the include path contains modules/mex/src/include.

Key Source Files and Headers

File	Role	GitHub Link
modules/mex/src/include/engine.h	Public C header exposing the Engine API	engine.h
modules/mex/src/cpp/Engine.cpp	Core implementation of engine functions, process handling, IPC, and data marshalling	Engine.cpp
modules/mex/src/include/mxArrayOf.hpp	Conversion utilities between mxArray and Nelson’s ArrayOf	mxArrayOf.hpp
modules/mex/src/include/MxVariables.h	Helper macros and definitions for MEX-compatible matrix types	MxVariables.h
modules/interprocess/src/include/NelsonInterprocess.hpp	Shared-memory segment, mutex handling, and command protocol	NelsonInterprocess.hpp

These files constitute the complete backend that enables C applications to embed Nelson as a computational engine.

Summary

• The Nelson Engine API provides MATLAB-compatible functions (engOpen, engEvalString, engPutVariable, etc.) to control a Nelson interpreter from C or C++.
• Communication occurs via shared memory IPC and named mutexes, with the engine running as a separate process managed through an opaque Engine* handle.
• Thread-safe reference counting ensures proper cleanup of shared resources when multiple applications or threads use the engine concurrently.
• Data exchange uses MATLAB-compatible mxArray structures, converted to Nelson’s internal ArrayOf type via ArrayOfToMxArray and MxArrayToArrayOf.
• The API supports both shared mode (attaches to existing GUI) and single-use mode (dedicated process), controlled via engOpen and engOpenSingleUse.

Frequently Asked Questions

How does the Nelson Engine API differ from MATLAB's Engine API?

The Nelson Engine API is designed to be functionally compatible with the MATLAB MEX Engine API, using identical function signatures for engOpen, engEvalString, engPutVariable, and related calls. However, Nelson’s implementation uses shared memory IPC rather than COM (Windows) or pipes (Unix) typically used by MATLAB, and it marshals data between mxArray and Nelson’s native ArrayOf type. This allows existing MATLAB engine code to compile against Nelson with minimal modifications while leveraging Nelson’s open-source architecture.

Can I run multiple Nelson engine instances simultaneously from the same C application?

Yes, the Nelson Engine API supports multiple concurrent instances through thread-safe reference counting. You can call engOpen or engOpenSingleUse multiple times to obtain distinct Engine* handles, each representing a separate connection. The internal countEngine variable tracks active handles, and shared IPC resources are only cleaned up when the last engine is closed via engClose. This design allows complex applications to isolate computational contexts or distribute workloads across multiple Nelson processes.

What data types can be transferred between C and Nelson using the Engine API?

The API supports all MATLAB-compatible matrix types through the mxArray interface, including mxDOUBLE_ARRAY, mxSINGLE_ARRAY, mxINT8_ARRAY, mxUINT8_ARRAY, mxINT16_ARRAY, mxUINT16_ARRAY, mxINT32_ARRAY, mxUINT32_ARRAY, mxINT64_ARRAY, mxUINT64_ARRAY, mxCHAR_ARRAY, mxSTRUCT_CLASS, and mxCELL_CLASS. Data conversion occurs in modules/mex/src/include/mxArrayOf.hpp, where MxArrayToArrayOf converts incoming mxArray structures to Nelson’s ArrayOf objects, and ArrayOfToMxArray handles the reverse direction when retrieving variables via engGetVariable.

How do I capture text output from Nelson when using the Engine API?

Use the engOutputBuffer function to redirect Nelson’s standard output to a C character buffer. Before calling engEvalString, allocate a character array and register it with engOutputBuffer(ep, buffer, bufferSize). When engEvalString completes, the buffer contains UTF-8 encoded text output from the command execution. The implementation in modules/mex/src/cpp/Engine.cpp (lines 486-499) converts internal wide strings to UTF-8 using Nelson::wstring_to_utf8 before populating the buffer, ensuring cross-platform text compatibility.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}