Nelson Engine Modes: CLI vs GUI vs Embedded Differences Explained

Nelson supports three distinct engine modes—CLI (terminal), GUI, and embedded—that determine whether the interpreter runs as a console application, a full graphical IDE, or exposes optional UI components like the script editor within a running session.

The nelson-lang/nelson repository provides a numerical computing environment that can operate in multiple engine configurations. Understanding the difference between Nelson engine modes is essential for choosing the right deployment strategy, whether you need headless scripting, interactive plotting capabilities, or a complete desktop interface with integrated editing tools.

Understanding the Three Nelson Engine Modes

Nelson distinguishes between terminal-based execution, full graphical environments, and optionally embedded UI components. Each mode activates different initialization paths in the core engine.

CLI Mode: Terminal-Based Execution

CLI mode runs the interpreter without a graphical window, displaying the REPL directly in the console and limiting I/O to text streams. The source code defines several CLI variants in modules/engine/src/include/NelSon_engine_mode.h:

• BASIC_TERMINAL: Found in NelSon_engine_mode.h, this mode provides a minimal console interface through the nelson_cli executable (modules/main/nelson_cli/main-cli.cpp).
• ADVANCED_TERMINAL: Used by nelson_adv_cli (modules/main/nelson_adv_cli/main-adv-cli.cpp), this mode supports graphics display in separate windows while maintaining a terminal-based command interface.

When you launch nelson_cli myscript.m, the executable passes the NELSON_ENGINE_MODE flag to StartNelson*() functions defined in modules/engine/src/cpp/StartNelson.cpp, which initializes a basic evaluator without GUI objects.

GUI Mode: Full Graphical Environment

GUI mode starts the complete graphical application including menus, variable explorers, and integrated plot panes. The nelson_gui executable (modules/main/nelson_gui/main-gui.cpp) explicitly calls runNelson(..., NELSON_ENGINE_MODE::GUI).

During initialization, InitGuiObjectsDynamic() creates all GUI-related objects, and the system instantiates a GuiTerminal as the front-end interface. This mode provides the full desktop experience for interactive numerical computing.

Embedded Mode: Optional UI Components

Embedded mode refers to features that can be loaded inside an existing engine session, specifically the embedded script editor and embedded help viewer. Unlike CLI and GUI modes which are selected at startup, embedded components are controlled at runtime by the useEmbeddedEditorFlag configuration setting.

In modules/nelson_manager/src/cpp/NelsonConfiguration.cpp (lines 515-522), the NelsonConfiguration::useEmbeddedEditor() method checks this boolean flag. When you execute the edit command from a script, editorBuiltin.cpp (modules/text_editor/builtin/cpp/editorBuiltin.cpp, lines 27-41) queries this configuration before opening the editor window.

How Nelson Stores and Detects the Engine Mode

The current mode persists as an integer in NelsonConfiguration::engineMode, defined in modules/nelson_manager/src/include/NelsonConfiguration.hpp and implemented in the corresponding .cpp file. The StartNelson.cpp module (lines 282-309) reads this value to determine which evaluator type to instantiate.

The MainEvaluator.cpp file creates the appropriate Evaluator object based on this mode flag, ensuring that CLI sessions receive a terminal evaluator while GUI sessions receive a graphical one.

Querying and Controlling Modes Programmatically

You can detect the current operating mode from Nelson scripts or C++ code to adjust behavior dynamically.

Checking the Current Mode from Scripts

The built-in getnelsonmode() function returns a string representation of the active mode:

m = getnelsonmode();   % returns "GUI", "BASIC_TERMINAL", or "ADVANCED_TERMINAL"
disp(['Current mode: ', m]);

This function is implemented in modules/engine/builtin/cpp/getnelsonmodeBuiltin.cpp (lines 25-33), which maps the internal enum value to a human-readable string.

Setting the Engine Mode in C++

For applications embedding the Nelson interpreter, you can force a specific mode before initialization:

#include "NelsonConfiguration.hpp"

int main(int argc, char* argv[])
{
    // Force GUI mode
    NelsonConfiguration::getInstance()->setNelsonEngineMode(NELSON_ENGINE_MODE::GUI);
    // Start the engine (the mode flag is read internally by StartNelson)
    return Nelson::StartNelsonA(argc, argv, NELSON_ENGINE_MODE::GUI);
}

The StartNelson.cpp implementation reads the mode from NelsonConfiguration during startup to determine the execution environment.

Launching the Embedded Editor

To open the embedded script editor from within a running session:

edit;   % invokes editorBuiltin.cpp

This command checks NelsonConfiguration::useEmbeddedEditor() and only opens the editor window when the configuration flag is true, allowing the same codebase to support both editor-enabled and editor-disabled deployments.

Summary

• CLI modes (BASIC_TERMINAL, ADVANCED_TERMINAL) run Nelson in a console without or with graphics support, initialized via nelson_cli or nelson_adv_cli.
• GUI mode provides the full desktop environment through nelson_gui, creating GuiTerminal and GUI objects via InitGuiObjectsDynamic().
• Embedded mode controls optional UI components like the script editor through the useEmbeddedEditorFlag checked by NelsonConfiguration::useEmbeddedEditor().
• The active mode is stored in NelsonConfiguration::engineMode and can be queried using getnelsonmode().
• Mode selection occurs in StartNelson.cpp and MainEvaluator.cpp, which dispatch to the appropriate evaluator implementation based on the startup executable.

Frequently Asked Questions

What is the difference between nelson_cli and nelson_adv_cli?

nelson_cli starts Nelson in BASIC_TERMINAL mode with pure text I/O and no graphics capabilities, while nelson_adv_cli uses ADVANCED_TERMINAL mode to provide a console interface that can still display plots in separate windows. The distinction is made in their respective main files (main-cli.cpp vs main-adv-cli.cpp) through different NELSON_ENGINE_MODE flags passed to the engine initialization.

Can I switch from CLI mode to GUI mode without restarting Nelson?

No, the engine mode is determined at startup by the executable entry point and stored in NelsonConfiguration::engineMode. While you can query the current mode using getnelsonmode(), switching between CLI and GUI requires restarting the interpreter with a different executable (nelson_gui vs nelson_cli) or programmatically setting the mode flag before calling StartNelsonA() in an embedded context.

What does the embedded editor flag control?

The embedded editor flag (useEmbeddedEditorFlag) controls whether optional UI components like the script editor and help viewer are available within a running Nelson session. According to NelsonConfiguration.cpp, this runtime flag determines if the edit command (implemented in editorBuiltin.cpp) will open an embedded editor window or return an error, allowing deployments to exclude editor functionality even in GUI mode.

How do I programmatically detect which engine mode is active?

Use the getnelsonmode() built-in function from Nelson scripts, which returns strings like "GUI" or "BASIC_TERMINAL" based on the current NelsonConfiguration::engineMode value. In C++ applications, you can access the same configuration directly through NelsonConfiguration::getInstance()->getNelsonEngineMode() to adjust application behavior based on the available UI capabilities.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}