Error Handling Mechanisms for Stack Overflow and Stack Underflow in NanoCore

NanoCore detects stack violations by checking the stack pointer against fixed bounds (0xEA minimum and 0xFF maximum) before every PUSH, POP, CALL, CALLR, and RET instruction, returning typed EmulatorError variants that preserve the offending pointer value without modifying CPU state.

NanoCore is a compact 8-bit emulator implemented in Rust that manages program execution within a strict 256-byte memory model. Understanding the error handling mechanisms for stack overflow and stack underflow in NanoCore is essential for writing robust assembly programs and debugging runtime failures. The system employs compile-time constants and runtime guards to ensure stack operations remain within the reserved memory region.

How NanoCore Defines Stack Boundaries

The NanoCore CPU treats the stack as a fixed region within the 256-byte address space. In src/cpu.rs (lines 54-56), the implementation defines two critical constants:

• STACK_MAX = 0xFF — The initial stack pointer value representing the top of memory.
• STACK_MIN = 0xEA (decimal 234) — The lowest valid address for stack data, reserving 22 bytes for stack operations.

The stack pointer (SP) grows downwards from 0xFF toward 0xEA. Any operation that would move SP below STACK_MIN or above STACK_MAX triggers an immediate error.

Error Variants for Stack Violations

NanoCore uses Rust’s type system to provide explicit error variants defined in src/error.rs (lines 31-34). When the emulator detects an invalid stack operation, it aborts the current cycle and returns one of two specific errors:

• EmulatorError::StackOverflow { sp } — Raised when a PUSH, CALL, or CALLR instruction attempts to decrement the stack pointer below STACK_MIN. The error carries the offending sp value for debugging.
• EmulatorError::StackUnderflow { sp } — Raised when a POP or RET instruction attempts to increment the stack pointer above STACK_MAX. This indicates an attempt to read from an empty stack.

These errors propagate through the public API methods NanoCore::cycle() and NanoCore::run(), allowing calling code to handle violations gracefully via pattern matching.

Runtime Detection in Instruction Execution

The emulator performs boundary checks immediately before modifying the stack pointer or memory. Each stack-manipulating instruction contains a guard condition that validates the current SP value.

PUSH and Stack Overflow

In src/nanocore.rs (lines 413-416), the PUSH instruction implementation checks:

if self.cpu.sp == CPU::STACK_MIN {
    return Err(EmulatorError::StackOverflow { sp: self.cpu.sp });
}

If the stack pointer has already reached the minimum bound, the operation aborts before writing to memory.

POP and Stack Underflow

The POP instruction in src/nanocore.rs (lines 335-337) validates the upper bound:

if self.cpu.sp == CPU::STACK_MAX {
    return Err(EmulatorError::StackUnderflow { sp: self.cpu.sp });
}

This prevents reading from addresses above the stack region.

CALL and CALLR Operations

Subroutine calls use the stack to store return addresses. Both CALL (lines 733-735) and CALLR (lines 755-757) in src/nanocore.rs perform the same overflow check as PUSH, ensuring sufficient space exists to save the program counter before jumping to the subroutine.

RET Instruction

The RET instruction in src/nanocore.rs (lines 670-672) checks for underflow before attempting to pop the return address. If the stack is empty (SP at 0xFF), the error returns immediately.

When any guard condition fails, the emulator does not modify the program counter or registers, preserving CPU state for inspection and recovery.

Practical Examples

Triggering Stack Overflow

The following Rust code demonstrates how excessive push operations trigger the overflow error:

use nanocore::{EmulatorError, nanocore::NanoCore};

fn overflow_demo() -> Result<(), EmulatorError> {
    // Create a program that pushes 10 values onto the stack.
    // The stack pointer starts at 0xFF; we set it to 0xEB (just above the minimum)
    // so the 10th PUSH will hit STACK_MIN (0xEA) and overflow.
    let mut prog = vec![];
    for _ in 0..10 {
        prog.push(0x07); // PUSH opcode
        prog.push(0x00); // operand: R0
    }
    prog.push(0x00); // HLT

    let mut nano = NanoCore::new();
    nano.load_program(&prog, 0)?;
    nano.cpu.sp = 0xEB; // one byte above STACK_MIN

    // Run until the overflow occurs.
    loop {
        match nano.cycle() {
            Ok(_) => continue,
            Err(EmulatorError::StackOverflow { sp }) => {
                println!("Stack overflow detected at SP={:#04X}", sp);
                return Ok(());
            }
            Err(e) => panic!("Unexpected error: {:?}", e),
        }
    }
}

Running overflow_demo() outputs:

Stack overflow detected at SP=0xEA

The test suite validates this behavior in tests/error_handling_tests.rs (lines 31-38).

Triggering Stack Underflow

This example shows how returning without a corresponding call creates an underflow:

use nanocore::{EmulatorError, nanocore::NanoCore};

fn underflow_demo() -> Result<(), EmulatorError> {
    // Assemble a single RET instruction (opcode 0x0F).
    // Without a preceding CALL the stack pointer stays at its initial value (0xFF),
    // so RET will try to pop from an empty stack.
    let prog = vec![0x0F, 0x00, 0x00]; // RET, followed by HLT for safety

    let mut nano = NanoCore::new();
    nano.load_program(&prog, 0)?;
    match nano.cycle() {
        Err(EmulatorError::StackUnderflow { sp }) => {
            println!("Stack underflow detected at SP={:#04X}", sp);
            Ok(())
        }
        other => panic!("Expected underflow, got: {:?}", other),
    }
}

Output:

Stack underflow detected at SP=0xFF

The corresponding unit test resides in tests/error_handling_tests.rs (lines 47-58).

Summary

• NanoCore implements a fixed 256-byte memory model with the stack occupying addresses 0xEA to 0xFF.
• Stack overflow occurs when PUSH, CALL, or CALLR instructions attempt to decrement the stack pointer below STACK_MIN (0xEA), triggering EmulatorError::StackOverflow.
• Stack underflow occurs when POP or RET instructions attempt to increment the stack pointer above STACK_MAX (0xFF), triggering EmulatorError::StackUnderflow.
• All boundary checks reside in src/nanocore.rs and prevent CPU state modification when violations are detected, enabling safe error recovery through Rust’s Result type.

Frequently Asked Questions

What is the maximum stack size in NanoCore?

The maximum stack size is 22 bytes, occupying addresses 0xEA through 0xFF in the 256-byte address space. The lower bound is defined as STACK_MIN = 0xEA in src/cpu.rs, while STACK_MAX remains at 0xFF.

How does NanoCore prevent stack corruption?

NanoCore prevents corruption by validating the stack pointer before executing any stack-modifying instruction. If a PUSH or CALL would move SP below STACK_MIN, or a POP or RET would move it above STACK_MAX, the emulator immediately returns an EmulatorError without writing to memory or updating registers.

Can I catch stack errors programmatically?

Yes. Both NanoCore::cycle() and NanoCore::run() return Result types containing EmulatorError variants. You can pattern match on EmulatorError::StackOverflow { sp } or EmulatorError::StackUnderflow { sp } to implement recovery logic or graceful shutdowns in your host application.

What happens to the CPU state when a stack error occurs?

When either overflow or underflow is detected, the emulator aborts the current instruction without modifying the program counter or any registers. The stack pointer remains at its original value, and memory contents are unchanged. This atomic failure mode allows developers to inspect the exact machine state at the moment of violation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}