Core Components of the Probe Workflow Engine: A Deep Dive into linyows/probe

The Probe workflow engine consists of ten tightly-coupled Go types—Workflow, Job, Step, JobScheduler, Executor, Result, Expr, Printer, and Outputs—that transform YAML definitions into a dependency-aware, repeatable execution pipeline.

The Probe workflow engine is an open-source automation framework written in Go that converts declarative YAML configurations into reliable execution pipelines. Developed in the linyows/probe repository, this engine emphasizes dependency management, safe expression evaluation, and observable execution through a carefully architected set of core components.

Workflow: The Top-Level Orchestrator

The Workflow type serves as the primary entry point and global coordinator. Defined in workflow.go, this component handles YAML parsing, global variable evaluation, and the initialization of the execution environment.

The Workflow.Start method performs four critical functions: it loads and validates the workflow definition, evaluates global variables using the expression engine, creates a JobScheduler instance, and initiates the execution loop. Upon completion, it delegates to the Printer component to render the final execution report.

Job: Logical Units of Work with Dependencies

The Job type represents a logical unit of work within the pipeline. Defined in job.go, each job maintains a collection of Step instances, declares execution dependencies via the needs field, and supports conditional execution through skipif guards.

The Job.Start method orchestrates the execution of all steps within the job, handling iteration when repeat configurations are specified. Jobs can declare dependencies on other jobs, creating a directed acyclic graph (DAG) that the scheduler uses to determine execution order.

Step: Individual Actions and Execution Logic

The Step type handles the granular execution of individual actions. Located in step.go, the Step.Do method implements the core execution logic including template preparation, conditional skipping, retry loops, timeout handling, and output extraction.

Steps support pluggable actions through the actionRunner abstraction, allowing the engine to execute HTTP requests, SSH commands, database queries, or custom logic without coupling to the core implementation. The uses field determines which action runner to invoke, while the with field provides parameterized configuration.

JobScheduler: DAG-Based Dependency Management

The JobScheduler constructs and manages the execution graph. Defined in scheduler.go, this component builds a DAG from job definitions, validates IDs and dependency cycles, and tracks execution status and repeat counters.

The JobScheduler.AddJob method registers jobs with the scheduler, while GetRunnableJobs returns jobs whose dependencies have completed successfully. The scheduler ensures that jobs run only after all entries in their needs list have finished, even when jobs are configured to repeat multiple times.

Executor: Managing Job Execution and Repetition

The Executor handles the runtime execution of individual jobs. Located in executor.go, the Executor.Execute method manages both synchronous and asynchronous execution modes, buffers step results, and finalizes job status.

The executor supports job-level repetition through the repeat configuration, allowing jobs to run multiple times either sequentially or concurrently based on the async flag. Results are buffered during execution and committed to the central Result object upon completion.

Supporting Infrastructure

Result Tracking and Outputs

The Result type provides in-memory storage for execution outcomes. Defined in result.go, the Result.AddStepResult method records start times, end times, success flags, and per-step outcomes for all jobs.

The Outputs component serves as a centralized store for values exported by steps. Located in outputs.go, the Outputs.Set method makes step outputs available to subsequent steps and jobs through template interpolation, enabling data flow between dependent jobs.

Safe Expression Evaluation with Expr

The Expr component provides secure template evaluation and conditional logic. Defined in expr.go, the Expr.EvalTemplate method processes variable interpolation, evaluates skipif conditions, and provides helper functions like random_int and parse_json.

This engine sanitizes environment access, enforces size limits, and implements timeout protection to mitigate injection attacks and denial-of-service scenarios.

User Interface and Reporting

The Printer component handles all terminal output. Located in printer.go, this type manages execution spinners, colored log output, and the final report rendering including DAG visualization.

Architectural Flow

The execution flow follows a precise orchestration pattern:

1. Workflow.Start initializes the JobScheduler and begins the execution loop
2. The scheduler validates dependencies and exposes runnable jobs via GetRunnableJobs
3. Executor.Execute runs each job, delegating to Job.Start
4. Job.Start iterates over its Steps, with each Step.Do handling preparation, execution, and result recording
5. All results flow into the shared Result object, which Printer renders upon completion

Summary

• Workflow orchestrates the entire execution lifecycle from YAML parsing to final reporting
• Job defines logical work units with dependency declarations and conditional execution
• Step implements individual actions with support for retries, timeouts, and output extraction
• JobScheduler manages the DAG-based dependency graph and determines execution order
• Executor handles job runtime, repetition, and result buffering
• Expr provides secure template evaluation and conditional logic
• Result and Outputs maintain execution state and enable data flow between components

Frequently Asked Questions

What is the Probe workflow engine used for?

The Probe workflow engine is designed for automating complex operational tasks such as infrastructure testing, deployment verification, and multi-step API workflows. It excels at scenarios requiring dependency management between tasks, conditional execution based on previous results, and safe handling of sensitive data through its expression engine.

How does Probe handle dependencies between jobs?

Probe constructs a directed acyclic graph (DAG) using the JobScheduler component, which validates that all job IDs referenced in needs declarations exist and contain no circular dependencies. The scheduler only returns jobs via GetRunnableJobs when all their declared dependencies have completed successfully, ensuring proper execution order even when jobs are configured to repeat multiple times.

Can Probe execute jobs concurrently?

Yes, the Executor component supports both synchronous and asynchronous execution modes through the repeat configuration's async flag. When enabled, jobs can run multiple instances concurrently, with the scheduler managing the lifecycle and the executor buffering results until all iterations complete.

What security measures does Probe implement for expression evaluation?

The Expr component implements multiple security layers including environment variable sanitization, size limits on evaluated expressions, and timeout protection to prevent denial-of-service attacks. It uses the expr-lang library with restricted functionality, providing only safe helper functions like random_int and parse_json while preventing arbitrary code execution.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}