# Core Components of the Probe Workflow Engine: A Deep Dive into linyows/probe > Explore the core components of the linyows probe workflow engine. Discover how Workflow, Job, Step, and more create a dependency-aware execution pipeline from YAML. - Repository: [Tomohisa Oda/probe](https://github.com/linyows/probe) - Tags: deep-dive - Published: 2026-03-06 --- **The Probe workflow engine consists of ten tightly-coupled Go types—Workflow, Job, Step, JobScheduler, Executor, Result, Expr, Printer, and Outputs—that transform YAML definitions into a dependency-aware, repeatable execution pipeline.** The **Probe workflow engine** is an open-source automation framework written in Go that converts declarative YAML configurations into reliable execution pipelines. Developed in the `linyows/probe` repository, this engine emphasizes dependency management, safe expression evaluation, and observable execution through a carefully architected set of core components. ## Workflow: The Top-Level Orchestrator The **Workflow** type serves as the primary entry point and global coordinator. Defined in [`workflow.go`](https://github.com/linyows/probe/blob/main/workflow.go), this component handles YAML parsing, global variable evaluation, and the initialization of the execution environment. The `Workflow.Start` method performs four critical functions: it loads and validates the workflow definition, evaluates global variables using the expression engine, creates a `JobScheduler` instance, and initiates the execution loop. Upon completion, it delegates to the `Printer` component to render the final execution report. ## Job: Logical Units of Work with Dependencies The **Job** type represents a logical unit of work within the pipeline. Defined in [`job.go`](https://github.com/linyows/probe/blob/main/job.go), each job maintains a collection of `Step` instances, declares execution dependencies via the `needs` field, and supports conditional execution through `skipif` guards. The `Job.Start` method orchestrates the execution of all steps within the job, handling iteration when repeat configurations are specified. Jobs can declare dependencies on other jobs, creating a directed acyclic graph (DAG) that the scheduler uses to determine execution order. ## Step: Individual Actions and Execution Logic The **Step** type handles the granular execution of individual actions. Located in [`step.go`](https://github.com/linyows/probe/blob/main/step.go), the `Step.Do` method implements the core execution logic including template preparation, conditional skipping, retry loops, timeout handling, and output extraction. Steps support **pluggable actions** through the `actionRunner` abstraction, allowing the engine to execute HTTP requests, SSH commands, database queries, or custom logic without coupling to the core implementation. The `uses` field determines which action runner to invoke, while the `with` field provides parameterized configuration. ## JobScheduler: DAG-Based Dependency Management The **JobScheduler** constructs and manages the execution graph. Defined in [`scheduler.go`](https://github.com/linyows/probe/blob/main/scheduler.go), this component builds a DAG from job definitions, validates IDs and dependency cycles, and tracks execution status and repeat counters. The `JobScheduler.AddJob` method registers jobs with the scheduler, while `GetRunnableJobs` returns jobs whose dependencies have completed successfully. The scheduler ensures that jobs run only after all entries in their `needs` list have finished, even when jobs are configured to repeat multiple times. ## Executor: Managing Job Execution and Repetition The **Executor** handles the runtime execution of individual jobs. Located in [`executor.go`](https://github.com/linyows/probe/blob/main/executor.go), the `Executor.Execute` method manages both synchronous and asynchronous execution modes, buffers step results, and finalizes job status. The executor supports job-level repetition through the `repeat` configuration, allowing jobs to run multiple times either sequentially or concurrently based on the `async` flag. Results are buffered during execution and committed to the central `Result` object upon completion. ## Supporting Infrastructure ### Result Tracking and Outputs The **Result** type provides in-memory storage for execution outcomes. Defined in [`result.go`](https://github.com/linyows/probe/blob/main/result.go), the `Result.AddStepResult` method records start times, end times, success flags, and per-step outcomes for all jobs. The **Outputs** component serves as a centralized store for values exported by steps. Located in [`outputs.go`](https://github.com/linyows/probe/blob/main/outputs.go), the `Outputs.Set` method makes step outputs available to subsequent steps and jobs through template interpolation, enabling data flow between dependent jobs. ### Safe Expression Evaluation with Expr The **Expr** component provides secure template evaluation and conditional logic. Defined in [`expr.go`](https://github.com/linyows/probe/blob/main/expr.go), the `Expr.EvalTemplate` method processes variable interpolation, evaluates `skipif` conditions, and provides helper functions like `random_int` and `parse_json`. This engine sanitizes environment access, enforces size limits, and implements timeout protection to mitigate injection attacks and denial-of-service scenarios. ### User Interface and Reporting The **Printer** component handles all terminal output. Located in [`printer.go`](https://github.com/linyows/probe/blob/main/printer.go), this type manages execution spinners, colored log output, and the final report rendering including DAG visualization. ## Architectural Flow The execution flow follows a precise orchestration pattern: 1. `Workflow.Start` initializes the `JobScheduler` and begins the execution loop 2. The scheduler validates dependencies and exposes runnable jobs via `GetRunnableJobs` 3. `Executor.Execute` runs each job, delegating to `Job.Start` 4. `Job.Start` iterates over its `Step`s, with each `Step.Do` handling preparation, execution, and result recording 5. All results flow into the shared `Result` object, which `Printer` renders upon completion ## Summary - **Workflow** orchestrates the entire execution lifecycle from YAML parsing to final reporting - **Job** defines logical work units with dependency declarations and conditional execution - **Step** implements individual actions with support for retries, timeouts, and output extraction - **JobScheduler** manages the DAG-based dependency graph and determines execution order - **Executor** handles job runtime, repetition, and result buffering - **Expr** provides secure template evaluation and conditional logic - **Result** and **Outputs** maintain execution state and enable data flow between components ## Frequently Asked Questions ### What is the Probe workflow engine used for? The Probe workflow engine is designed for automating complex operational tasks such as infrastructure testing, deployment verification, and multi-step API workflows. It excels at scenarios requiring dependency management between tasks, conditional execution based on previous results, and safe handling of sensitive data through its expression engine. ### How does Probe handle dependencies between jobs? Probe constructs a directed acyclic graph (DAG) using the `JobScheduler` component, which validates that all job IDs referenced in `needs` declarations exist and contain no circular dependencies. The scheduler only returns jobs via `GetRunnableJobs` when all their declared dependencies have completed successfully, ensuring proper execution order even when jobs are configured to repeat multiple times. ### Can Probe execute jobs concurrently? Yes, the `Executor` component supports both synchronous and asynchronous execution modes through the `repeat` configuration's `async` flag. When enabled, jobs can run multiple instances concurrently, with the scheduler managing the lifecycle and the executor buffering results until all iterations complete. ### What security measures does Probe implement for expression evaluation? The `Expr` component implements multiple security layers including environment variable sanitization, size limits on evaluated expressions, and timeout protection to prevent denial-of-service attacks. It uses the expr-lang library with restricted functionality, providing only safe helper functions like `random_int` and `parse_json` while preventing arbitrary code execution.