# How Probe Handles Embedded Jobs and Composite Workflows: A Technical Deep Dive

> Discover how Probe handles embedded jobs and composite workflows by referencing external files and sharing context. Learn the technical details for efficient workflow management.

- Repository: [Tomohisa Oda/probe](https://github.com/linyows/probe)
- Tags: deep-dive
- Published: 2026-03-06

---

**Probe implements composite workflows by treating steps with `uses: embedded` as references to external workflow files, loading and executing them independently while sharing variable context and output handling with the parent job.**

The `linyows/probe` repository provides a sophisticated mechanism for **embedded jobs** (also called *composite workflows*) that enables reusable, modular workflow design. This capability allows you to decompose complex automation into smaller, testable sub-workflows that execute within the context of a parent job. Understanding how Probe handles these composite structures requires examining the path resolution, independent execution, and rendering logic implemented across the codebase.

## How Embedded Jobs Work in Probe

Probe's composite workflow system operates on a simple but powerful abstraction: any step specifying `uses: embedded` triggers a specialized execution path that loads and runs a separate workflow file. This approach differs from simple action plugins because the embedded file is parsed as a complete `Job` struct with its own steps, defaults, and execution context.

The mechanism relies on three core principles:

- **Relative Path Resolution**: Embedded workflows are resolved relative to the parent workflow's directory using the `basePath` established during initial parsing
- **Independent Execution**: Each embedded job runs in its own printer context via `job.RunIndependently`, isolating output while preserving the parent's variable scope
- **Flattened Visualization**: DAG renderers expand embedded steps inline, showing the complete composite structure in both Mermaid and ASCII formats

## Three-Phase Execution Architecture

Probe's handling of embedded jobs follows a distinct three-phase lifecycle implemented across [`dag.go`](https://github.com/linyows/probe/blob/main/dag.go), [`embedded/client.go`](https://github.com/linyows/probe/blob/main/embedded/client.go), and the rendering engines.

### Phase 1: Path Resolution and Workflow Loading

When Probe parses the initial workflow in `probe.go:91-97`, it stores the directory of the first YAML file in `workflow.basePath`. This path serves as the anchor for all relative lookups of embedded jobs.

The resolution process occurs in `dag.go:60-95` within `DagRendererBase.ResolvePath`. This method:

1. Expands variables in the step's `with.path` using workflow context
2. Resolves the absolute path relative to `basePath` using `filepath.Abs`
3. Validates file existence before attempting to load

The `LoadEmbeddedJob` function (`dag.go:98-117`) then reads the referenced file and unmarshals it into a `probe.Job` struct, preparing it for execution.

### Phase 2: Independent Job Execution

The embedded action plugin (`actions/embedded/main.go:30-38`) creates an `embedded.Request` and invokes `embedded.Execute`. The actual execution logic in `embedded/client.go:65-100` handles several critical tasks:

- **Defaults Propagation**: `applyDefaultsToSteps` merges any `defaults` defined in the embedded job into its individual steps
- **Isolated Output**: A fresh printer instance (`probe.NewPrinter(true, []string{jobID})`) ensures the embedded job's output streams are captured separately
- **Error Classification**: System-level failures (file not found, YAML parse errors) return as Go `error` types and abort the workflow, while job-level failures populate `Result.Res.Error` allowing the parent workflow to continue

The embedded job runs via `job.RunIndependently`, returning success status, outputs, and reports that merge into the parent step's result map through standard `Step.finalize` handling.

### Phase 3: DAG Rendering and Visualization

Both `DagMermaidRenderer` (`dag_mermaid.go:65-84`) and `DagAsciiRenderer` ([`dag_ascii.go`](https://github.com/linyows/probe/blob/main/dag_ascii.go)) implement expansion logic for composite workflows. When rendering:

1. The renderer detects `step.Uses == "embedded"`
2. Calls `ExpandPath` and `ResolvePath` to locate the embedded file
3. Loads the embedded job via `LoadEmbeddedJob`
4. Iterates over `embeddedJob.Steps` to emit sub-nodes inline

This produces a flattened visual representation showing the full composite workflow hierarchy without requiring manual documentation of the sub-workflow structure.

## Code Example: Creating a Composite Workflow

The following example demonstrates a parent workflow that embeds an integration testing suite as a reusable sub-workflow:

```yaml

# parent.yml

jobs:
  - name: Deploy
    steps:
      - name: Run unit tests
        uses: shell
        with:
          cmd: go test ./...
      - name: Run integration suite (embedded)
        uses: embedded
        with:
          path: ./embedded/integration.yml
          vars:
            env: staging

```

```yaml

# embedded/integration.yml

jobs:
  - name: Integration
    steps:
      - name: Start containers
        uses: docker
        with:
          compose: docker-compose.yml
      - name: Run tests
        uses: shell
        with:
          cmd: go test -tags=integration ./...

```

When executed, Probe loads [`embedded/integration.yml`](https://github.com/linyows/probe/blob/main/embedded/integration.yml) relative to [`parent.yml`](https://github.com/linyows/probe/blob/main/parent.yml)'s directory, passes the `vars.env` variable into the embedded context, and executes the integration steps sequentially. The outputs from both "Start containers" and "Run tests" become available to subsequent steps in the parent workflow. Running `probe render --format mermaid parent.yml` displays all four steps in a single composite graph.

## Key Implementation Files

Understanding Probe's embedded job architecture requires familiarity with these specific source files:

- **[`probe.go`](https://github.com/linyows/probe/blob/main/probe.go)** – Establishes `workflow.basePath` for relative path resolution during initial workflow loading
- **[`dag.go`](https://github.com/linyows/probe/blob/main/dag.go)** – Contains `DagRendererBase.ResolvePath` (lines 60-95) and `LoadEmbeddedJob` (lines 98-117) for path handling and workflow loading
- **[`embedded/client.go`](https://github.com/linyows/probe/blob/main/embedded/client.go)** – Implements the request lifecycle including parsing, defaults application, and independent execution via `job.RunIndependently`
- **[`actions/embedded/main.go`](https://github.com/linyows/probe/blob/main/actions/embedded/main.go)** – Provides the plugin glue that invokes the embedded client from the action system
- **[`dag_mermaid.go`](https://github.com/linyows/probe/blob/main/dag_mermaid.go) and [`dag_ascii.go`](https://github.com/linyows/probe/blob/main/dag_ascii.go)** – Contain the rendering logic (lines 65-84 in the Mermaid implementation) that expands embedded steps for visualization
- **[`step.go`](https://github.com/linyows/probe/blob/main/step.go)** – Executes the embedded action and handles result aggregation through the standard step output mechanism

## Summary

Probe handles embedded jobs and composite workflows through a robust three-phase system:

- **Path resolution** uses `workflow.basePath` and `DagRendererBase.ResolvePath` to locate embedded workflow files relative to the parent
- **Independent execution** via [`embedded/client.go`](https://github.com/linyows/probe/blob/main/embedded/client.go) runs sub-workflows in isolated printer contexts while sharing variable scope, with distinct handling for system-level versus job-level errors
- **Visual expansion** in DAG renderers produces complete workflow diagrams by inline-expanding embedded steps during Mermaid or ASCII generation
- **Result aggregation** merges embedded outputs into the parent job's step results, enabling complex workflow-of-workflows patterns without custom orchestration

## Frequently Asked Questions

### How does Probe resolve paths for embedded workflow files?

Probe resolves embedded workflow paths relative to the parent workflow's directory using the `basePath` variable set during initial parsing in `probe.go:91-97`. The `DagRendererBase.ResolvePath` method in `dag.go:60-95` expands workflow variables in the path string, then uses `filepath.Abs` to resolve the absolute path relative to `basePath`, verifying file existence before loading.

### What happens when an embedded job fails?

Probe distinguishes between system-level failures and job-level failures. System errors (file not found, YAML parse errors) return as Go `error` types and abort the entire workflow. Job execution failures (failed steps within the embedded workflow) populate the `Result.Res.Error` field, allowing the parent workflow to continue execution and handle the failure according to its own error handling logic.

### Can embedded jobs access variables from the parent workflow?

Yes, embedded jobs share the same variable context as the parent workflow. When defining an embedded step, the `with.vars` map passes variables into the embedded job's evaluation context. These variables become available for expansion within the embedded workflow file's step definitions, enabling dynamic configuration of reusable sub-workflows.

### How does Probe visualize composite workflows in DAG renders?

Both `DagMermaidRenderer` (`dag_mermaid.go:65-84`) and `DagAsciiRenderer` detect steps with `uses: embedded`, load the referenced workflow file via `LoadEmbeddedJob`, and iterate over the embedded job's steps to emit them as sub-nodes inline. This produces a flattened visual representation showing the complete composite workflow hierarchy rather than treating the embedded job as an opaque single node.