# How gh-aw Compiles Markdown Workflows into GitHub Actions YAML: The 4-Phase Pipeline Explained

> Discover how gh-aw compiles markdown workflows into GitHub Actions YAML using its four-phase pipeline. Learn about the parse, validate, generate, and final validation steps.

- Repository: [GitHub/gh-aw](https://github.com/github/gh-aw)
- Tags: deep-dive
- Published: 2026-02-16

---

**The `gh-aw` CLI compiles markdown workflows into GitHub Actions YAML through a four-phase pipeline—parse and load, validate, generate YAML, and final validation—implemented across [`compiler.go`](https://github.com/github/gh-aw/blob/main/compiler.go) and [`compiler_yaml.go`](https://github.com/github/gh-aw/blob/main/compiler_yaml.go) in the `github/gh-aw` repository.**

When you run `gh aw compile`, you trigger a sophisticated compilation process that transforms human-readable markdown into the machine-readable [`.lock.yml`](https://github.com/github/gh-aw/blob/main/.lock.yml) files that GitHub Actions executes. This article breaks down exactly how `gh-aw` compiles markdown workflows into GitHub Actions YAML, referencing the actual source implementation in the `github/gh-aw` repository.

## The Four-Phase Compilation Pipeline

The compilation process is logically divided into four distinct phases, each handled by specific functions in the `pkg/workflow` directory.

### Phase 1: Parse and Load

The pipeline begins in [`pkg/workflow/compiler.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/compiler.go) with the `CompileWorkflow` method. This function orchestrates the initial parsing by calling `ParseWorkflowFile`, which reads the markdown file from disk and extracts the YAML front-matter from the markdown body.

The parser builds a `WorkflowData` structure that holds everything the compiler needs, including workflow metadata, permissions, and the raw markdown content. The parser also computes a **front-matter hash** via `parser.ComputeFrontmatterHashFromFile` to ensure reproducibility—this hash is later embedded as a comment in the generated YAML.

### Phase 2: Validate

Once loaded, the workflow data passes through `validateWorkflowData` in [`compiler.go`](https://github.com/github/gh-aw/blob/main/compiler.go) (lines 14-63). This phase performs comprehensive safety checks before any YAML generation occurs:

- **Expression safety** – `validateExpressionSafety` ensures only whitelisted `${{ … }}` patterns are used
- **Runtime-import validation** – `validateRuntimeImportFiles` guarantees imported markdown files can be loaded at runtime
- **Permission and feature-flag checks** – Validates dangerous permissions, safe-output settings, network firewall configurations, and concurrency expressions
- **Tool-set consistency** – Ensures all referenced tools are compatible

Errors and warnings are emitted using the `console` package via `formatCompilerError` and `formatCompilerMessage`, providing formatted console output with consistent styling.

### Phase 3: Generate YAML

The core transformation happens in `generateYAML` (located in [`pkg/workflow/compiler_yaml.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/compiler_yaml.go), lines 59-92). This phase constructs the actual GitHub Actions YAML through several sub-components:

**Job Construction** – `buildJobs` creates job objects, resolves inter-job dependencies, and validates against duplicate step IDs using the internal `JobManager`.

**Header Generation** – `generateWorkflowHeader` writes a structured comment block containing:
- The `gh-aw` logo and version
- Source file reference
- Description metadata
- Import/include manifest
- Front-matter hash for reproducibility

**Body Generation** – `generateWorkflowBody` writes top-level workflow fields (`name`, `on`, `permissions`, `concurrency`, `env`) using deterministic ordering via `yaml.MarshalWithFieldOrder` from [`pkg/workflow/yaml.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/yaml.go).

**Prompt Chunking** – The markdown body is split by `splitContentIntoChunks` (max 20KB per chunk, max 5 chunks) to obey GitHub Actions step size limits. Each chunk becomes a heredoc block in a unified "prompt generation" step created by `generateUnifiedPromptCreationStep`. The `ExpressionExtractor` replaces `${{ }}` expressions with environment variable placeholders for safe runtime interpolation.

**Post-Steps** – Additional steps are appended for safe-output collection (`generateOutputCollectionStep`), artifact upload, and workflow overview generation (`generateWorkflowOverviewStep`).

### Phase 4: Final Validation and Output

The final phase occurs in `generateAndValidateYAML` and `writeWorkflowOutput` (back in [`compiler.go`](https://github.com/github/gh-aw/blob/main/compiler.go)). Before writing to disk, the compiler performs rigorous validation:

- **Expression size limits** – `validateExpressionSizes` ensures no expression exceeds 21KB
- **Template injection prevention** – `validateNoTemplateInjection` guards against malicious template patterns
- **Schema validation** – `validateGitHubActionsSchema` validates against the embedded [`schemas/github-workflow.json`](https://github.com/github/gh-aw/blob/main/schemas/github-workflow.json) (unless `--no-validation` is used)
- **Container and runtime checks** – Validates container images, runtime packages, and repository features

If validation passes, `writeWorkflowOutput` writes the content to `<filename>.lock.yml`. This function:
- Skips writing if the content is unchanged (preserving timestamps)
- Warns if the file exceeds the 500KB recommended limit (`MaxLockFileSize`)
- Prints a success message with the relative path and file size

If validation fails, an `*.invalid.yml` file is written for debugging, and a formatted console error is returned.

## Key Implementation Files in github/gh-aw

Understanding the compilation pipeline requires familiarity with these specific files in the repository:

| File | Role | Key Functions |
|------|------|---------------|
| [`pkg/workflow/compiler.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/compiler.go) | Core orchestration and validation | `CompileWorkflow`, `validateWorkflowData`, `generateAndValidateYAML`, `writeWorkflowOutput` |
| [`pkg/workflow/compiler_yaml.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/compiler_yaml.go) | YAML generation and prompt chunking | `generateYAML`, `buildJobs`, `splitContentIntoChunks`, `generateUnifiedPromptCreationStep` |
| [`pkg/workflow/yaml.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/yaml.go) | Deterministic YAML marshaling | `MarshalWithFieldOrder`, `UnquoteYAMLKey`, `CleanYAMLNullValues` |
| [`pkg/parser/frontmatter.go`](https://github.com/github/gh-aw/blob/main/pkg/parser/frontmatter.go) | Front-matter parsing and hashing | `ParseWorkflowFile`, `ComputeFrontmatterHashFromFile` |
| [`pkg/workflow/job_manager.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/job_manager.go) | Job dependency resolution | `JobManager`, `buildJobs` |
| [`pkg/workflow/expressions.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/expressions.go) | Expression safety and extraction | `validateExpressionSafety`, `ExpressionExtractor` |
| [`schemas/github-workflow.json`](https://github.com/github/gh-aw/blob/main/schemas/github-workflow.json) | Embedded validation schema | Used by `validateGitHubActionsSchema` |

## Practical Usage Examples

### Compiling from the CLI

The most common way to compile markdown workflows into GitHub Actions YAML is using the `gh aw compile` command:

```bash
$ gh aw compile .github/workflows/example.md
⚙️  Parsing workflow file
✅  Successfully generated workflow (example.md) (12 KB)

```

This creates [`.github/workflows/example.lock.yml`](https://github.com/github/gh-aw/blob/main/.github/workflows/example.lock.yml) containing the compiled GitHub Actions workflow. The CLI automatically handles all four compilation phases, including validation against the embedded JSON schema.

### Programmatic Compilation in Go

You can also invoke the compiler programmatically within your own Go applications:

```go
package main

import (
    "log"

    "github.com/github/gh-aw/pkg/workflow"
)

func main() {
    comp := workflow.NewCompiler()
    // optional: comp.SetStrictMode(true) // treat missing permissions as errors
    
    if err := comp.CompileWorkflow(".github/workflows/example.md"); err != nil {
        log.Fatalf("Compilation failed: %v", err)
    }
}

```

The `CompileWorkflow` method triggers the full pipeline: parsing the markdown, validating safety constraints, generating the YAML with chunked prompts, and writing the [`.lock.yml`](https://github.com/github/gh-aw/blob/main/.lock.yml) file.

### Inspecting the Generated Lock File

After compilation, examine the generated file to verify the transformation:

```bash
$ cat .github/workflows/example.lock.yml | head -n 20

# Generated by gh-aw (github.com/github/gh-aw)

# Source: .github/workflows/example.md

name: "example"
on:
  push:
    branches:
      - main

permissions: {}

concurrency:
  group: "example-${{ github.run_id }}"
  cancel-in-progress: true

env:
  GH_AW_PROMPT: /tmp/gh-aw/aw_prompt.txt

```

The header includes the front-matter hash for reproducibility, while the body uses deterministic field ordering guaranteed by `yaml.MarshalWithFieldOrder` in [`pkg/workflow/yaml.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/yaml.go).

## Summary

- **Four-phase pipeline**: `gh-aw` compiles markdown workflows into GitHub Actions YAML through Parse & Load, Validate, Generate YAML, and Final Validation phases.
- **Entry point**: The `CompileWorkflow` function in [`pkg/workflow/compiler.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/compiler.go) orchestrates the entire process, from reading the markdown file to writing the [`.lock.yml`](https://github.com/github/gh-aw/blob/main/.lock.yml) output.
- **Safety first**: The validation phase checks expression safety, permissions, template injection risks, and validates against the embedded [`schemas/github-workflow.json`](https://github.com/github/gh-aw/blob/main/schemas/github-workflow.json) schema.
- **Prompt chunking**: Large markdown bodies are automatically split into 20KB chunks (max 5) using `splitContentIntoChunks` to comply with GitHub Actions step size limits.
- **Deterministic output**: The `yaml.MarshalWithFieldOrder` function ensures consistent field ordering in the generated YAML, while the front-matter hash guarantees reproducibility.

## Frequently Asked Questions

### What is the maximum size for a compiled workflow file?

The `gh-aw` compiler warns when the generated [`.lock.yml`](https://github.com/github/gh-aw/blob/main/.lock.yml) file exceeds **500KB** (defined as `MaxLockFileSize` in [`pkg/workflow/compiler.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/compiler.go)). While GitHub Actions does not enforce a strict limit, files larger than this threshold may hit repository or performance constraints. If your workflow exceeds this limit, consider splitting it into multiple workflows or reducing embedded content.

### How does gh-aw handle large markdown prompts that exceed GitHub Actions step limits?

When the markdown body exceeds **20KB**, `gh-aw` automatically chunks the content using `splitContentIntoChunks` in [`pkg/workflow/compiler_yaml.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/compiler_yaml.go). The compiler splits the content into a maximum of **5 chunks**, each wrapped in a heredoc block within the generated YAML. This ensures compliance with GitHub Actions step size constraints while preserving the full prompt content for runtime AI processing.

### Can I skip schema validation when compiling a workflow?

Yes. The compiler supports skipping the final JSON schema validation step through the `--no-validation` CLI flag or programmatically by setting the appropriate compiler option. When enabled, the `validateGitHubActionsSchema` function in [`pkg/workflow/compiler_yaml.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/compiler_yaml.go) is bypassed, though expression size checks (21KB limit) and template injection prevention (`validateNoTemplateInjection`) still run for safety.

### What happens if validation fails during compilation?

If any validation check fails during the `generateAndValidateYAML` phase, `gh-aw` writes the invalid content to an `*.invalid.yml` file for debugging purposes instead of the standard [`.lock.yml`](https://github.com/github/gh-aw/blob/main/.lock.yml) file. The compiler returns a formatted error message using `formatCompilerError` in [`pkg/workflow/compiler.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/compiler.go), which provides detailed context about which validation rule failed (expression safety, permission checks, or schema validation) and the specific line or file causing the issue.