# How gh-aw Compiles Markdown Workflows to GitHub Actions YAML Internally

> Discover how gh-aw compiles markdown workflows to GitHub Actions YAML using a four-phase pipeline: parse, validate, generate, and finalize. Explore the internal implementation details.

- Repository: [GitHub/gh-aw](https://github.com/github/gh-aw)
- Tags: internals
- Published: 2026-02-16

---

**The `gh-aw` CLI compiles markdown workflows to GitHub Actions YAML through a four-phase pipeline—parse, validate, generate, and finalize—implemented across [`compiler.go`](https://github.com/github/gh-aw/blob/main/compiler.go) and [`compiler_yaml.go`](https://github.com/github/gh-aw/blob/main/compiler_yaml.go) in the `pkg/workflow` package.**

The `gh-aw` extension transforms human-readable Markdown workflow definitions into executable GitHub Actions YAML files. Understanding how it compiles markdown workflows to GitHub Actions YAML internally helps developers debug compilation errors and optimize their workflow definitions for the AI-driven execution model.

## The Four-Phase Compilation Pipeline

The compilation process in `gh-aw` follows a strict sequence implemented in [`pkg/workflow/compiler.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/compiler.go). Each phase handles a specific transformation from the source markdown to the final [`.lock.yml`](https://github.com/github/gh-aw/blob/main/.lock.yml) output.

### Phase 1: Parse and Load

The pipeline begins with `(*Compiler).CompileWorkflow`, which orchestrates the entire process. It first invokes `ParseWorkflowFile` to read the markdown from disk and extract the YAML front-matter.

During this phase, the compiler:

- Reads the source file from the path provided (e.g., [`.github/workflows/example.md`](https://github.com/github/gh-aw/blob/main/.github/workflows/example.md))
- Splits the front-matter block from the markdown body using the parser package
- Computes a front-matter hash via `parser.ComputeFrontmatterHashFromFile` for reproducibility
- Constructs a `WorkflowData` structure that holds all metadata needed for subsequent phases

This phase is located at [`compiler.go`](https://github.com/github/gh-aw/blob/main/compiler.go) lines 81-100.

### Phase 2: Validate

Before generating any YAML, the compiler runs `validateWorkflowData` to ensure the workflow is safe and compliant. This phase performs comprehensive checks defined in [`compiler.go`](https://github.com/github/gh-aw/blob/main/compiler.go) lines 14-63.

Validation includes:

- **Expression safety**: `validateExpressionSafety` ensures only whitelisted `${{ … }}` patterns are used to prevent injection attacks
- **Runtime imports**: `validateRuntimeImportFiles` guarantees that any imported markdown can be loaded at runtime
- **Permission and security checks**: Validates dangerous permissions, safe-output configurations, network firewall settings, and concurrency group expressions
- **Tool-set consistency**: Ensures all referenced tools and features are compatible

Errors and warnings are emitted through `formatCompilerError` or `formatCompilerMessage`, which use `console.FormatError` for uniform CLI styling.

### Phase 3: Generate YAML

The core transformation happens in `generateYAML` (located in [`compiler_yaml.go`](https://github.com/github/gh-aw/blob/main/compiler_yaml.go) lines 59-92). This phase constructs the actual GitHub Actions YAML structure through several sub-components:

**Job Construction**

The `buildJobs` function creates job objects, resolves inter-job dependencies, and validates against duplicate step IDs using the `JobManager` from [`pkg/workflow/job_manager.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/job_manager.go).

**Header Generation**

`generateWorkflowHeader` writes:

- A logo comment identifying the file as generated by `gh-aw`
- Source file attribution
- Description and import manifest
- The front-matter hash for traceability

**Body Generation**

`generateWorkflowBody` serializes top-level fields in deterministic order using `MarshalWithFieldOrder` from [`pkg/workflow/yaml.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/yaml.go):

- `name`
- `on` (triggers)
- `permissions`
- `concurrency`
- `run-name`
- `env`
- Rendered jobs from `JobManager`

**Prompt Chunking**

For AI-driven workflows, the markdown body is split into chunks of maximum 20 KB, with a hard limit of 5 chunks total, via `splitContentIntoChunks`. Each chunk becomes a heredoc block in a unified "prompt creation" step generated by `generateUnifiedPromptCreationStep`. Expressions inside the markdown are extracted using `NewExpressionExtractor` and replaced with environment variable placeholders, enabling runtime interpolation while keeping the AI prompt clean.

**Post-Steps**

Additional steps are appended for:

- Creating [`aw_info.json`](https://github.com/github/gh-aw/blob/main/aw_info.json) (`generateCreateAwInfo`)
- Workflow overview (`generateWorkflowOverviewStep`)
- Safe output collection (`generateOutputCollectionStep`)
- Artifact upload

### Phase 4: Finalize and Write Output

The final phase, implemented in `generateAndValidateYAML` ([`compiler.go`](https://github.com/github/gh-aw/blob/main/compiler.go) lines 67-88), ensures the generated YAML is safe and compliant before writing to disk.

Validation steps include:

- **Expression size limits**: `validateExpressionSizes` ensures no expression exceeds 21 KB
- **Template injection prevention**: `validateNoTemplateInjection` guards against unsafe template patterns
- **Schema validation**: `validateGitHubActionsSchema` validates against the embedded [`schemas/github-workflow.json`](https://github.com/github/gh-aw/blob/main/schemas/github-workflow.json) (unless `skipValidation` is true)
- **Container and runtime checks**: Validates container images, runtime packages, firewall rules, and repository features

If validation passes, `writeWorkflowOutput` ([`compiler.go`](https://github.com/github/gh-aw/blob/main/compiler.go) lines 51-78) writes the generated YAML to a [`.lock.yml`](https://github.com/github/gh-aw/blob/main/.lock.yml) file next to the source markdown. This function:

- Skips writing if the content is unchanged (preserving timestamps)
- Warns when the file exceeds the 500 KB recommended limit (`MaxLockFileSize`)
- Prints a success message with the relative path and file size

If validation fails, an `*.invalid.yml` file is written for debugging, and a formatted console error is returned.

## Key Implementation Files

The compilation pipeline spans several files in the `pkg/workflow` directory:

- **[`compiler.go`](https://github.com/github/gh-aw/blob/main/compiler.go)**: Core orchestration including `CompileWorkflow`, validation dispatcher, YAML generation, and file output handling
- **[`compiler_yaml.go`](https://github.com/github/gh-aw/blob/main/compiler_yaml.go)**: YAML generation logic including job building, header generation, prompt chunking, and post-steps
- **[`yaml.go`](https://github.com/github/gh-aw/blob/main/yaml.go)**: Deterministic YAML marshaling utilities (`MarshalWithFieldOrder`, `UnquoteYAMLKey`, `CleanYAMLNullValues`) using `goccy/go-yaml`
- **[`job_manager.go`](https://github.com/github/gh-aw/blob/main/job_manager.go)**: Job dependency resolution and step validation
- **[`expressions.go`](https://github.com/github/gh-aw/blob/main/expressions.go)**: Expression extraction and safety validation
- **[`parser/frontmatter.go`](https://github.com/github/gh-aw/blob/main/parser/frontmatter.go)**: Front-matter parsing and hash computation

The embedded JSON schema at [`schemas/github-workflow.json`](https://github.com/github/gh-aw/blob/main/schemas/github-workflow.json) provides the validation rules for the final output.

## How Prompt Generation and Chunking Works

A distinctive feature of `gh-aw` is its AI-driven execution model, which requires special handling of the markdown body:

1. **Content Splitting**: The `splitContentIntoChunks` function in [`compiler_yaml.go`](https://github.com/github/gh-aw/blob/main/compiler_yaml.go) divides the markdown body into chunks of maximum 20 KB, with a hard limit of 5 chunks total. This prevents exceeding GitHub Actions step size limits.

2. **Expression Extraction**: Before chunking, `NewExpressionExtractor` identifies `${{ … }}` expressions and replaces them with environment variable placeholders. This allows the AI engine to receive clean text while preserving the ability to interpolate values at runtime.

3. **Heredoc Generation**: Each chunk is wrapped in a shell heredoc within a unified prompt creation step generated by `generateUnifiedPromptCreationStep`. These steps write the chunks to temporary files (e.g., [`/tmp/gh-aw/prompt-0.txt`](https://github.com/github/gh-aw/blob/main//tmp/gh-aw/prompt-0.txt)) that the AI runtime can access.

This approach ensures that large markdown workflows can be processed by GitHub Actions without hitting the 48 KB step limit or 21 KB expression limit, while maintaining the dynamic capabilities of the workflow expression syntax.

## Example: Compiling a Workflow

### CLI Compilation

To compile a markdown workflow from the command line:

```bash
$ gh aw compile .github/workflows/example.md
⚙️  Parsing workflow file
✅  Successfully generated workflow (example.md) (12 KB)

```

This command creates a `Compiler` instance, invokes `CompileWorkflow`, and writes the output to [`.github/workflows/example.lock.yml`](https://github.com/github/gh-aw/blob/main/.github/workflows/example.lock.yml).

### Programmatic Compilation

You can also compile workflows programmatically using the Go API:

```go
package main

import (
    "log"

    "github.com/github/gh-aw/pkg/workflow"
)

func main() {
    comp := workflow.NewCompiler()
    // optional: comp.SetStrictMode(true) // treat missing permissions as errors
    if err := comp.CompileWorkflow(".github/workflows/example.md"); err != nil {
        log.Fatalf("Compilation failed: %v", err)
    }
}

```

The `CompileWorkflow` method triggers the full four-phase pipeline described above.

### Inspecting the Generated Output

The generated [`.lock.yml`](https://github.com/github/gh-aw/blob/main/.lock.yml) file includes metadata comments and deterministic field ordering:

```bash
$ cat .github/workflows/example.lock.yml | head -n 20

# Generated by gh-aw (github.com/github/gh-aw)

# Source: .github/workflows/example.md

name: "example"
on:
  push:
    branches:
      - main

permissions: {}

concurrency:
  group: "example-${{ github.run_id }}"
  cancel-in-progress: true

env:
  GH_AW_PROMPT: /tmp/gh-aw/aw_prompt.txt

```

The top-level field ordering (`name`, `on`, `permissions`, …) is guaranteed by `MarshalWithFieldOrder` in [`pkg/workflow/yaml.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/yaml.go).

### Chunked Prompt Example

When the markdown body exceeds 20 KB, the compiler generates multiple heredoc steps:

```yaml
- name: Build prompt – chunk 1
  run: |
    cat <<'EOF' > /tmp/gh-aw/prompt-0.txt
    ...first 20 KB of cleaned markdown...
    EOF

- name: Build prompt – chunk 2
  run: |
    cat <<'EOF' > /tmp/gh-aw/prompt-1.txt
    ...next 20 KB...
    EOF

```

These steps are generated by `splitContentIntoChunks` (see [`compiler_yaml.go`](https://github.com/github/gh-aw/blob/main/compiler_yaml.go) lines 89-115).

## Summary

- **Four-phase pipeline**: `gh-aw` compiles markdown workflows to GitHub Actions YAML through parse, validate, generate, and finalize phases implemented in [`pkg/workflow/compiler.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/compiler.go) and [`pkg/workflow/compiler_yaml.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/compiler_yaml.go).
- **Safety-first validation**: The compiler checks expression safety, permissions, template injection, and schema compliance against [`schemas/github-workflow.json`](https://github.com/github/gh-aw/blob/main/schemas/github-workflow.json) before writing output.
- **AI-ready chunking**: Large markdown bodies are split into 20 KB heredoc chunks with expression extraction to bypass GitHub Actions step limits while preserving runtime interpolation.
- **Deterministic output**: The `MarshalWithFieldOrder` utility ensures consistent YAML field ordering, and front-matter hashes provide reproducibility.
- **Lock file generation**: Valid workflows are written to [`.lock.yml`](https://github.com/github/gh-aw/blob/main/.lock.yml) files next to their source markdown, with automatic change detection and size warnings for files exceeding 500 KB.

## Frequently Asked Questions

### What is the entry point for compiling a markdown workflow in gh-aw?

The entry point is the `CompileWorkflow` method in [`pkg/workflow/compiler.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/compiler.go). When you run `gh aw compile <path>.md`, the CLI creates a `Compiler` instance and calls this method, which orchestrates the four-phase pipeline: parsing the markdown, validating the workflow data, generating the YAML structure, and writing the final lock file.

### How does gh-aw handle large markdown files that exceed GitHub Actions step size limits?

The compiler uses the `splitContentIntoChunks` function in [`pkg/workflow/compiler_yaml.go`](https://github.com/github/gh-aw/blob/main/pkg/workflow/compiler_yaml.go) to divide the markdown body into chunks of maximum 20 KB, with a hard limit of 5 chunks total. Each chunk becomes a heredoc block in a unified prompt creation step generated by `generateUnifiedPromptCreationStep`. Before chunking, `NewExpressionExtractor` replaces `${{ … }}` expressions with environment variable placeholders to preserve runtime interpolation capabilities.

### What validation checks does gh-aw perform before generating the final YAML?

Before writing the [`.lock.yml`](https://github.com/github/gh-aw/blob/main/.lock.yml) file, `generateAndValidateYAML` in [`compiler.go`](https://github.com/github/gh-aw/blob/main/compiler.go) executes several safety checks: `validateExpressionSizes` ensures no expression exceeds 21 KB; `validateNoTemplateInjection` guards against unsafe template patterns; and `validateGitHubActionsSchema` validates the output against the embedded [`schemas/github-workflow.json`](https://github.com/github/gh-aw/blob/main/schemas/github-workflow.json). The compiler also checks container images, runtime packages, firewall rules, and repository features during the validation phase.

### Why does gh-aw generate a [`.lock.yml`](https://github.com/github/gh-aw/blob/main/.lock.yml) file instead of overwriting the original markdown?

The [`.lock.yml`](https://github.com/github/gh-aw/blob/main/.lock.yml) file serves as the compiled, machine-readable artifact that GitHub Actions executes, while preserving the human-readable `.md` source for editing. This separation allows `gh-aw` to perform change detection (skipping writes when content is unchanged to preserve timestamps), embed metadata like front-matter hashes for reproducibility, and warn when the compiled output exceeds the 500 KB recommended limit (`MaxLockFileSize`). If validation fails, the compiler writes an `*.invalid.yml` file for debugging instead.