How to Debug YAML Parsing Issues with Token Tracing in yaml-cpp

Enable token tracing in yaml-cpp by calling parser.EnableDebug(true) before parsing, or subclass YAML::Scanner to emit custom debug output that shows every token's type and position.

yaml-cpp processes YAML documents through a tightly-coupled three-stage pipeline that converts raw text into structured Node objects. When parsing failures occur, the token stream provides the most diagnostic value, revealing exactly where the lexer and parser diverge from expectations. The library provides public API hooks for debugging YAML parsing issues with token tracing, allowing you to inspect the flow from lexical analysis through node construction.

Understanding the yaml-cpp Parsing Architecture

The library implements a classic compiler pipeline with three distinct phases. Understanding these stages helps you choose the appropriate debugging strategy.

Stage 1: Lexical Analysis in src/scanner.cpp

The Scanner class in src/scanner.cpp reads the raw input stream and breaks it into a sequence of tokens such as Scalar, Tag, and BlockStart. Each token carries a Mark object indicating its exact line and column position in the source text. This stage determines how characters group into semantic units.

Stage 2: Syntax Analysis in src/parser.cpp

The Parser class in src/parser.cpp consumes the token stream and builds an abstract syntax tree (AST) according to the YAML 1.2 grammar. It drives a NodeBuilder that creates the high-level Node objects exposed to library users. This is where grammar violations trigger parse errors.

Stage 3: Node Construction in src/nodebuilder.cpp

NodeBuilder in src/nodebuilder.cpp assembles the final Node hierarchy, handling anchors, aliases, and tag resolution. Errors here typically involve unresolved references or type mismatches rather than syntax issues.

Method 1: Enable Built-in Token Tracing

For quick debugging of parsing failures, yaml-cpp provides the EnableDebug() method in src/parser.cpp. This installs a debug listener that prints every token as it is consumed, together with its Mark coordinates.

#include <yaml-cpp/yaml.h>
#include <iostream>

int main() {
    const char* yaml = R"(
        name: Alice
        age: 30
        skills:
          - C++
          - Python
    )";

    YAML::Parser parser;
    parser.SetInput(yaml);
    parser.EnableDebug(true);            // <- enable token tracing

    YAML::Node doc;
    parser >> doc;                        // parsing occurs here

    std::cout << "Root node type: " << doc.Type() << '\n';
    return 0;
}

When executed, this produces output showing the token flow:

[DEBUG] Token: BlockMappingStart @ line 3, col 1
[DEBUG] Token: Key                 @ line 3, col 2
[DEBUG] Token: Scalar               @ line 3, col 4  ("name")
[DEBUG] Token: Value               @ line 3, col 8
...

The EnableDebug(true) parameter activates the trace without requiring modifications to the scanner source code.

Method 2: Custom Scanner Subclass for Lexical Debugging

When you need to inspect how raw characters become tokens—for example, investigating indentation edge cases or custom tag handling—you can subclass YAML::Scanner and override the token emission method.

#include <yaml-cpp/yaml.h>
#include <sstream>
#include <iostream>

class DebugScanner : public YAML::Scanner {
public:
    explicit DebugScanner(std::istream& in) : YAML::Scanner(in) {}
protected:
    void EmitToken(const YAML::Token& tok) override {
        std::cerr << "[SCANNER] " << tok.Type()
                  << " @ " << tok.StartMark().line << ':'
                  << tok.StartMark().column << '\n';
        YAML::Scanner::EmitToken(tok);   // forward to original logic
    }
};

int main() {
    const char* yaml = "key: { nested: [1,2,3] }";
    std::istringstream iss(yaml);
    DebugScanner scanner(iss);
    YAML::Parser parser(scanner);
    parser.EnableDebug(true);

    YAML::Node root;
    parser >> root;
    return 0;
}

This approach exposes the Token type definitions from src/token.h and their associated Mark information, giving you granular visibility into the lexical analysis phase.

Choosing the Right Debugging Strategy

Select your tracing approach based on where you suspect the failure occurs:

• Quick full-pipeline inspection: Use parser.EnableDebug(true) when you need immediate visibility into the token stream without code changes.
• Lexical analysis debugging: Subclass YAML::Scanner when investigating scanner behavior, character-by-character tokenization, or Mark calculation issues.
• Test framework integration: When using GoogleTest, wrap parsing calls with SCOPED_TRACE("Parsing ...") as demonstrated in test/integration/clone_node_test.cpp to correlate test failures with specific parsing stages.

Summary

• yaml-cpp parses documents through three stages: Scanner (lexical), Parser (syntactic), and NodeBuilder (construction).
• Enable token tracing by calling parser.EnableDebug(true) before parsing to see token types and source positions.
• For low-level debugging, subclass YAML::Scanner and override EmitToken() to intercept tokens during lexical analysis.
• Token objects carry Mark data indicating exact line and column positions in the source file.
• Reference the test suite in test/integration/clone_node_test.cpp for examples of integrating parsing diagnostics with testing frameworks.

Frequently Asked Questions

How do I enable token tracing without modifying the yaml-cpp source?

Call parser.EnableDebug(true) after creating your YAML::Parser object but before extracting nodes. This method, implemented in src/parser.cpp, activates a debug listener that prints each token's type and location to standard error without requiring you to rebuild the library.

What information does each token contain during tracing?

Each token carries a Type() identifying its grammatical role (e.g., Scalar, BlockMappingStart, Key) and a StartMark() object containing the exact line and column numbers where the token begins. These definitions reside in src/token.h.

Why would I need to subclass Scanner instead of using EnableDebug?

Subclass YAML::Scanner when you need to debug the lexical analysis itself—such as investigating how specific character sequences become tokens, verifying indentation calculations, or tracing token emission order. The EnableDebug() method only shows tokens after they reach the parser, while a custom scanner can intercept tokens at the moment of creation in src/scanner.cpp.

Where does yaml-cpp handle anchor and alias resolution during parsing?

Anchor and alias resolution occurs in src/nodebuilder.cpp, where the NodeBuilder class constructs the final Node hierarchy. While token tracing shows when Anchor and Alias tokens appear in the stream, their semantic resolution happens during the node construction phase after parsing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}