How to Debug Performance Issues in Quarkdown's Pipeline Stages

Use PipelineHooks callbacks combined with MillisStopwatch to measure elapsed time after each pipeline stage, enabling debug logging via -Dloglevel=debug to output granular timings without impacting production performance.

Quarkdown processes documents through a sequential pipeline where each stage receives the previous output and transforms it for the next. If you need to debug performance issues in Quarkdown's pipeline stages, the architecture provides built-in extension points in PipelineHooks.kt and an immutable MillisStopwatch timer for precise instrumentation as implemented in iamgio/quarkdown.

Understand the Pipeline Architecture

The pipeline is orchestrated by Pipeline.kt, which coordinates a fixed chain of stages created by PipelineChainFactory.fullChain. Each stage implements the PipelineStage<I,O> interface defined in PipelineStage.kt, processing input from the previous stage and passing output to the next. The sequence follows this order: libraries registration → lexing → parsing → function expansion → tree traversal → rendering → post-rendering → resource generation.

Key components for performance debugging include:

• PipelineHooks.kt – Provides optional callbacks invoked after each stage, serving as the primary extensibility point for timing instrumentation.
• MillisStopwatch.kt – An immutable timer used by the CLI to measure compilation latency with minimal overhead.
• Log.kt – A Log4j-based façade where Log.debug evaluates its lambda only when debug mode is active, ensuring zero-cost in production.

Enable Debug Logging

Before adding instrumentation, activate the debug logger to capture timing output. The Log implementation checks logger.level == Level.DEBUG (Log.kt lines 27-29), evaluating debug lambdas only when the level matches.

Set the system property before running the CLI:

System.setProperty("loglevel", "debug")

Or pass it via JVM arguments:

java -Dloglevel=debug -jar quarkdown-cli.jar

Implement Timing Hooks with PipelineHooks

The cleanest method to profile each stage is creating a custom PipelineHooks instance that records timestamps after every transformation. The CLI already demonstrates this pattern in CompileCommand.kt (lines 89-106), using a MillisStopwatch to track total compilation time. Extend this approach to capture per-stage latency.

import com.quarkdown.core.pipeline.*
import com.quarkdown.core.pipeline.stages.*
import com.quarkdown.core.log.Log
import com.quarkdown.cli.util.MillisStopwatch

val stopwatch = MillisStopwatch()

val timingHooks = PipelineHooks(
    afterRegisteringLibraries = { libs ->
        Log.debug { "Libraries registered (${libs.size}) – ${stopwatch.elapsedMillis()} ms" }
    },
    afterLexing = { tokens ->
        Log.debug { "Lexing finished – ${stopwatch.elapsedMillis()} ms (tokens=${tokens.count()})" }
    },
    afterParsing = { ast ->
        Log.debug { "Parsing finished – ${stopwatch.elapsedMillis()} ms (nodes=${ast.children.size})" }
    },
    afterExpanding = { ast ->
        Log.debug { "Function call expansion – ${stopwatch.elapsedMillis()} ms" }
    },
    afterTreeTraversal = { ast ->
        Log.debug { "Tree traversal – ${stopwatch.elapsedMillis()} ms" }
    },
    afterRendering = { output ->
        Log.debug { "Rendering – ${stopwatch.elapsedMillis()} ms (size=${output.length})" }
    },
    afterPostRendering = { output ->
        Log.debug { "Post-rendering – ${stopwatch.elapsedMillis()} ms" }
    },
    afterAllRendering = { output ->
        Log.debug { "All rendering – ${stopwatch.elapsedMillis()} ms" }
    },
    afterGeneratingResources = { resources ->
        Log.debug { "Resource generation – ${stopwatch.elapsedMillis()} ms (resources=${resources.size})" }
    }
)

Inject the hooks when constructing the Pipeline:

val pipeline = Pipeline(
    context = mutableContext,
    options = PipelineOptions(),
    libraries = myLibraries,
    renderer = ::myRenderer,
    hooks = timingHooks
)
pipeline.execute(source)

Because Log.debug receives a lambda that executes only when debug mode is active, this instrumentation adds virtually no overhead in standard runs.

Instrument Individual Stages for Deep Profiling

When a specific stage requires closer inspection—such as FunctionCallExpansionStage.kt—instrument the peek method directly. The peek method is called before the stage's hook, allowing isolated measurement independent of callback execution.

object TimedFunctionCallExpansionStage : PeekPipelineStage<AstRoot> {
    override val hook = PipelineHooks::afterExpanding

    override fun peek(input: AstRoot, data: SharedPipelineData) {
        val sw = MillisStopwatch()
        FunctionCallNodeExpander(
            data.context,
            errorHandler = data.pipeline.options.errorHandler,
        ).expandAll()
        Log.debug { "FunctionCallExpansionStage took ${sw.elapsedMillis()} ms" }
    }
}

Interpreting Performance Results

After running with instrumentation, analyze the logged durations to identify bottlenecks:

• Lexing – Typically fast; high times indicate a massive token stream or complex custom lexer extensions.
• Parsing – Cost grows with document size and complexity of block/inline structures.
• Function call expansion – Often the primary bottleneck when documents use many custom functions, especially with heavy I/O or nested expansions.
• Rendering – May dominate for large HTML/PDF outputs or when third-party post-processors (e.g., Mermaid, MathJax) are invoked.

Compare timings against a baseline document. Large deviations in specific stages indicate where to apply optimizations, such as caching pure function results or simplifying regex patterns in lexer definitions.

Summary

• Activate debug logging with -Dloglevel=debug to capture timing output via Log.debug.
• Create a PipelineHooks instance that records MillisStopwatch snapshots after each stage to profile the sequential pipeline.
• Pass timing hooks to the Pipeline constructor and execute your document to generate per-stage latency reports.
• Instrument individual stages like FunctionCallExpansionStage by adding stopwatches inside their peek methods for granular analysis.
• Analyze logged durations against baselines to pinpoint whether lexing, parsing, function expansion, or rendering causes slowdowns, then target optimizations accordingly.

Frequently Asked Questions

How do I enable debug logging without modifying the source code?

Pass -Dloglevel=debug as a JVM argument when running the Quarkdown CLI. The Log.kt implementation checks this property to activate DEBUG level output, allowing you to see Log.debug messages containing your timing data without changing any Kotlin source files.

Will adding timing hooks slow down my production builds?

No. The Log.debug method accepts a lambda that only evaluates when the logger level is DEBUG (Log.kt line 27-29). In production runs where the level is INFO or higher, the lambda body (including the elapsedMillis() call) never executes, resulting in zero overhead from your instrumentation hooks.

Which pipeline stage should I investigate first if compilation is slow?

Start with function call expansion, implemented in FunctionCallExpansionStage.kt. This stage frequently becomes the bottleneck when documents contain many custom function calls or complex nested expansions. If that stage shows normal duration, investigate the rendering stage next, particularly if using external post-processors like Mermaid or MathJax.

Can I profile a single stage without affecting the rest of the pipeline?

Yes. Create a custom stage implementation that wraps the original behavior with a local MillisStopwatch. The peek method in PipelineStage.kt executes before the hook callback, allowing you to time specific stages independently. Alternatively, provide a PipelineHooks instance that only implements the callback for the specific stage you want to measure (e.g., afterExpanding), leaving other callbacks as no-ops.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}