# Architectural Patterns in dotnet/skills: A Deep Dive into the Plug-In-Centric Design

> Explore architectural patterns in dotnet/skills including plug-in discovery, strategy judges, and source-generated serialization for scalable skill validation.

- Repository: [.NET Platform/skills](https://github.com/dotnet/skills)
- Tags: architecture
- Published: 2026-07-08

---

**The dotnet/skills repository implements a modular, command-driven architecture using plug-in discovery, strategy-based judges, and source-generated serialization to enable scalable skill validation.**

The dotnet/skills codebase serves as a validation and evaluation engine for AI skills, employing sophisticated architectural patterns to maintain modularity and performance. This system separates concerns through a combination of discovery patterns, command encapsulation, and runtime strategy selection. Understanding these architectural patterns reveals how the repository manages hundreds of independent skills while maintaining clean dependencies and high throughput.

## Plug-In Architecture and Service Discovery

The foundation of the system rests on a **plug-in-centric architecture** that treats skills and agents as independent modules. Each plug-in contains a [`plugin.json`](https://github.com/dotnet/skills/blob/main/plugin.json) manifest describing its name, version, and paths to skill and agent files.

### Manifest-Based Plugin Loading

The runtime discovers plug-ins by walking the directory tree and parsing these manifests. In [`eng/skill-validator/src/Shared/PluginDiscovery.cs`](https://github.com/dotnet/skills/blob/main/eng/skill-validator/src/Shared/PluginDiscovery.cs), the discovery logic locates the plug-in root from any file path within the skill hierarchy:

```csharp
// Given a file path inside a skill, locate the plug‑in root and parse its manifest.
var skillInfo = new SkillInfo { Path = "/path/to/skills/dotnet-test/skills/find-untested-sources/Skill.cs" };

var ctx = PluginDiscovery.FindPluginContext(skillInfo);
if (ctx is null)
{
    Console.WriteLine("No plug‑in found.");
    return;
}

var (pluginRoot, pluginInfo) = ctx;
Console.WriteLine($"Plugin: {pluginInfo.Name} (v{pluginInfo.Version})");
Console.WriteLine($"Root folder: {pluginRoot}");
Console.WriteLine("Skills declared in manifest:");
foreach (var skillPath in pluginInfo.SkillPaths)
    Console.WriteLine($"  - {skillPath}");

```

### Reflection-Based Skill Discovery

Static helper classes decouple the core validator from concrete implementations. The `SkillDiscovery` and `AgentDiscovery` classes in [`eng/skill-validator/src/Shared/SkillDiscovery.cs`](https://github.com/dotnet/skills/blob/main/eng/skill-validator/src/Shared/SkillDiscovery.cs) use reflection to locate all skills belonging to a discovered plug-in, enforcing the **Service Locator pattern** without hard-coded dependencies.

## Command Pattern for CLI Workflows

The CLI layer acts as a thin wrapper that delegates work to discrete command classes, implementing the **Command pattern** to encapsulate distinct workflows. Each command class handles a specific operation:

- **`CheckCommand`** ([`eng/skill-validator/src/Check/CheckCommand.cs`](https://github.com/dotnet/skills/blob/main/eng/skill-validator/src/Check/CheckCommand.cs)) – Handles validation logic
- **`EvaluateCommand`** ([`eng/skill-validator/src/Evaluate/EvaluateCommand.cs`](https://github.com/dotnet/skills/blob/main/eng/skill-validator/src/Evaluate/EvaluateCommand.cs)) – Manages evaluation runs
- **`RejudgeCommand`** and **`ConsolidateCommand`** – Handle re-running and result aggregation

This structure allows the CLI to invoke complex operations through a uniform interface:

```csharp
// The CLI invokes the command class directly.
var evalFile = "my-eval.yaml";
var options = new EvaluateOptions
{
    EvalFile = evalFile,
    MaxConcurrency = 4,
    EnableOverfittingCheck = true,
};

await EvaluateCommand.RunAsync(options);

```

## Strategy Pattern for Evaluation Logic

The evaluation system employs the **Strategy pattern** to swap algorithms at runtime. Different judges implement a common `IJudge` interface defined in [`eng/skill-validator/src/Evaluate/Judge.cs`](https://github.com/dotnet/skills/blob/main/eng/skill-validator/src/Evaluate/Judge.cs), allowing the system to select evaluation strategies based on configuration options:

- **`OverfittingJudge`** – Detects overfitting in skill performance
- **`PairwiseJudge`** – Compares outputs pairwise
- **`MetricsCollector`** – Gathers performance metrics

Runtime selection occurs without modifying the evaluation engine core:

```csharp
IJudge judge = options.UsePairwise
    ? new PairwiseJudge()
    : (IJudge)new OverfittingJudge();

var result = await judge.RunAsync(evaluationContext);

```

## Builder, Factory, and Repository Patterns

Data management combines multiple patterns to ensure configuration flexibility and persistent state handling.

### YAML Configuration Building

Evaluation configurations use a **Builder pattern** implementation in [`eng/skill-validator/src/Evaluate/EvalSchema.cs`](https://github.com/dotnet/skills/blob/main/eng/skill-validator/src/Evaluate/EvalSchema.cs). The parsing logic creates strongly-typed models through `EvalSchema.Parse`, acting as a factory that transforms YAML files into executable evaluation plans.

### File-Based Session Persistence

The `SessionDatabase` class in [`eng/skill-validator/src/Evaluate/SessionDatabase.cs`](https://github.com/dotnet/skills/blob/main/eng/skill-validator/src/Evaluate/SessionDatabase.cs) implements the **Repository and Unit-of-Work patterns** for lightweight file-based storage. It manages metrics and session files while implementing `IDisposable` to ensure resources release when validation runs complete.

## Concurrency Control and Resource Management

Parallel evaluation of multiple skills could exhaust host resources without proper throttling. The **Concurrency-Limiter pattern** wraps `SemaphoreSlim` in [`eng/skill-validator/src/Shared/ConcurrencyLimiter.cs`](https://github.com/dotnet/skills/blob/main/eng/skill-validator/src/Shared/ConcurrencyLimiter.cs) to control parallelism:

```csharp
using var limiter = new ConcurrencyLimiter(maxConcurrency: 8);
await Parallel.ForEachAsync(skills, async (skill, ct) =>
{
    await limiter.EnterAsync(ct);
    try
    {
        await EvaluateSkillAsync(skill);
    }
    finally
    {
        limiter.Release();
    }
});

```

This protects the system while maximizing throughput across CPU-bound validation tasks.

## Advanced Patterns for Maintainability

Several additional patterns support code organization and performance optimization.

### Source-Generated JSON Serialization

To avoid runtime reflection overhead, the repository uses `System.Text.Json` source generators. The `SkillValidatorJsonContext` class in [`eng/skill-validator/src/SkillValidatorJsonContext.cs`](https://github.com/dotnet/skills/blob/main/eng/skill-validator/src/SkillValidatorJsonContext.cs) provides compile-time-checked, high-performance JSON serialization for [`plugin.json`](https://github.com/dotnet/skills/blob/main/plugin.json) files and result outputs.

### Partial Classes for Separation of Concerns

Core services utilize the **Partial-Class pattern** to separate generated code from hand-written logic. For example, `SkillValidatorYamlContext` and `OverfittingJudge` split their implementations across multiple files, keeping source-generated serializers isolated from business logic.

### Adapter and Decorator Patterns

- **Adapter pattern**: The [`run-vally-evals.sh`](https://github.com/dotnet/skills/blob/main/run-vally-evals.sh) script and `adapt.mjs` module adapt external benchmark outputs into the internal result model, bridging third-party tools with the validation engine.
- **Decorator-like helpers**: Utility types such as `Spinner`, `Ansi`, and `RetryHelper` in [`eng/skill-validator/src/Shared/Spinner.cs`](https://github.com/dotnet/skills/blob/main/eng/skill-validator/src/Shared/Spinner.cs) wrap cross-cutting concerns including progress UI, console coloring, and retry logic without polluting core business code.

## Summary

The dotnet/skills repository demonstrates a mature application of enterprise architectural patterns:

- **Plug-in Architecture** enables independent skill development and discovery via manifest files
- **Command Pattern** encapsulates CLI workflows in discrete, testable classes
- **Strategy Pattern** allows runtime selection of evaluation algorithms through the `IJudge` interface
- **Repository Pattern** provides lightweight persistence through `SessionDatabase`
- **Concurrency Limiter** prevents resource exhaustion during parallel evaluation
- **Source Generation** eliminates runtime reflection costs for JSON serialization

These patterns collectively create a modular, extensible validation engine capable of scaling to hundreds of independent skills while maintaining clean architectural boundaries.

## Frequently Asked Questions

### What is the primary architectural style of dotnet/skills?

The repository follows a **plug-in-centric, command-driven** architectural style. Skills are self-contained plug-ins discovered at runtime through manifest files, while the CLI delegates operations to command objects that encapsulate specific workflows like validation and evaluation.

### How does the repository handle parallel skill evaluation safely?

The codebase implements a **Concurrency-Limiter pattern** using `SemaphoreSlim` wrapped in the `ConcurrencyLimiter` class. This throttles parallel execution to a configurable maximum, preventing thread pool exhaustion while still allowing concurrent evaluation of independent skills.

### Why does the codebase use source-generated JSON serialization instead of reflection?

The `SkillValidatorJsonContext` class provides **source-generated JSON serialization** using `System.Text.Json` generators. This approach offers compile-time validation of serialization contracts and eliminates runtime reflection overhead, significantly improving cold-start performance for the validation engine.

### How are external benchmark tools integrated into the validation workflow?

The system uses the **Adapter pattern** through the [`run-vally-evals.sh`](https://github.com/dotnet/skills/blob/main/run-vally-evals.sh) shell script and `adapt.mjs` module. These components translate external benchmark outputs into the internal result model, allowing the core validation engine to consume third-party evaluation data without direct coupling to external tool implementations.