# Supported Data Sources for Ponytail: Project vs. Local Configuration

> Discover Ponytail's supported data sources: project (repository files, SKILL.md) and local (user plugins, global extensions). Control them with the --setting-sources flag.

- Repository: [DietrichGebert/ponytail](https://github.com/DietrichGebert/ponytail)
- Tags: supported-data-sources
- Published: 2026-09-10

---

**Ponytail supports two distinct data sources—the project source (your repository files and SKILL.md definitions) and the local source (user-wide plugins and global extensions)—which you control via the `--setting-sources` flag.**

Ponytail is an agentic coding assistant that generates suggestions by ingesting code from specific origins defined in its runtime configuration. Understanding the supported data sources for Ponytail is essential for controlling whether the agent accesses only your repository files or incorporates globally installed extensions. The system distinguishes between **project** sources, which serve as the canonical source of truth for your codebase, and **local** sources, which reference external plugins installed outside the current repository.

## The Two Supported Data Sources for Ponytail

Ponytail's ingestion pipeline recognizes exactly two data origins. These are not file formats but scope definitions that determine which directories the agent can access when generating responses.

### Project Source

The **project** source comprises all files within the repository where Ponytail is installed. This includes source code files, configuration assets, and critically, the skill definitions stored in [`SKILL.md`](https://github.com/DietrichGebert/ponytail/blob/main/SKILL.md) files. According to the source code analysis, this is the canonical source of truth for Ponytail's operation. For example, the core ponytail skill itself resides in [`skills/ponytail/SKILL.md`](https://github.com/DietrichGebert/ponytail/blob/main/skills/ponytail/SKILL.md), demonstrating how the project source houses the structured definitions that drive agent behavior.

### Local Source

The **local** source encompasses user-wide plugins and locally installed extensions that exist outside the current project directory. This includes globally available tools such as Hermes or Copilot plugins. When enabled, this source allows Ponytail to incorporate functionality from your global development environment while maintaining isolation from the project's own version-controlled code.

## Configuring Data Sources via CLI

You specify which data sources to enable using the `--setting-sources` flag in the benchmark harness or plugin runtime. This flag accepts a comma-separated list of source names.

```bash

# Run a benchmark using both sources (default behavior)

npx promptfoo@latest eval -c benchmarks/promptfooconfig.yaml \
  --setting-sources project,local

# Explicitly restrict to project only (exclude global plugins)

npx promptfoo@latest eval -c benchmarks/promptfooconfig.yaml \
  --setting-sources project

```

When you omit the flag, the system typically defaults to using both sources. Restricting to `project` only is useful for deterministic testing or when you want to ensure suggestions derive solely from your repository's committed code.

You can also query the active configuration programmatically:

```js
// In a custom script you can query the active sources
const { getActiveSources } = require('ponytail');

console.log(getActiveSources()); // → ['project', 'local']

```

## Implementation in Source Code

The dual-source architecture is documented across several key files in the `DietrichGebert/ponytail` repository:

- **[`benchmarks/results/2026-06-18-agentic.md`](https://github.com/DietrichGebert/ponytail/blob/main/benchmarks/results/2026-06-18-agentic.md)**: Contains the benchmark configuration demonstrating CLI isolation using `--setting-sources project,local` to define test arms.
- **[`skills/ponytail/SKILL.md`](https://github.com/DietrichGebert/ponytail/blob/main/skills/ponytail/SKILL.md)**: Exemplifies the project source by housing the full skill body and structured definitions that the agent reads from the repository.
- **[`plugin.yaml`](https://github.com/DietrichGebert/ponytail/blob/main/plugin.yaml)**: The plugin manifest that drives the Ponytail runtime and references the supported data source architecture.
- **[`benchmarks/promptfooconfig.yaml`](https://github.com/DietrichGebert/ponytail/blob/main/benchmarks/promptfooconfig.yaml)**: The configuration file used by the evaluation harness that consumes the `--setting-sources` parameter.

## Summary

- Ponytail recognizes exactly two supported data sources: **project** (repository-scoped files) and **local** (user-wide global plugins).
- The **project** source is the canonical source of truth, containing all [`SKILL.md`](https://github.com/DietrichGebert/ponytail/blob/main/SKILL.md) definitions and version-controlled code.
- The **local** source provides access to global extensions like Hermes or Copilot plugins installed outside the repository.
- Control source availability using the `--setting-sources` flag with comma-separated values (`project,local`).
- Configuration examples exist in [`benchmarks/results/2026-06-18-agentic.md`](https://github.com/DietrichGebert/ponytail/blob/main/benchmarks/results/2026-06-18-agentic.md) and the promptfoo evaluation config.

## Frequently Asked Questions

### What is the difference between project and local sources in Ponytail?

The **project** source includes only files within the current repository, such as source code and [`SKILL.md`](https://github.com/DietrichGebert/ponytail/blob/main/SKILL.md) skill definitions, making it the canonical reference for your codebase. The **local** source includes plugins and extensions installed globally on your machine, such as Hermes or Copilot plugins, which reside outside version control.

### How do I exclude local plugins from Ponytail's context?

Pass `--setting-sources project` to the CLI without including `local`. For example: `npx promptfoo@latest eval -c benchmarks/promptfooconfig.yaml --setting-sources project`. This restricts the agent to repository files only.

### Where are skill definitions stored in the project source?

Skill definitions reside in [`SKILL.md`](https://github.com/DietrichGebert/ponytail/blob/main/SKILL.md) files distributed throughout the repository. The canonical example is [`skills/ponytail/SKILL.md`](https://github.com/DietrichGebert/ponytail/blob/main/skills/ponytail/SKILL.md), which contains the structured skill body that Ponytail ingests when processing the project source.

### Can I use custom data sources beyond project and local?

No. According to the source code and [`plugin.yaml`](https://github.com/DietrichGebert/ponytail/blob/main/plugin.yaml) manifest, Ponytail's runtime only recognizes the **project** and **local** sources. There is no mechanism for defining additional custom data origins in the current implementation.