When to Use Reference Files in Claude Skills: A Complete Guide to Managing Large Files

Reference files in Claude Skills should be used when documentation exceeds approximately 10,000 words or contains static data that would clutter the main SKILL.md, while large files should be managed through file splitting, grep patterns, and streaming libraries to maintain token efficiency.

The ComposioHQ/awesome-claude-skills repository establishes clear patterns for organizing complex skill documentation. When building sophisticated Claude Skills that require extensive API specifications, schema definitions, or policy manuals, understanding when to offload content to the references/ directory becomes critical for maintaining performance and usability.

When to Use Reference Files in Claude Skills

According to the skill-creator/SKILL.md file in the ComposioHQ/awesome-claude-skills repository, reference files serve as the preferred storage for any large or detailed documentation that Claude should be able to look up without embedding directly in the main skill definition.

Documentation Exceeding Context Limits

Add a reference file when content surpasses approximately 10,000 words, which would overwhelm the context window. This includes full API specifications, complete schema definitions, and comprehensive policy manuals. The official guidance in skill-creator/SKILL.md states: "If files are large (>10k words), include grep search patterns in SKILL.md" to enable targeted retrieval without consuming unnecessary tokens.

Static Reference Data

Use references/ for static data consulted repeatedly across multiple skill executions. When information remains constant and functions as a lookup resource rather than executable instructions, separating it into dedicated files prevents redundant token consumption during every skill invocation.

Maintaining a Lean SKILL.md

Reference files allow you to keep SKILL.md lean, ensuring Claude can load the skill quickly and pull in heavy material only on demand. This progressive disclosure pattern guarantees optimal response times for routine queries while reserving detailed documentation for specific user requests. The scripts/init_skill.py utility automatically creates the references/ directory when scaffolding new skills, establishing this structure from project inception.

How Claude Loads Reference Files

Claude implements progressive disclosure when processing skills. The system first reads SKILL.md, detects lines such as Load references/finance.md, and only then pulls the file into context. This mechanism ensures that only the needed portion of the repository occupies the LLM's token budget, as documented in the Progressive Disclosure Design Principle section of skill-creator/SKILL.md.

Strategies to Manage Large Files in Claude Skills

When dealing with oversized assets, the ComposioHQ/awesome-claude-skills repository recommends three specific technical approaches to maintain token efficiency and performance.

Split Files into Logical Chunks

Break massive documents into manageable segments. For example, divide a comprehensive API guide into references/api_part1.md, references/api_part2.md, and so on. This segmentation allows Claude to load only the relevant section rather than ingesting the entire specification. The skill-creator/reference/example.md file demonstrates the expected structure for these standalone reference documents.

Add Grep Patterns for Targeted Loading

In SKILL.md, provide concise search patterns that enable Claude to locate the relevant chunk without loading the whole collection. The grep pattern acts as a filter, matching user queries to specific reference files:

# SKILL.md snippet

- Load references/api_part*.md when the query matches `GET /users`  # grep: `/users`

Use Streaming-Friendly Libraries for Binary Assets

For very large PDFs, Excel sheets, or images, implement read-only processing techniques. The document-skills/xlsx/SKILL.md file demonstrates practical methods for handling massive spreadsheets while keeping memory usage low, following "read-only / write-only" recommendations that prevent loading entire binary files into the LLM context.

Implementation Examples

Loading an API Specification

When users query specific APIs, instruct Claude to load only the relevant reference:

# SKILL.md snippet

- When the user asks about the Payments API, load the spec:
  Load references/payments_api.md   # grep: `payments`

Using Grep with Split Documentation

For segmented guides, use wildcard patterns with grep filters:

# SKILL.md snippet

- Load references/guide_part*.md when the query contains `authentication`
  # grep: `auth|login|token`

Processing Large Excel Files Efficiently

When scripting interactions with large binary assets, use streaming parameters:

import pandas as pd

# Use read_only to avoid loading the whole workbook into memory

df = pd.read_excel('assets/large_dataset.xlsx',
                   sheet_name='Sheet1',
                   usecols=['A', 'C', 'E'],
                   engine='openpyxl',
                   read_only=True)

# Process rows in chunks

for chunk in pd.read_excel(df, chunksize=500):
    # ... handle each chunk ...

    pass

Summary

• Reference files in Claude Skills are required when documentation exceeds 10,000 words or contains static lookup data that would bloat SKILL.md.
• Progressive disclosure ensures Claude only loads reference files when explicitly triggered by user queries, preserving token budget.
• File splitting and grep patterns enable efficient management of large text documents by loading only relevant segments.
• Streaming libraries with read-only flags are essential for processing large binary files like Excel spreadsheets without memory overflow.
• The skill-creator/SKILL.md and document-skills/xlsx/SKILL.md files in ComposioHQ/awesome-claude-skills provide the canonical implementation patterns.

Frequently Asked Questions

How large does a file need to be before I should use a reference file?

According to the ComposioHQ/awesome-claude-skills documentation, you should move content to a reference file when it exceeds approximately 10,000 words or whenever it contains detailed documentation that would clutter the main SKILL.md. The specific threshold depends on whether the content is static reference material that Claude needs to look up rather than core skill instructions.

What is progressive disclosure in Claude Skills?

Progressive disclosure is a design principle where Claude reads SKILL.md first and only loads reference files when encountering explicit load commands such as Load references/filename.md. This mechanism ensures that large documentation does not consume token budget unless the user specifically requests information contained in that reference file, keeping skill initialization fast and efficient.

Can I use wildcards when specifying reference files?

Yes, you can use wildcard patterns such as references/api_part*.md to match multiple segmented files. Combine these with grep patterns in comments (e.g., # grep: /users``) to help Claude identify which specific file from the wildcard set contains the relevant information without loading all matching files into context.

How do I handle large binary files like Excel spreadsheets in Claude Skills?

For large binary assets, implement streaming-friendly processing using libraries like Pandas with read_only=True parameters and chunk-based iteration. The document-skills/xlsx/SKILL.md file in the ComposioHQ/awesome-claude-skills repository demonstrates techniques to process massive spreadsheets while minimizing memory usage, avoiding loading entire workbooks into the LLM context.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}