How to Add Custom Languages Using the ast-grep YAML Tier

You add custom languages to the code-graph-rag parser by creating a YAML configuration file in the ast_grep_patterns directory that defines file extensions and ast-grep patterns for functions, classes, and imports, which the AstGrepTier class automatically loads and applies at runtime without modifying any Python source code.

The code-graph-rag repository implements a flexible, pattern-driven parsing system through its ast-grep tier, enabling support for languages that lack native Tree-sitter LanguageSpec definitions. By leveraging declarative YAML configuration files, developers can extend the graph extraction pipeline to new programming languages simply by defining ast-grep patterns that capture structural elements. This approach eliminates the need to write custom Python parsers for each new language supported.

How the AstGrepTier Processes Languages

The AstGrepTier class in codebase_rag/parsers/ast_grep_tier.py orchestrates the entire parsing workflow through a pipeline that relies entirely on YAML-driven pattern matching rather than hardcoded language logic.

Configuration Loading and Mapping

At initialization, the AstGrepTier constructor invokes load_pattern_configs to scan the directory codebase_rag/parsers/ast_grep_patterns and read every *.yaml file present【/cache/repos/github.com/vitali87/code-graph-rag/main/codebase_rag/parsers/ast_grep_tier.py#L38-L61】. The loader builds an internal mapping from file extensions to _LangConfig objects, each storing the language identifier, ast-grep language ID, and the three pattern categories. This mapping enables the tier to dynamically support any language present in the configuration directory without code modification.

File Handling and AST Generation

When processing a repository, the handles(suffix) method checks whether a file extension exists in the loaded mapping【/cache/repos/github.com/vitali87/code-graph-rag/main/codebase_rag/parsers/ast_grep_tier.py#L95-L97】. For matching files, the source code is fed to ast_grep_py.SgRoot using the ast_grep_id specified in the configuration. The resulting AST is then queried against the pattern lists to identify functions, classes, and imports.

Definition and Import Extraction

For every function or class match, the tier extracts the $NAME metavariable to build a qualified name in the format module_qn.name, creating a MODULE → DEFINES → FUNCTION/CLASS relationship【/cache/repos/github.com/vitali87/code-graph-rag/main/codebase_rag/parsers/ast_grep_tier.py#L27-L59】. Import patterns that expose $PATH generate an EXTERNAL_MODULE node and establish MODULE → IMPORTS → EXTERNAL_MODULE edges【/cache/repos/github.com/vitali87/code-graph-rag/main/codebase_rag/parsers/ast_grep_tier.py#L66-L78】.

Required YAML Configuration Fields

Each YAML file in the ast_grep_patterns directory must define six specific fields that instruct the tier how to parse the target language:

• language: Human-readable language name (informational only)
• ast_grep_id: Grammar identifier passed to ast_grep_py to select the correct parser (e.g., ruby, python)
• extensions: List or comma-separated string of file extensions that trigger this configuration (e.g., [".rb"] or .rb,.rbw)
• functions: Ordered list of ast-grep patterns capturing function definitions; the first matching pattern wins. Must use $NAME to capture the function identifier
• classes: Pattern list for class-like entities (classes, modules, structs). Also uses $NAME for the entity name
• imports: Patterns capturing import statements; use $PATH to capture the imported module path

The metavariable conventions ($NAME and $PATH) are defined as constants _NAME_METAVAR and _PATH_METAVAR in the tier source【/cache/repos/github.com/vitali87/code-graph-rag/main/codebase_rag/parsers/ast_grep_tier.py#L24-L28】.

Step-by-Step: Adding a Custom Language

Follow this workflow to add support for any language supported by ast-grep, such as Ruby, PHP, or a custom domain-specific language.

1. Create the YAML Configuration File

Create a new file in codebase_rag/parsers/ast_grep_patterns/ named after your language (e.g., mylang.yaml). The filename is arbitrary but should clearly identify the target language.

2. Define Language Metadata and Extensions

Populate the required identification fields:

language: mylang
ast_grep_id: mylang
extensions:
  - ".mylang"
  - ".mlg"

3. Write Pattern Lists

Define patterns ordered from most specific to most general. The tier evaluates patterns sequentially and uses the first match it encounters:

functions:
  - "public fn $NAME"
  - "def $NAME"
  - "function $NAME"
classes:
  - "class $NAME extends $SUPER"
  - "class $NAME"
imports:
  - "import $PATH from"
  - "require $PATH"

4. Deploy and Verify

Save the file and restart the ingestion service. The AstGrepTier automatically loads the new configuration on startup. Verify recognition programmatically:

from codebase_rag.parsers.ast_grep_tier import AstGrepTier

tier = AstGrepTier(ingestor, repo_path, project_name)
print(tier.handles(".mylang"))  # → True

Practical Implementation Examples

Adding a Fictitious "Foo" Language

Create codebase_rag/parsers/ast_grep_patterns/foo.yaml with this structure:

language: foo
ast_grep_id: foo
extensions:
  - ".foo"
functions:
  - "fn $NAME"
  - "def $NAME"
classes:
  - "class $NAME"
imports:
  - "use $PATH"

When processing a file containing fn greet, the tier creates a node with qualified name ProjectName.src.example.greet and a DEFINES edge from the encompassing module node.

Processing Files Programmatically

Instantiate the tier and process individual files:

from pathlib import Path
from codebase_rag.parsers.ast_grep_tier import AstGrepTier

repo_path = Path("/path/to/repo")
tier = AstGrepTier(ingestor, repo_path, "MyProject")

# Process a specific file

file_path = repo_path / "src/example.foo"
tier.process_file(file_path, {})  # Second arg is parent container info

This invocation automatically loads foo.yaml, parses the source using ast_grep_py, and emits the appropriate MODULE, FUNCTION, and IMPORT nodes with their relationships.

Configuring Import-Only Languages

For domain-specific languages that lack function or class concepts but include include-style statements, omit the optional fields:

language: simpledsl
ast_grep_id: simpledsl
extensions:
  - ".sdsl"
imports:
  - "include $PATH"

The tier still generates a MODULE node for each file and creates IMPORTS edges for every matched include statement.

Summary

• The ast-grep YAML tier provides a zero-code method to extend code-graph-rag to new languages through declarative pattern files.
• Configuration files live in codebase_rag/parsers/ast_grep_patterns/ and define extensions, ast-grep IDs, and capture patterns for functions, classes, and imports.
• Pattern matching relies on ast-grep metavariables $NAME (for definitions) and $PATH (for imports), processed by AstGrepTier to build qualified names and graph relationships.
• Dynamic loading occurs at startup via load_pattern_configs, making new language support available immediately upon file creation without restarting Python processes or modifying source code.

Frequently Asked Questions

What dependencies are required to use the ast-grep YAML tier?

The tier requires the ast_grep_py Python package. If this package is not installed, the AstGrepTier constructor catches the ImportError and disables the tier with a warning message, falling back to other available parsers in the pipeline【/cache/repos/github.com/vitali87/code-graph-rag/main/codebase_rag/parsers/ast_grep_tier.py#L81-L88】.

Can I use multiple file extensions for a single language?

Yes. The extensions field accepts either a YAML list or a comma-separated string. The configuration loader in load_pattern_configs normalizes both formats into a consistent internal representation, allowing you to map multiple extensions (e.g., .rb, .rbw, .rake) to the same ast-grep configuration.

How does the tier handle pattern conflicts within the same file?

The tier processes patterns in the order they appear in the YAML array and selects the first match it encounters. Organize your patterns from most specific to most general to ensure correct extraction. For example, place def self.$NAME before def $NAME to correctly identify static methods in Ruby-like languages.

Is it possible to add custom relationship types beyond DEFINES and IMPORTS?

No. The current implementation in codebase_rag/parsers/ast_grep_tier.py hardcodes two relationship types: MODULE → DEFINES → FUNCTION/CLASS for definition patterns using $NAME, and MODULE → IMPORTS → EXTERNAL_MODULE for import patterns using $PATH. Extending to additional relationship types would require modifying the _process_file method and adding new metavariable conventions to the source code.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}