How to Use ast-grep for Structural Search and Replace in Python: A Complete Guide

Use the ast-grep-py library through a wrapper service to perform language-agnostic AST pattern matching and code transformation, avoiding regex false positives.

The code-graph-rag repository by vitali87 provides a production-ready integration with ast-grep-py that enables structural search and replace operations across multiple programming languages. This guide demonstrates how to use ast-grep for searching code by AST patterns and performing safe refactors that preserve formatting.

Understanding the Core Architecture

The ast-grep integration in code-graph-rag consists of four primary components:

• AstGrepService – Central service in codebase_rag/tools/ast_grep_service.py that loads source files, builds language-specific AST roots, and executes pattern matches.
• create_structural_search_tool – Factory function in codebase_rag/main.py (lines 1648-1650) that exposes search capabilities as a LangChain-compatible tool.
• create_structural_editor_tool – Factory function in codebase_rag/main.py (lines 1651-1653) that enables replacement operations while preserving code formatting.
• AstGrepTier – Parser tier in codebase_rag/parsers/ast_grep_tier.py that registers file extension mappings and loads pattern configuration files.

All components check for the optional ast_grep_py dependency via has_ast_grep() in codebase_rag/utils/dependencies.py, falling back gracefully when unavailable.

Prerequisites and Setup

Before using ast-grep for structural search and replace, install the optional dependency:

pip install ast-grep-py

Verify availability programmatically:

from codebase_rag.utils.dependencies import has_ast_grep

if has_ast_grep():
    print("ast-grep is available for structural operations")
else:
    print("Install ast-grep-py to enable structural search and replace")

The has_ast_grep() function checks for the ast_grep_py package at runtime, allowing the codebase to function without it for non-structural workflows.

Performing Structural Search Operations

Basic Pattern Matching with AstGrepService

Initialize the service and search for AST patterns using JSON or YAML syntax accepted by ast-grep-py:

from codebase_rag.tools.ast_grep_service import AstGrepService

service = AstGrepService(project_root="path/to/your/repo")

# Find all function definitions missing docstrings

pattern = """
{
  "type": "FunctionDef",
  "children": [
    {
      "type": "body",
      "children": [
        {
          "type": "Expr",
          "constraint": "not Str"
        }
      ]
    }
  ]
}
"""

matches = service.search(language="python", pattern=pattern)

for file_path, line, col in matches:
    print(f"{file_path}:{line}:{col} – function missing docstring")

The search() method (lines 147-152 in ast_grep_service.py) walks the SgRoot AST and yields all matching SgNode positions. This approach is immune to false positives from string literals or comments that plague regex-based searches.

Using the LangChain Search Tool

For LLM-driven agents, instantiate the pre-configured tool:

from codebase_rag.main import create_structural_search_tool, AstGrepService

service = AstGrepService(project_root="path/to/repo")
search_tool = create_structural_search_tool(service)

result = search_tool.run("""{
  "language": "python",
  "pattern": "{ \\"type\\": \\"ImportFrom\\", \\"children\\": [{ \\"type\\": \\"module\\", \\"constraint\\": \\"Str == 'os'\\" }] }"
}""")

The tool returns structured match data that LLMs can process for downstream reasoning.

Executing Structural Replace Operations

Single-Pattern Replacement

Replace matched AST nodes while preserving surrounding formatting:

from codebase_rag.tools.ast_grep_service import AstGrepService

service = AstGrepService(project_root="path/to/your/repo")

# Target: all print() calls

search_pattern = """
{
  "type": "Call",
  "children": [
    {
      "type": "func",
      "constraint": "Str == 'print'"
    }
  ]
}
"""

# {{args}} interpolates the original call's arguments

replacement = "logger.info({{args}})"

service.replace(
    language="python",
    pattern=search_pattern,
    replacement=replacement,
    dry_run=False  # Set True to preview without writing files

)

The replace() method (lines 187-197 in ast_grep_service.py) uses SgNode.replace_with() from the underlying library to construct new nodes and writes updated source back to disk.

Using the LangChain Editor Tool

Expose replacement capabilities to agents:

from codebase_rag.main import create_structural_editor_tool, AstGrepService

service = AstGrepService(project_root="path/to/repo")
edit_tool = create_structural_editor_tool(service)

result = edit_tool.run("""{
  "language": "python",
  "pattern": "{ \\"type\\": \\"Call\\", \\"children\\": [{ \\"type\\": \\"func\\", \\"constraint\\": \\"Str == 'print'\\" }] }",
  "replacement": "logger.info({{args}})"
}""")

Adding Support for Additional Languages

The ast-grep integration is language-agnostic. Extend coverage to new languages like Ruby through three steps:

1. Map file extensions in codebase_rag/constants/ast_nodes.py:

# Add to the language ID mapping

AST_GREP_LANGUAGES = {
    ".py": "python",
    ".js": "javascript",
    ".rb": "ruby",  # New entry

    # ... existing mappings

}

2. Create a pattern config at codebase_rag/parsers/ast_grep_patterns/ruby.yaml:

ast_grep_id: ruby
default_patterns:
  - name: class_definition
    pattern: |
      class $NAME
        $BODY
      end

3. Automatic registration: AstGrepTier scans the ast_grep_patterns/ directory at startup and loads all YAML configurations without code changes.

Advanced Pattern Syntax

ast-grep patterns use a JSON/YAML structure where:

• type – AST node type from the target language's grammar.
• children – Nested node specifications for tree traversal.
• constraint – Boolean or comparison expressions using ==, !=, not, etc.
• Meta-variables ($NAME, {{args}}) – Capture and interpolate matched content.

Reference the ast-grep pattern syntax documentation for language-specific node types.

Integration with the RAG Pipeline

According to the code-graph-rag source code, structural tools integrate into the main LangChain agent in codebase_rag/main.py. The agent can receive natural language prompts like:

• "Find all if statements that lack an else clause"
• "Replace every print call with a logger call"

The AstGrepTier parser tier enables these capabilities by:

• Walking source directories via _iter_source_files()
• Filtering by supported extensions using mappings in constants/ast_python.py, constants/ast_php.py, etc.
• Registering pattern rules from ast_grep_rules/ for custom analyzers in codebase_rag/analyzers/ast_grep_analyzer.py

Testing and Validation

The repository includes comprehensive tests in codebase_rag/tests/test_structural_tools.py demonstrating tool behavior with and without ast_grep_py installed. Run these to verify your integration:

python -m pytest codebase_rag/tests/test_structural_tools.py -v

Use dry_run=True in service.replace() to preview changes without modifying files:

preview = service.replace(
    language="python",
    pattern=search_pattern,
    replacement=replacement,
    dry_run=True
)
print(preview)  # Shows proposed file modifications

Summary

• Install ast-grep-py optionally and verify with has_ast_grep() from codebase_rag/utils/dependencies.py.
• Initialize AstGrepService with your project root to enable structural operations.
• Use service.search() for pattern matching and service.replace() for transformations, both preserving AST structure.
• Leverage LangChain tools via create_structural_search_tool() and create_structural_editor_tool() for agent integration.
• Extend language support by adding mappings in constants/ast_nodes.py and YAML configs in parsers/ast_grep_patterns/.
• Preview changes with dry_run=True before applying replacements.

Frequently Asked Questions

What is the difference between ast-grep and regex search?

ast-grep operates on the abstract syntax tree rather than raw text, eliminating false positives from matches inside strings or comments. It understands language grammar—so searching for a "function call" matches only actual call expressions, not the word "call" in a docstring. The code-graph-rag implementation in AstGrepService provides this capability across Python, JavaScript, PHP, Rust, Go, and extensible languages.

How does the dry_run parameter work in replace operations?

Setting dry_run=True in AstGrepService.replace() executes the pattern match and replacement logic but outputs proposed changes without writing files. This enables safe previewing of refactors. The implementation builds replacement nodes via SgNode.replace_with() and returns the modified source strings rather than persisting them. Set dry_run=False to apply changes to disk.

Can I use ast-grep patterns from YAML files instead of inline JSON?

Yes. The AstGrepTier in codebase_rag/parsers/ast_grep_tier.py automatically loads pattern configurations from parsers/ast_grep_patterns/. Place YAML files with ast_grep_id and pattern definitions there, then reference them by name in your search calls. The AstGrepAnalyzer in codebase_rag/analyzers/ast_grep_analyzer.py demonstrates consuming custom rules from ast_grep_rules/ for specialized analysis tasks.

What happens if ast-grep-py is not installed?

All structural tools gracefully degrade. has_ast_grep() returns False, and the tool factories in codebase_rag/main.py return no-op implementations that log warnings. The repository remains functional for non-structural RAG operations using text-based and tree-sitter parsers without requiring the optional dependency.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}