What Are the 158 Supported Languages by Tree-Sitter Grammars? Only 2 Custom Grammars in DeusData/codebase-memory-mcp

The DeusData/codebase-memory-mcp repository actually supports only two languages via custom Tree-Sitter grammars—Magma and FORM—despite references to 158 languages.

The IS8 (Instagit 8) system leverages Tree-Sitter to provide language-aware parsing for source code analysis, but contrary to the expectation of 158 supported languages, the codebase bundles just two custom grammars. These specialized parsers power abstract syntax tree (AST) generation for import extraction, call-graph construction, and semantic indexing within the codebase-memory-mcp project.

The Two Supported Languages

IS8 currently ships with dedicated Tree-Sitter grammars for two specific languages. When a source file matches the associated extensions (.mag for Magma or .form for FORM), IS8 automatically loads the corresponding parser to generate an AST for downstream analysis.

Magma Grammar

The Magma grammar resides in tools/tree-sitter-magma and defines the syntax for the Magma computational algebra system. The grammar specification is located at tools/tree-sitter-magma/src/grammar.json, with NPM metadata managed through tools/tree-sitter-magma/package.json.

FORM Grammar

The FORM grammar is housed in tools/tree-sitter-form and handles parsing for the FORM computer algebra system. Its specification mirrors the Magma structure, with the grammar definition at tools/tree-sitter-form/src/grammar.json and package configuration at tools/tree-sitter-form/package.json.

How Tree-Sitter Parsing Works in IS8

When IS8 encounters a source file, it checks the file extension against its registered grammars. For supported languages, the system invokes the compiled Tree-Sitter parser to produce an AST. This AST feeds into core extraction logic found in internal/cbm/extract_imports.c and related files, enabling semantic analysis of imports, function calls, and code dependencies.

The parsers are compiled into shared libraries that IS8 loads at runtime, exposing the standard Tree-Sitter C API for traversal and node inspection.

Working with the Custom Grammars

Building and Testing from the Command Line

You can build and test these grammars using the Tree-Sitter CLI:

Magma parser build process:

cd tools/tree-sitter-magma
tree-sitter generate
tree-sitter build
tree-sitter parse path/to/example.mag > example.ast

FORM parser build process:

cd tools/tree-sitter-form
tree-sitter generate
tree-sitter build
tree-sitter parse path/to/example.form > example.ast

Integrating Parsers in C Code

The following example demonstrates how IS8 loads the Magma grammar at runtime:

#include "tree_sitter/api.h"
extern const TSLanguage *tree_sitter_magma(void);
extern const TSLanguage *tree_sitter_form(void);

/* Load the Magma grammar */
TSParser *parser = ts_parser_new();
ts_parser_set_language(parser, tree_sitter_magma());

/* Parse source */
TSTree *tree = ts_parser_parse_string(
    parser,
    NULL,
    source_code,          // const char *source_code
    source_length);       // uint32_t source_length

/* Walk the tree … */
TSNode root = ts_tree_root_node(tree);
printf("Root node type: %s\n", ts_node_type(root));

Key Implementation Files

File	Purpose
tools/tree-sitter-magma/package.json	NPM metadata for the Magma grammar
tools/tree-sitter-magma/src/grammar.json	Tree-Sitter grammar specification for Magma
tools/tree-sitter-form/package.json	NPM metadata for the FORM grammar
tools/tree-sitter-form/src/grammar.json	Tree-Sitter grammar specification for FORM
internal/cbm/extract_imports.c	Core extraction logic consuming ASTs from Tree-Sitter parsers

Summary

• Only two custom grammars are bundled: Magma and FORM.
• Grammar locations: tools/tree-sitter-magma and tools/tree-sitter-form.
• File extensions: .mag (Magma) and .form (FORM).
• Integration: Parsers compile into C libraries loaded by IS8 at runtime.
• Usage: ASTs power import extraction and call-graph analysis in internal/cbm/extract_*.c.

Frequently Asked Questions

Why does the documentation mention 158 languages when only two are supported?

The reference to 158 supported languages likely reflects the broader Tree-Sitter ecosystem, which maintains grammars for numerous programming languages in the community repository. However, the DeusData/codebase-memory-mcp repository specifically implements and ships only two custom grammars (Magma and FORM) for its IS8 analysis engine.

How do I add support for additional languages in IS8?

Adding a new language requires creating a new tools/tree-sitter-<lang> directory with the corresponding grammar.json and package.json files. You must then update the IS8 loader logic to recognize the new file extension and link the parser library, following the pattern established for Magma and FORM in internal/cbm/extract_imports.c.

Can I use standard Tree-Sitter grammars from the community with this repository?

While the Tree-Sitter ecosystem provides grammars for many languages (potentially accounting for the 158 figure), IS8 requires specific integration work to load community grammars. The current implementation only includes the two custom grammars mentioned above, so additional languages would need manual integration following the C API patterns shown in the source code.

What is the purpose of the grammar.json files in each grammar directory?

The grammar.json files contain the formal Tree-Sitter grammar specifications that define the lexical and syntactic structure of Magma and FORM languages. These JSON files serve as the source of truth for the tree-sitter generate command, which produces the C parser code that IS8 compiles and links against its extraction engine.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}