How 30-seconds-of-code Handles Language Definitions and Grammars for Code Highlighting

The 30-seconds-of-code repository manages syntax highlighting by storing language metadata in YAML files and mapping markdown language aliases to highlighter-specific grammar names through a centralized extraction pipeline that supports both Prism and Shiki.

The Chalarangelo/30-seconds-of-code project maintains thousands of code snippets across multiple programming languages, requiring a flexible system for language definitions and grammars for code highlighting that can adapt to different highlighters without duplicating content logic.

Where Language Metadata and Grammar Mappings Are Stored

The repository separates language information into two complementary data sources to keep content portable and highlighter-agnostic.

Language definitions live in content/languages/*.yaml (for example, javascript.yaml). These files contain human-readable metadata including the short identifier, full name, display name, and reference links:

# content/languages/javascript.yaml

short: js
long: javascript
name: JavaScript
additionalReferences: []
references:
  documentation: https://developer.mozilla.org/en-US/docs/Web/JavaScript

Grammar mappings reside in content/grammars.yaml. This file maps the language identifiers used in markdown code fences (like js or py) to the grammar names required by the underlying highlighter (such as JavaScript or Python):

# content/grammars.yaml (excerpt)

js: JavaScript
ts: TypeScript
py: Python

How Language Data Is Extracted and Loaded

During the build process, src/lib/contentUtils/extractor.js orchestrates data loading through the extractData() function, which accepts a highlighter parameter defaulting to 'shiki':

// src/lib/contentUtils/extractor.js
export const extractData = async (highlighter = 'shiki') => {
  const languages = await extractLanguageData(languageGlob);
  const grammars = await FileHandler.read(grammarPath);
  // ... additional processing
}

The extractLanguageData function, defined in src/lib/contentUtils/modelWorkers/language.js, reads every *.yaml file in the languages directory and constructs a Map keyed by the long identifier:

// src/lib/contentUtils/modelWorkers/language.js
acc.set(long, {
  id: long,
  long,
  short,
  name,
  references,
  allLanguageReferences: [long, ...additionalReferences],
});

Simultaneously, FileHandler.read(grammarPath) loads content/grammars.yaml into a plain object where keys represent markdown language aliases and values contain the target grammar names.

Wiring the Highlighter: Prism and Shiki Integration

Once extracted, the grammar map is injected into the chosen highlighter through a static setup method, enabling dynamic registration of supported languages.

For Prism, located at src/lib/contentUtils/markdownParser/codeHighlighters/prism.js:

static setup(grammars) {
  loadLanguages(Object.keys(grammars));
}

For Shiki, located at src/lib/contentUtils/markdownParser/codeHighlighters/shiki.js:

static setup(grammars) {
  const bundledLanguages = loadBundledLanguages(grammars);
  // ... creates lazy imports for each grammar
}

Both highlighters now maintain an internal mapping that translates markdown language identifiers to their specific grammar implementations.

The Highlighting Pipeline in Action

When parsing markdown content, the highlightCode plugin in src/lib/contentUtils/markdownParser/plugins/ast/highlightCode.js visits every fenced code node in the AST:

// src/lib/contentUtils/markdownParser/plugins/ast/highlightCode.js
visit(tree, `code`, node => {
  const metadata = extractMetadata(node);
  const { languageName } = metadata;
  
  const promise = codeHighlighter
    .highlightCode(node.value, languageName, metadata)
    .then(highlightedCode => {
      node.type = `html`;
      node.value = wrapHighlightedCode(highlightedCode, attributes);
    });
  promises.push(promise);
});

The languageName value derives directly from the code fence's info string (e.g., ```js). The highlighter uses the pre-loaded grammar map to resolve this alias to the correct syntax definition before generating the highlighted HTML output.

Adding a New Language to the System

Extending support for additional languages requires only two declarative changes, with no modifications to the highlighting logic required.

Step 1: Create a new YAML definition in content/languages/. For example, to add Rust support:

# content/languages/rust.yaml

short: rs
long: rust
name: Rust
additionalReferences: []
references: {}

Step 2: Append the grammar alias to content/grammars.yaml:

rust: Rust

Step 3: Rebuild the site. The extractData() function automatically discovers the new language file, registers the grammar with the active highlighter, and enables fenced code blocks written as ```rust to render with proper syntax highlighting.

Summary

• Language metadata resides in content/languages/*.yaml files containing display names and references.
• Grammar mappings in content/grammars.yaml bridge markdown language aliases to highlighter-specific grammar names.
• Data extraction occurs in src/lib/contentUtils/extractor.js, which builds a Map of language data and loads grammar configurations.
• Highlighter initialization passes the grammar map to Prism or Shiki via their respective setup() methods in src/lib/contentUtils/markdownParser/codeHighlighters/.
• Runtime highlighting traverses the AST in src/lib/contentUtils/markdownParser/plugins/ast/highlightCode.js, resolving language identifiers through the pre-loaded map.
• Adding languages is a zero-code configuration task requiring only new YAML files and grammar entries.

Frequently Asked Questions

What file format stores the language definitions in 30-seconds-of-code?

Language definitions are stored as YAML files in the content/languages/ directory. Each file (such as javascript.yaml or python.yaml) contains structured metadata including the short identifier, long identifier, display name, and reference URLs for that programming language.

How does the repository map js to the JavaScript grammar during highlighting?

The mapping occurs through content/grammars.yaml, which contains the entry js: JavaScript. When the build process runs extractData() in src/lib/contentUtils/extractor.js, this file is loaded and passed to the highlighter's setup() method. During markdown parsing, the highlightCode plugin looks up the language identifier from the code fence (e.g., js) in this map to determine which grammar name (e.g., JavaScript) to use for tokenization.

Can I use both Prism and Shiki highlighters in the same build?

The architecture supports both, but the build system selects one highlighter at a time based on the highlighter parameter passed to extractData(). The markdownParser factory in src/lib/contentUtils/markdownParser/index.js instantiates either the Prism or Shij implementation, both of which conform to the same interface with setup() and highlightCode() methods, ensuring consistent behavior regardless of which engine processes the syntax highlighting.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}