How the Deep Merge Algorithm Recursively Handles Nested JSON Dictionaries in Spec-Kit

The deep merge algorithm in Spec-Kit recursively merges nested JSON dictionaries by copying the base dictionary, iterating through the update dictionary, and recursively calling itself whenever both the existing value and new value are dictionaries, while non-dictionary values are replaced directly.

The Spec-Kit repository provides a robust utility for combining JSON configuration files without losing nested data structures. Understanding how the deep merge algorithm recursively handles nested JSON dictionaries is essential for safely merging settings templates and existing configurations. This analysis examines the implementation details found in the source code to reveal exactly how the recursion preserves existing values while incorporating updates.

Core Implementation in src/specify_cli/__init__.py

The recursive logic resides in the deep_merge helper function, defined as an inner function within merge_json_files at lines 83-92 of src/specify_cli/__init__.py. This implementation prioritizes immutability and type safety when combining dictionary structures.

def deep_merge(base: dict, update: dict) -> dict:
    """Recursively merge update dict into base dict."""
    result = base.copy()
    for key, value in update.items():
        if (
            key in result
            and isinstance(result[key], dict)
            and isinstance(value, dict)
        ):
            # Both sides are dictionaries → recurse

            result[key] = deep_merge(result[key], value)
        else:
            # New key or non-dict value → replace / add

            result[key] = value
    return result

Step-by-Step Recursion Logic

The algorithm follows a six-step process to ensure safe merging of arbitrarily deep nesting:

1. Shallow Copy Preservation: The function begins with result = base.copy() to create a shallow copy of the existing dictionary, ensuring the original structure remains untouched.

2. Update Iteration: It loops through every key, value pair in the update dictionary using update.items(), visiting each field that requires potential modification.

3. Type Detection: For each key, it checks if the key exists in the result and whether both result[key] and the incoming value are dictionary instances using isinstance(..., dict).

4. Recursive Descent: When both values are dictionaries, the algorithm calls deep_merge(result[key], value) recursively, climbing deeper into the hierarchy to merge nested objects.

5. Direct Assignment: For new keys or non-dictionary values—including lists, strings, and numbers—the update value is assigned directly via result[key] = value, overwriting any existing scalar values or adding new entries.

6. Result Return: The fully merged dictionary is returned to the caller, ready for serialization back to disk.

Practical Code Examples

Simple Merge with Nested Configuration

The following example demonstrates how the algorithm combines overlapping nested objects while preserving distinct keys:

from pathlib import Path
from specify_cli import merge_json_files

# Existing JSON (saved on disk)

existing = Path("settings.json")
existing.write_text('{"database": {"host": "localhost", "port": 5432}, "debug": false}')

# New content to merge

new_content = {
    "database": {"port": 5433, "user": "admin"},
    "feature_flags": {"beta": True}
}

merged = merge_json_files(existing, new_content, verbose=False)
print(merged)

Output:

{
  "database": {
    "host": "localhost",
    "port": 5433,
    "user": "admin"
  },
  "debug": false,
  "feature_flags": {
    "beta": true
  }
}

The database object is merged recursively: port is overwritten, user is added, while host remains unchanged. The feature_flags key is new, so it is appended directly.

Deeply Nested Multi-Level Merge

For structures with multiple layers of nesting, the recursion descends arbitrarily deep:

old = {"a": {"b": {"c": 1, "d": 2}}, "x": 10}
new = {"a": {"b": {"c": 5, "e": 3}}, "y": 20}

merged = merge_json_files(Path("tmp.json"), new, verbose=False)
print(merged)

Output:

{
  "a": {
    "b": {
      "c": 5,
      "d": 2,
      "e": 3
    }
  },
  "x": 10,
  "y": 20
}

The recursion dives three levels deep (a → b) to merge c, preserve d, and add e. Top-level keys x and y are handled as simple additions.

Handling Edge Cases and Non-Dictionary Values

Because the recursion only triggers when both sides are dictionaries, lists and other mutable containers are replaced rather than merged. This design avoids ambiguous merge strategies, such as deciding whether to concatenate or intersect two lists.

The public wrapper merge_json_files(existing_path, new_content, verbose) orchestrates the entire workflow. It loads the existing file from disk, invokes deep_merge to combine the data structures, and returns the merged dictionary. If the existing file is missing or contains malformed JSON, the wrapper gracefully falls back to returning the new content directly, ensuring the application never crashes on invalid input.

Summary

• The deep_merge function in src/specify_cli/__init__.py creates a shallow copy of the base dictionary to preserve the original data structure before modification.
• Recursion occurs only when both the existing value and the update value are dictionary instances, enabling safe handling of arbitrarily deep nesting.
• Non-dictionary values—including lists, strings, and numbers—are always replaced rather than merged, preventing ambiguous list concatenation or type coercion.
• The merge_json_files wrapper handles file I/O, JSON parsing, and error recovery, returning the merged result or falling back to new content when files are missing or malformed.

Frequently Asked Questions

How does the algorithm handle list values during a merge?

List values are treated as scalar values and replaced entirely rather than merged element-by-element. When the update dictionary contains a list for a key that exists in the base dictionary, the isinstance(..., dict) check fails, causing the algorithm to execute the else branch and assign result[key] = value directly. This replacement strategy avoids ambiguity about whether lists should be concatenated, intersected, or deduplicated.

Where is the deep merge function located in the Spec-Kit repository?

The deep_merge function is implemented as an inner function inside merge_json_files in src/specify_cli/__init__.py, specifically at lines 83-92. This location places the recursive logic alongside the file handling wrapper that loads existing JSON and writes merged results, keeping the merge utility self-contained within the CLI module.

Does the deep merge modify the original JSON files in place?

No, the algorithm explicitly avoids modifying the original base dictionary in place. The first operation inside deep_merge is result = base.copy(), which creates a shallow copy of the top-level dictionary. While nested dictionaries are mutated during the recursive merge, the original base dictionary passed to the function remains unchanged, ensuring safe reuse of configuration objects across multiple merge operations.

What happens when the existing file is missing or malformed?

The merge_json_files wrapper function handles missing or malformed files by catching exceptions during the file load and JSON parse phases. When an error occurs, the function returns the new_content dictionary directly without attempting to merge, effectively treating the missing file as an empty dictionary. This fallback ensures that configuration updates always succeed, even when the target file does not exist or contains invalid JSON syntax.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}