How the palace_graph Module Detects Tunnels in MemPalace

The palace_graph module detects tunnels through a three-stage pipeline: loading persisted JSON definitions with strict permissions, normalizing and deduplicating bidirectional wing pairs via create_tunnel(), and optionally synthesizing dynamic connections through compute_topic_tunnels() based on shared topic overlap.

The palace_graph module in the MemPalace repository governs how the memory palace navigates between isolated wings. It treats tunnels as first-class cross-wing connections that enable queries to jump between top-level categories. This article examines the complete tunnel detection lifecycle implemented in mempalace/palace_graph.py, from file-based persistence to runtime inference.

What Are Tunnels in the Palace Graph?

The palace graph represents the memory palace as a directed graph where wings serve as top-level categories and rooms function as time-based sub-categories. By design, the graph restricts direct traversal between wings.

Tunnels are the sole mechanism allowing a query to jump from one wing to another. They function as explicit bridges in the graph topology, enabling cross-wing navigation that would otherwise violate the hierarchical structure.

The Three-Step Tunnel Detection Pipeline

Tunnel detection operates through a strict pipeline that combines static persistence with dynamic generation. The process is fully encapsulated within mempalace/palace_graph.py.

Step 1: Loading Persisted Tunnel Definitions

When the module imports, it immediately invokes the private helper _load_tunnels(). This function reads a JSON file named tunnels.json stored alongside the palace data.

The loader implements defensive file handling:

• Returns an empty list if the file is missing or corrupted
• Creates the file with restrictive permissions if it does not exist (0o600 for the file, 0o700 for its parent directory)

This permission model ensures that tunnel definitions— which may contain sensitive structural metadata—remain accessible only to the file owner.

Step 2: Normalization and Deduplication

The create_tunnel() function handles the normalization logic for tunnel endpoints. It accepts four parameters: wing_a, field_a, wing_b, and field_b.

The function eliminates duplicate entries through bidirectional deduplication. It treats a tunnel from wing_a to wing_b as identical to its reverse (wing_b to wing_a), merging them into a single canonical entry. If the second call supplies a new label, the function updates the existing tunnel record rather than creating a duplicate.

Step 3: Dynamic Topic Tunnel Generation

Beyond static definitions, the module can synthesize topic tunnels dynamically via compute_topic_tunnels(). This function receives a mapping of {wing: [topic_strings]} and an integer min_count.

For every pair of wings that share at least min_count identical topic strings, the function automatically creates a symmetric tunnel entry. This enables automatic cross-wing linking for related subjects without manual configuration, effectively detecting latent connections based on content similarity rather than explicit graph edges.

Querying Detected Tunnels at Runtime

Once detected and loaded, tunnels are queried through the public follow_tunnels(wing, field, col) function. This lookup operation searches the loaded tunnel list for connections originating from or targeting the specified wing and field combination.

The function returns a list of tuples in the format (target_wing, target_field, preview), allowing the calling code to present navigational options or automatically traverse the graph to related wings.

Practical Code Examples

List all defined tunnels with their labels:

from mempalace.palace_graph import list_tunnels

for t in list_tunnels():
    print(f"{t['wing_a']}.{t['field_a']} ↔ {t['wing_b']}.{t['field_b']} ({t.get('label')})")

Create a new tunnel with automatic deduplication and label updates:

from mempalace.palace_graph import create_tunnel, list_tunnels

create_tunnel("wing_code", "auth", "wing_people", "users", label="Auth ↔ Users")

# Duplicate in reverse order – will be merged, label updated

create_tunnel("wing_people", "users", "wing_code", "auth", label="Users ↔ Auth")

print(list_tunnels())   # → one tunnel entry with the latest label

Follow outgoing tunnels from a specific wing and field:

from mempalace.palace_graph import follow_tunnels

outgoing = follow_tunnels("wing_code", "auth", col="title")
for target_wing, target_field, preview in outgoing:
    print(f"→ {target_wing}.{target_field}: {preview}")

Auto-generate tunnels based on shared topic overlap:

from mempalace.palace_graph import compute_topic_tunnels, list_tunnels

topics_by_wing = {
    "wing_code":   ["jwt", "auth"],
    "wing_people": ["auth", "profile"],
    "wing_ops":    ["deploy", "auth"],
}

# Create a tunnel for every shared topic (min_count=1)

compute_topic_tunnels(topics_by_wing, min_count=1)

print(list_tunnels())

Summary

• Tunnel detection combines static JSON persistence (tunnels.json) with dynamic topic analysis to create cross-wing navigation paths.
• The _load_tunnels() helper enforces restrictive file permissions (0o600) and directory permissions (0o700) while handling missing or corrupted files gracefully.
• create_tunnel() handles bidirectional deduplication automatically, treating wing_a ↔ wing_b as identical to wing_b ↔ wing_a and updating labels on conflict.
• compute_topic_tunnels() enables automatic cross-wing linking by analyzing shared topic frequency against a configurable min_count threshold.
• Runtime navigation relies on follow_tunnels() to return (target_wing, target_field, preview) tuples for graph traversal.

Frequently Asked Questions

What file format stores tunnel definitions?

The module persists tunnel definitions in a JSON file named tunnels.json, located alongside the palace data directory. The private _load_tunnels() function manages all read operations, creating the file with 0o600 permissions if it does not exist.

How does the module handle duplicate tunnel definitions?

The create_tunnel() function normalizes endpoint order and removes reverse-ordered duplicates. If you create a tunnel from wing_a to wing_b and later create the reverse from wing_b to wing_a, the module merges them into a single entry and updates the label if the new call provides one.

What permissions does the tunnel storage use?

According to the source code in mempalace/palace_graph.py, the tunnel file receives 0o600 permissions (read/write for owner only), while its parent directory is created with 0o700 permissions. This prevents other system users from accessing the graph structure metadata.

Can tunnels be created automatically without manual definition?

Yes. The compute_topic_tunnels() function generates tunnels dynamically by analyzing topic overlap between wings. When two wings share at least min_count identical topic strings, the function creates a symmetric tunnel entry automatically, enabling discovery of implicit relationships without explicit user configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}