How to Troubleshoot "Encryption Key Mismatch" Errors in MasterDnsVPN

An "encryption key mismatch" occurs when the derived cryptographic material on the client differs from the server's stored key, causing UDP packet decryption failures that manifest as dropped connections or "invalid ciphertext" errors.

MasterDnsVPN uses symmetric encryption to secure UDP payloads between clients and servers. When the client and server disagree on the raw encryption key or the selected cipher method, the Codec cannot decrypt traffic, resulting in silent packet drops or explicit error logs. This guide explains how to diagnose and resolve these mismatches using the actual source code implementation from the masterking32/MasterDnsVPN repository.

Understanding the Encryption Architecture

The encryption handshake relies on two independent configuration sources that must produce identical derived keys.

Server-side key handling is managed in internal/security/encryption_key.go. On startup, the server calls EnsureServerEncryptionKey(cfg), which reads the file path returned by cfg.EncryptionKeyPath() (defined in internal/config/server.go, lines 26-33). If the file is missing or contains a key of incorrect length, the server generates a new hex key of requiredKeyLength and writes it to disk with 0600 permissions.

Client-side key handling is defined in internal/config/client.go (lines 59-61). The client reads the ENCRYPTION_KEY field from its configuration, trims whitespace in finalizeClientConfig (lines 68-70), and aborts with "ENCRYPTION_KEY is required" if the field is empty.

Codec construction happens in internal/security/codec.go (lines 70-99). Both sides instantiate a Codec via NewCodec(method, rawKey), which transforms the raw key using deriveKey (SHA-256, MD5, or truncation) into a byte slice of the exact length required for the chosen algorithm (e.g., 16 bytes for AES-128-GCM, 32 bytes for ChaCha20). If the derived keys differ, decryption fails at codec.go line 29 with an "invalid ciphertext" error.

CLI overrides provide debugging flexibility. The server binary (cmd/server/main.go, lines 42-46) accepts a --genkey flag to force regeneration, while the client binary (cmd/client/main.go, lines 164-168) accepts a -k flag to pass an explicit key via command line.

Common Causes of Encryption Key Mismatches

Server key file regeneration occurs when ENCRYPTION_KEY_FILE points to a non-existent path or a file with incorrect permissions. The server logs keyInfo.Generated = true (visible in cmd/server/main.go lines 43-45) and creates a fresh key, breaking existing clients still using the old key.

Client key whitespace contamination happens when the ENCRYPTION_KEY value in client.cfg contains trailing spaces or newline characters. While finalizeClientConfig trims the value, manual editing or copy-paste errors often introduce invisible characters that alter the derived key.

Method ID mismatch surfaces when DATA_ENCRYPTION_METHOD differs between server.cfg and client.cfg. This integer (0-5) determines the cipher algorithm; if the server uses method 2 (ChaCha20) and the client uses method 1 (XOR), the NewCodec function produces incompatible encryption contexts even with identical raw keys.

Key length violations trigger silent failures. AES-192-GCM requires exactly 24 bytes, while other methods typically require 32 bytes. If the stored key in encryption_key.go (lines 40-44) fails the len(key) validation against requiredKeyLength, the server regenerates it, causing a mismatch.

Step-by-Step Troubleshooting Workflow

1. Verify the encryption method ID

   Both configuration files must specify the same integer for DATA_ENCRYPTION_METHOD.

   grep DATA_ENCRYPTION_METHOD server.cfg
   grep DATA_ENCRYPTION_METHOD client.cfg

2. Inspect the server key file

   Locate the path via cfg.EncryptionKeyPath() and verify the file exists, is readable, and contains exactly the required number of hex characters (32 for most methods, 24 for AES-192-GCM, 16 for AES-128-GCM).

   KEY_PATH=$(grep ENCRYPTION_KEY_FILE server.cfg | awk -F'=' '{print $2}' | xargs)
   cat "$KEY_PATH" | wc -c
   # Expected: 32 (or required length for your method)
   

3. Validate file permissions

   The server writes the key with mode 0600. Incorrect permissions prevent reading, triggering regeneration.

   stat -c "%a %U %G" "$KEY_PATH"
   # Expected: 600, owned by the service user
   

4. Check the client raw key

   Ensure ENCRYPTION_KEY in client.cfg matches the server file content exactly, with no surrounding whitespace or quotes.

   grep ENCRYPTION_KEY client.cfg

5. Test with explicit CLI overrides

   Isolate configuration file issues by specifying values directly.

   # Server: generate a known key to /tmp/test_key.txt
   
   masterdnsvpn-server --genkey -key-file /tmp/test_key.txt
   
   # Client: use the same key string
   
   masterdnsvpn-client -k $(cat /tmp/test_key.txt)

6. Enable debug logging

   Set LOG_LEVEL = "DEBUG" in both configurations. The server will log key generation events and the derived key length, while the client will surface codec errors leading to "too many mismatched dns responses" in tunnel_runtime.go.

7. Monitor for specific error signatures

   • Server side: Look for invalid ciphertext errors originating from codec.go line 29.
   • Client side: Look for decryption failures bubbling up from codec.Decrypt.

8. Synchronize fresh keys

   If corruption is suspected, regenerate and redistribute:

   masterdnsvpn-server --genkey
   # Securely copy the generated file to client hosts and update ENCRYPTION_KEY
   

9. Verify end-to-end connectivity

   Run a DNS query through the tunnel. Successful resolution without "mismatch" logs confirms the keys are synchronized.

Practical Code Examples

Generating a New Server Key via CLI

Force the server to create a new encryption key file, useful when the existing key is lost or compromised.

masterdnsvpn-server --genkey

This invokes the logic in cmd/server/main.go (lines 42-46), which calls security.EnsureServerEncryptionKey and writes the output to the path specified by ENCRYPTION_KEY_FILE.

Overriding the Client Key from Command Line

Bypass the configuration file to test specific key strings during debugging.

masterdnsvpn-client -k 0123456789abcdef0123456789abcdef

The implementation in cmd/client/main.go (lines 164-168) stores this override in overrides.Values["EncryptionKey"], which finalizeClientConfig (lines 68-71) applies before validation.

Programmatic Key Verification

Use this Go snippet to verify that your server configuration loads the expected key length without triggering regeneration.

package main

import (
	"fmt"
	"masterdnsvpn-go/internal/config"
	"masterdnsvpn-go/internal/security"
)

func main() {
	sCfg, err := config.LoadServerConfig("server.cfg")
	if err != nil {
		panic(err)
	}
	
	keyInfo, err := security.EnsureServerEncryptionKey(sCfg)
	if err != nil {
		panic(err)
	}
	
	fmt.Printf("Key length: %d bytes, Loaded: %v, Generated: %v\n",
		len(keyInfo.Key), keyInfo.Loaded, keyInfo.Generated)
}

This reproduces the validation logic in internal/security/encryption_key.go (lines 29-58) and confirms whether the key file meets the requiredKeyLength for the selected DATA_ENCRYPTION_METHOD.

Summary

• Encryption key mismatches stem from divergent derived keys or mismatched DATA_ENCRYPTION_METHOD IDs between client and server.
• Server-side issues usually involve missing key files, incorrect permissions, or accidental regeneration via --genkey.
• Client-side issues typically involve whitespace in ENCRYPTION_KEY or empty configuration values.
• Validation requires checking file lengths (16, 24, or 32 bytes), permissions (0600), and method ID parity.
• Resolution involves regenerating the key with CLI flags, securely distributing the new value, and verifying via debug logs or DNS queries.

Frequently Asked Questions

What does "invalid ciphertext" mean in MasterDnsVPN logs?

This error originates in internal/security/codec.go (line 29) when the Decrypt function fails to authenticate a packet. It indicates that the derived encryption key on the receiving side does not match the key used to encrypt the packet. Verify that both endpoints use identical raw keys and the same DATA_ENCRYPTION_METHOD.

Why does my server generate a new key on every restart?

The server calls EnsureServerEncryptionKey in internal/security/encryption_key.go (lines 29-58) each time it starts. If the file specified by ENCRYPTION_KEY_FILE is missing, unreadable, or contains a key of incorrect length, the server automatically generates a new one. Check that the path in server.cfg is absolute, the file exists, and permissions are set to 0600.

Can I use different encryption methods on the client and server?

No. The DATA_ENCRYPTION_METHOD (an integer 0-5) must be identical on both sides. This value determines the cipher algorithm (XOR, ChaCha20, AES-GCM) and the key derivation function in NewCodec. A mismatch causes immediate decryption failures because the algorithms are incompatible.

How long should the encryption key be?

The required length depends on the selected method. AES-128-GCM (method 3) requires 16 bytes; AES-192-GCM (method 4) requires 24 bytes; ChaCha20 and other methods require 32 bytes. The server validates this in encryption_key.go (lines 40-44) against requiredKeyLength. Keys are typically represented as hex strings, so the character count is double the byte count (e.g., 64 hex characters for 32 bytes).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}