How Shadowsocks-Windows Implements Encryption: AEAD Ciphers, Stream Ciphers, and the Encryptor Factory

Shadowsocks-Windows abstracts all cryptographic operations behind the IEncryptor interface, using a factory pattern to instantiate AEAD encryptors (mbed TLS, OpenSSL, libsodium) or stream ciphers while handling key derivation, salt generation, and chunk framing automatically.

The shadowsocks/shadowsocks-windows repository provides a production-grade encryption layer that balances security with performance. This implementation supports modern AEAD ciphers like AES-256-GCM and ChaCha20-Poly1305 alongside a legacy stream cipher fallback, all orchestrated through a centralized EncryptorFactory. Understanding this architecture requires examining the interface contracts, the concrete cipher implementations, and the factory registration pattern that binds them together.

The IEncryptor Interface and Base Abstraction

All encryptors in Shadowsocks-Windows implement the IEncryptor interface defined in shadowsocks-csharp/Encryption/IEncryptor.cs. This contract mandates methods for both TCP and UDP traffic, plus a property for handling SOCKS5 address headers.

public interface IEncryptor : IDisposable
{
    int AddrBufLength { set; get; }
    void Encrypt(byte[] buf, int length, byte[] outbuf, out int outlength);
    void Decrypt(byte[] buf, int length, byte[] outbuf, out int outlength);
    void EncryptUDP(byte[] buf, int length, byte[] outbuf, out int outlength);
    void DecryptUDP(byte[] buf, int length, byte[] outbuf, out int outlength);
}

The EncryptorBase class in shadowsocks-csharp/Encryption/EncryptorBase.cs provides shared infrastructure for all implementations. It stores the method and password, defines constants like MAX_INPUT_SIZE, and declares the abstract methods that concrete encryptors override. This base class also implements the AddrBufLength property used to handle variable-length SOCKS5 address headers during the initial handshake.

AEAD Cipher Implementation

Authenticated Encryption with Associated Data (AEAD) represents the primary encryption family in Shadowsocks-Windows. The abstract AEADEncryptor class in shadowsocks-csharp/Encryption/AEAD/AEADEncryptor.cs orchestrates the high-level protocol, leaving specific cryptographic primitives to backend-specific subclasses.

Cipher Metadata and Registry

Each concrete AEAD implementation maintains a static dictionary mapping cipher names to EncryptorInfo structs. These metadata records—defined within EncryptorBase—specify the key size, salt size, nonce size, tag size, and algorithm type.

In shadowsocks-csharp/Encryption/AEAD/AEADMbedTLSEncryptor.cs and AEADOpenSSLEncryptor.cs, you will find registrations like:

private static readonly Dictionary<string, EncryptorInfo> _ciphers = new()
{
    {"aes-128-gcm", new EncryptorInfo("AES-128-GCM", 16, 16, 12, 16, CIPHER_AES)},
    {"aes-256-gcm", new EncryptorInfo("AES-256-GCM", 32, 32, 12, 16, CIPHER_AES)},
    {"chacha20-poly1305", new EncryptorInfo("CHACHA20-POLY1305", 32, 32, 12, 16, CIPHER_CHACHA20POLY1305)}
};

Key Derivation Architecture

Shadowsocks-Windows employs a two-tier key derivation system. First, the master key is derived from the user password using a custom MD5-based KDF implemented in AEADEncryptor.DeriveKey. This loops through MD5 hashes until sufficient key material is generated.

Second, a per-connection session key is derived from the master key and a random salt using HKDF. The method DeriveSessionKey in AEADEncryptor calls backend-specific HKDF implementations—such as MbedTLS.hkdf or the OpenSSL equivalent—to produce unique session keys for every TCP connection or UDP packet.

Salt and Nonce Management

When initiating a TCP connection, the encryptor generates a random salt using RNG.GetBytes with length specified by the cipher's saltLen. This salt is transmitted in cleartext as the first bytes of the stream. The peer uses this identical salt to derive the same session key independently.

For nonce management, AEAD encryptors use a 12-byte nonce (for GCM and ChaCha20-Poly1305) initialized to zero. After each encrypted chunk, the nonce is incremented using Sodium.sodium_increment or equivalent arithmetic to ensure uniqueness across the session.

TCP Chunk Framing Protocol

Shadowsocks-Windows segments TCP streams into chunks of maximum size 0x3FFF (16,383 bytes). Each chunk undergoes a specific framing process implemented in AEADEncryptor.ChunkEncrypt:

• The payload length is encrypted as a 2-byte block with authentication tag
• The actual payload is encrypted with a separate authentication tag
• Both are concatenated to the output buffer

The resulting stream format is: [salt][encrypted length + tag][encrypted payload + tag]. Decryption reverses this process, first decrypting the length field to determine how many bytes to read for the payload.

Backend-Specific Implementations

Concrete implementations handle the actual cryptographic operations through different native libraries:

• mbed TLS: AEADMbedTLSEncryptor calls MbedTLS.cipher_auth_encrypt and cipher_auth_decrypt for authenticated encryption
• OpenSSL: AEADOpenSSLEncryptor uses the EVP API with EVP_CipherUpdate, EVP_CipherFinal_ex, and EVP_CIPHER_CTX_ctrl for tag handling
• libsodium: AEADSodiumEncryptor (following the same pattern) invokes libsodium's AEAD functions

Each backend implements cipherEncrypt and cipherDecrypt methods that the abstract AEADEncryptor calls during chunk processing.

UDP Encryption

For UDP traffic, the framing differs from TCP. The EncryptUDP method in AEADEncryptor prepends a fresh random salt to each datagram, initializes the cipher context with InitCipher, and encrypts the entire payload in a single AEAD operation. No chunking is required since UDP preserves message boundaries.

Stream Cipher Support (PlainEncryptor)

While modern Shadowsocks deployments prefer AEAD ciphers, the codebase maintains support for unencrypted "plain" or "none" methods via PlainEncryptor in shadowsocks-csharp/Encryption/Stream/PlainEncryptor.cs. This class implements IEncryptor by simply copying input buffers to output buffers without transformation. It registers the methods "plain" and "none" in its SupportedCiphers() method, serving as a fallback when encryption is disabled.

The Encryptor Factory Pattern

The EncryptorFactory class in shadowsocks-csharp/Encryption/EncryptorFactory.cs decouples the networking code from specific cryptographic implementations using reflection-based instantiation.

Static Cipher Registration

The factory's static constructor discovers all supported ciphers at application startup. It populates a private dictionary _registeredEncryptors that maps method names (like "aes-256-gcm") to concrete Type objects:

foreach (string method in AEADOpenSSLEncryptor.SupportedCiphers())
    _registeredEncryptors[method] = typeof(AEADOpenSSLEncryptor);

foreach (string method in AEADSodiumEncryptor.SupportedCiphers())
    _registeredEncryptors[method] = typeof(AEADSodiumEncryptor);

foreach (string method in AEADMbedTLSEncryptor.SupportedCiphers())
    _registeredEncryptors[method] = typeof(AEADMbedTLSEncryptor);

// Remove duplicates if libsodium provides AES-256-GCM
if (AEADSodiumEncryptor.isSupportAES256GCM())
    _registeredEncryptors.Remove("aes-256-gcm");

// Register plain fallback last
foreach (string method in PlainEncryptor.SupportedCiphers())
    _registeredEncryptors[method] = typeof(PlainEncryptor);

Runtime Instantiation

The GetEncryptor method normalizes the method name to lowercase, retrieves the corresponding Type from the registry, and uses reflection to invoke the constructor:

Type t = _registeredEncryptors[method];
ConstructorInfo c = t.GetConstructor(new Type[] { typeof(string), typeof(string) });
return (IEncryptor)c.Invoke(new object[] { method, password });

This design allows the TCP relay and UDP handlers to request encryptors by method name string alone, without knowing which crypto library (OpenSSL, mbed TLS, or libsodium) will ultimately handle the operations.

Practical Usage Examples

Creating an Encryptor via the Factory

using Shadowsocks.Encryption;

string method = "aes-256-gcm";
string password = "secure-password";

IEncryptor encryptor = EncryptorFactory.GetEncryptor(method, password);
encryptor.AddrBufLength = 3; // SOCKS5 header length for IPv4

Encrypting TCP Payloads

byte[] plaintext = System.Text.Encoding.UTF8.GetBytes("Sensitive data");
byte[] ciphertext = new byte[EncryptorBase.MAX_INPUT_SIZE * 2];

encryptor.Encrypt(plaintext, plaintext.Length, ciphertext, out int cipherLen);

// ciphertext now contains: [salt][encrypted length+tag][encrypted data+tag]

Handling UDP Packets

byte[] udpData = System.Text.Encoding.UTF8.GetBytes("UDP message");
byte[] udpPacket = new byte[1500];

encryptor.EncryptUDP(udpData, udpData.Length, udpPacket, out int packetLen);
// Send udpPacket[0..packetLen] via socket

Summary

• Shadowsocks-Windows encryption relies on the IEncryptor interface to abstract TCP and UDP cryptographic operations, with EncryptorBase providing shared constants and password storage.
• AEAD ciphers (AES-GCM, ChaCha20-Poly1305) are implemented through the AEADEncryptor base class, which handles HKDF key derivation, random salt generation, nonce incrementing, and chunk-based TCP framing.
• Backend-specific implementations in AEADMbedTLSEncryptor, AEADOpenSSLEncryptor, and AEADSodiumEncryptor bind these operations to native crypto libraries via P/Invoke.
• Stream ciphers are supported through PlainEncryptor, which provides a pass-through implementation for "plain" and "none" methods.
• The EncryptorFactory maintains a registry of all supported methods and instantiates the appropriate concrete class using reflection, decoupling the networking layer from cryptographic details.

Frequently Asked Questions

What AEAD ciphers does Shadowsocks-Windows support?

Shadowsocks-Windows supports AES-128-GCM, AES-192-GCM, AES-256-GCM, and ChaCha20-Poly1305 through the AEAD encryptor implementations. The specific ciphers available depend on which crypto backends are compiled into the application—mbed TLS, OpenSSL, or libsodium. Each backend registers its supported algorithms with the EncryptorFactory during static initialization.

How does the encryptor factory choose between mbed TLS and OpenSSL?

The EncryptorFactory does not dynamically choose at runtime; rather, it registers all available backends in its static constructor. If multiple backends support the same cipher (such as AES-256-GCM being available in both mbed TLS and libsodium), the factory removes duplicates to ensure only one implementation handles each method name. The specific priority is: OpenSSL first, then Sodium, then mbed TLS, with deduplication logic ensuring no method name collisions remain.

Why does Shadowsocks-Windows use HKDF for session keys instead of using the password directly?

Shadowsocks-Windows uses a custom MD5-based KDF to derive a master key from the password once, then uses HKDF to derive per-connection session keys from random salts and the master key. This two-tier approach ensures that compromising one session's key does not expose the master key or other sessions' traffic. The random salt transmitted in each connection ensures key uniqueness even when the password remains constant across connections.

What is the maximum payload size for TCP chunks in Shadowsocks-Windows?

The maximum payload size for individual TCP chunks is 0x3FFF bytes (16,383 bytes). This limit is enforced by the ChunkEncrypt method in AEADEncryptor. Larger streams are automatically segmented into multiple chunks, each with its own authentication tag, preventing memory exhaustion attacks while maintaining the authenticated encryption guarantees of the AEAD construction.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}