VNC Password Cracking with Patator: Protocol-Specific Considerations

Patator implements VNC brute-forcing through the VNC_login class by forcing classic VNC authentication (security type 0x02) and performing DES encryption with a bit-reversed key derivation routine, handling RFB protocol versions 3.3 through 3.8 with specific constraints on password length and challenge-response validation.

The lanjelot/patator framework provides specialized support for VNC password auditing through its vnc_login module. Understanding the protocol-specific implementation details is essential for effective penetration testing against Remote Framebuffer (RFB) services. This analysis examines how Patator handles the VNC handshake, authentication mechanisms, and cryptographic requirements defined in the RFB specification.

VNC Protocol Implementation in Patator

Located in src/patator/patator.py, the VNC_login class (around line 3690) encapsulates the complete VNC authentication workflow. The implementation divides the attack into two distinct stages: connection establishment handled by VNC.connect and cryptographic authentication performed by VNC.login.

This separation allows Patator to manage protocol-level errors distinctly from authentication failures, enabling precise retry logic through the framework's -x ignore/retry flags.

RFB Handshake and Version Negotiation

The VNC.connect method (lines 3578-3590) initiates the TCP socket and validates the server banner. Patator reads the 11-byte RFB protocol version string and raises a VNC_Error if malformed data follows the banner, ensuring protocol integrity before authentication begins.

During negotiation in VNC.login (lines 3594-3602), Patator implements intelligent version selection:

• 003.008 (preferred)
• 003.007 (fallback)
• 003.003 (default for legacy servers)

The client selects the highest compatible version based on the server's advertised major and minor fields. This matters because different versions expose different security type negotiation mechanisms.

Authentication Flow and DES Encryption

Security Type Enforcement

For RFB versions 3.7 and above, Patator explicitly forces security type 0x02 (classic VNC authentication) regardless of the server's advertised supported types (lines 3612-3619). The code iterates through the server's security type list but deliberately selects 0x02 to ensure compatibility with the DES-based challenge-response mechanism.

Challenge-Response Mechanism

The server transmits a 16-byte random challenge, which Patator validates for exact length compliance (lines 3625-3630). The response generation involves three critical steps:

1. Password normalization: Truncating or padding passwords to exactly 8 bytes with null characters (\x00)
2. Key derivation: Transforming the password via the gen_key function (lines 3631-3666), which implements the VNC-specific bit-reversal algorithm required for proper DES key generation
3. Encryption: Using pycryptodome's DES class in ECB mode to encrypt the challenge (lines 3668-3674)

Result Interpretation

Patator interprets the server's 4-byte status response where code 0 indicates successful authentication and code 1 signals failure (lines 3676-3685). The method returns a normalized (code, mesg) tuple, allowing Patator's engine to distinguish between genuine authentication failures and connection anomalies for its retry logic.

Critical Protocol Constraints

Understanding these limitations ensures accurate vulnerability assessment:

• RFB Version Compatibility: Automatic selection between protocol versions 3.3, 3.7, and 3.8 based on server capabilities
• Security Type Limitations: Only supports classic VNC authentication (type 0x02), ignoring Tight, VeNCrypt, or other extended authentication methods
• Password Length Restrictions: Hard 8-byte limit with silent truncation—longer passwords are automatically shortened before key generation
• Cryptographic Specificity: Requires the non-standard bit-reversal key generation (gen_key) rather than using the raw password as a DES key
• Error Handling: VNC_Error exceptions for protocol anomalies help distinguish connection issues from authentication failures when using retry flags

Practical Usage Examples

Basic single-password verification against a target host:

patator vnc_login host=192.168.1.10 password=secret

Dictionary attack with optimized retry logic and early termination on success:

patator vnc_login host=192.168.1.10 password=FILE0 0=passwords.txt \
    -t 1 \
    -x 'retry:fgrep!=Authentication failure' \
    --max-retries -1 \
    -x quit:code=0

Command explanation:

• password=FILE0 reads passwords from the file specified by 0=passwords.txt
• -t 1 restricts execution to one thread (VNC servers often throttle parallel connections)
• -x 'retry:fgrep!=Authentication failure' retries on any response except explicit authentication failure strings
• --max-retries -1 removes retry limits for large dictionaries
• -x quit:code=0 terminates the run immediately upon successful authentication (status code 0)

Summary

• Patator forces classic VNC authentication (type 0x02) across all supported RFB versions 3.3 through 3.8
• The gen_key function in src/patator/patator.py implements mandatory bit-reversal for proper DES key generation per the VNC specification
• Passwords are strictly limited to 8 bytes, padded with null characters (\x00) before cryptographic processing
• Protocol version negotiation automatically selects the highest compatible RFB version based on the server banner
• The tool uses pycryptodome for ECB-mode DES encryption of the 16-byte server challenge received during authentication

Frequently Asked Questions

What RFB protocol versions does Patator support?

Patator supports RFB versions 003.003, 003.007, and 003.008. The implementation in VNC.login automatically negotiates the highest available version based on the server banner, defaulting to 003.003 for older implementations that do not support the version negotiation mechanism introduced in 003.007.

Why does Patator only support classic VNC authentication?

Patator explicitly forces security type 0x02 (classic VNC) during the handshake to ensure compatibility with its DES-based challenge-response implementation. Other security types like VeNCrypt, Tight authentication, or Apple Remote Desktop extensions require different cryptographic handling and authentication flows not implemented in the current VNC_login class.

How does Patator handle passwords longer than 8 characters?

The VNC protocol specification limits passwords to 8 bytes. Patator truncates longer passwords and pads shorter ones with null characters (\x00) before applying the gen_key bit-reversal transformation. This means passwords exceeding 8 characters are silently truncated, and only the first 8 bytes participate in the DES key derivation.

What cryptographic library does Patator use for VNC authentication?

Patator relies on pycryptodome (listed in requirements.txt) to perform DES encryption in ECB mode. The tool implements a custom gen_key function (lines 3631-3666) to handle the VNC-specific bit-reversal required for proper key derivation before the DES operation, as the protocol requires reversing the bit order within each byte of the password.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}