How CubeEgress Performs Transparent HTTPS MITM Inspection in CubeSandbox

CubeEgress intercepts outbound HTTPS traffic using OpenResty's transparent proxy mode, dynamically generates per-SNI leaf certificates signed by an internal CA during the TLS handshake, and forwards requests to the original destination while enabling full payload inspection and policy enforcement.

The CubeEgress component of the TencentCloud/CubeSandbox repository implements a security proxy that performs transparent man-in-the-middle (MITM) inspection on HTTPS connections originating from sandboxed workloads. By combining Linux transparent proxying with dynamic certificate generation in Lua, CubeEgress can decrypt TLS traffic for security scanning without requiring client-side configuration changes.

Architecture Overview

CubeEgress operates as a high-performance L7 proxy built on OpenResty (NGINX with LuaJIT). The inspection pipeline follows a strict sequence from startup initialization through TLS interception to final request forwarding:

1. Entrypoint validation — start.sh verifies the internal CA and placeholder certificates
2. Transparent socket binding — NGINX listens on ports 8080 (HTTP) and 8443 (HTTPS) with the transparent option
3. TLS handshake interception — ssl_certificate_by_lua_block triggers dynamic certificate generation
4. Per-SNI leaf signing — cert_signer.lua generates ECDSA certificates signed by the internal CA
5. Certificate injection — The Lua ngx.ssl API replaces the placeholder cert with the generated leaf
6. Upstream forwarding — Original SNI is preserved via proxy_ssl_name while traffic flows to the real destination
7. Policy enforcement — access_phase.lua evaluates security rules and injects credentials

Startup Validation and Initialization

The entrypoint script at CubeEgress/start.sh performs critical setup before launching the proxy. It validates that the internal CA (cube-root-ca.crt and cube-root-ca.key) and a placeholder certificate exist, ensuring the certificate chain can be established.

# start.sh validates CA and placeholder before launching

[[ -f "${CA_CERT}" ]] || fatal "CA cert missing: ${CA_CERT}"
[[ -f "${PLACEHOLDER_CERT}" ]] || fatal "placeholder cert missing"
configure_listen_ip   # Rewrites nginx.conf with sandbox gateway IP

exec "${NGINX_BIN}" -g "daemon off;"

The script also rewrites the listen IP address in nginx.conf to match the sandbox gateway address, ensuring the proxy binds to the correct network interface for transparent interception.

Transparent Proxy Configuration

The NGINX configuration in CubeEgress/nginx.conf defines two server blocks that operate in transparent proxy mode. The HTTPS server listens on port 8443 with the transparent reuseport directives, which allows the socket to intercept traffic originally destined for external IPs.

server {
    listen 192.168.0.1:8443 ssl transparent reuseport;
    ssl_certificate     /etc/cube/ca/placeholder.crt;
    ssl_certificate_key /etc/cube/ca/placeholder.key;

    ssl_certificate_by_lua_block {
        local ssl = require("ngx.ssl")
        local sni = ssl.server_name()
        local raw_addr, addr_type = ssl.raw_server_addr()
        local dst_ip = raw_addr and addr_type == "inet" and #raw_addr == 4
                      and string.format("%d.%d.%d.%d",
                                        string.byte(raw_addr,1),
                                        string.byte(raw_addr,2),
                                        string.byte(raw_addr,3),
                                        string.byte(raw_addr,4))
        local ok, err = cert_signer.serve(sni, dst_ip)
        if not ok then
            ngx.log(ngx.ERR, "cert_signer failed: ", err)
            return ngx.exit(ngx.ERROR)
        end
    }
    
    proxy_ssl_name $cube_original_sni;
    proxy_pass https://$server_addr:$server_port;
}

The ssl_certificate_by_lua_block directive registers a Lua handler that executes during the TLS handshake, before the certificate is sent to the client. This hook receives the SNI (Server Name Indication) and destination IP address, then calls the certificate signer module.

Dynamic Certificate Generation

The heart of the MITM capability resides in CubeEgress/lua/cert_signer.lua, which manages the internal CA and generates leaf certificates on demand.

CA Bootstrap

During NGINX initialization (init_by_lua*), the bootstrap() function loads the CA certificate and private key into module-level variables:

function _M.bootstrap(opts)
    local cert_pem = read_file(opts.ca_cert_path)
    local key_pem  = read_file(opts.ca_key_path)
    CA_CERT = x509_lib.new(cert_pem, "PEM")
    CA_KEY  = pkey_lib.new(key_pem, { format = "PEM" })
    ngx.log(ngx.INFO, "cert_signer bootstrapped")
end

Certificate Serving and Caching

The serve(sni, dst_ip) function implements a cache-first strategy using a shared memory dictionary (cert_cache). When a cache miss occurs, it acquires a distributed lock via lua-resty-lock to prevent stampede conditions during high-concurrency scenarios, then invokes sign_leaf() to generate a new certificate.

function _M.serve(sni, dst_ip)
    local cache_key = sni or dst_ip
    local cached = cert_cache:get(cache_key)
    if cached then
        return apply_cert(cached)
    end
    
    local lock = resty_lock.new("cert_locks")
    lock:lock(cache_key)
    
    -- Double-check after acquiring lock
    cached = cert_cache:get(cache_key)
    if cached then
        lock:unlock()
        return apply_cert(cached)
    end
    
    local cert_der, key_der = sign_leaf(sni, dst_ip)
    cert_cache:set(cache_key, {cert = cert_der, key = key_der}, CACHE_TTL)
    lock:unlock()
    return apply_cert({cert = cert_der, key = key_der})
end

Leaf Certificate Signing

The sign_leaf() function generates an ECDSA P-256 key pair and constructs an X.509 certificate with appropriate extensions:

local function sign_leaf(sni, dst_ip)
    local leaf_key = pkey_lib.new({ type = "EC", curve = "prime256v1" })
    local cert = x509_lib.new()
    
    cert:set_version(3)
    cert:set_serial_number(bn_lib.new(math.floor(ngx.now()*1e6)))
    cert:set_not_before(ngx.time() - 60)
    cert:set_not_after(ngx.time() + LEAF_TTL_SEC)
    cert:set_subject_name(name_lib.new():add("CN", sni or "unknown"))
    cert:set_issuer_name(CA_CERT:get_subject_name())
    cert:set_pubkey(leaf_key)
    
    -- Add Subject Alternative Names
    local sans = altname_lib.new()
    if sni and not is_ip_literal(sni) then
        sans:add("DNS", sni)
    else
        sans:add("IP", dst_ip or sni)
    end
    cert:add_extension(ext_lib.new("subjectAltName", sans))
    
    -- Add standard extensions
    cert:add_extension(ext_lib.new("basicConstraints", "critical,CA:FALSE"))
    cert:add_extension(ext_lib.new("keyUsage", "critical,digitalSignature,keyEncipherment"))
    cert:add_extension(ext_lib.new("extendedKeyUsage", "serverAuth"))
    
    cert:sign(CA_KEY, digest_lib.new("sha256"))
    return cert:tostring("DER"), leaf_key:tostring("PrivateKey", "DER")
end

TLS Handshake Injection

After generating the certificate, the ssl_certificate_by_lua_block clears the placeholder certificate and installs the new leaf using the ngx.ssl API:

local ssl = require("ngx.ssl")
ssl.clear_certs()
ssl.set_der_cert(cert_der)
ssl.set_der_priv_key(key_der)

This replaces the certificate presented to the client with a valid, CA-signed leaf that matches the requested SNI. The client sees a trusted certificate chain (assuming the internal CA is installed as a system trust anchor), while CubeEgress maintains the ability to decrypt and inspect the traffic.

Request Processing and Audit

Once the TLS handshake completes, the request enters the access phase handled by CubeEgress/lua/access_phase.lua. This phase evaluates security policies, performs credential injection, and prepares audit logs.

function _M.decide()
    local ctx = {
        sni    = ngx.var.ssl_server_name,
        host   = host_without_port(ngx.var.http_host),
        scheme = ngx.var.scheme,
        method = ngx.req.get_method(),
        path   = ngx.var.uri,
    }
    
    local decision = policy.lookup(ctx)
    if not decision.allow then
        return ngx.exit(ngx.HTTP_FORBIDDEN)
    end
    
    if decision.inject then
        inject_headers(decision.inject)
    end
    
    ngx.ctx.cube_decision = decision
end

The policy.lookup() function (defined in CubeEgress/lua/policy.lua) matches the request against configured security rules, while CubeEgress/lua/audit.lua records the transaction details for compliance and forensics.

Key Implementation Files

File	Purpose	Location
start.sh	Entrypoint that validates CA, configures listen IP, launches OpenResty	[CubeEgress/start.sh](https://github.com/TencentCloud/CubeSandbox/blob/master/CubeEgress/start.sh)
nginx.conf	OpenResty configuration with transparent listeners and TLS hooks	[CubeEgress/nginx.conf](https://github.com/TencentCloud/CubeSandbox/blob/master/CubeEgress/nginx.conf)
cert_signer.lua	Dynamic certificate generation, CA bootstrap, and caching logic	[CubeEgress/lua/cert_signer.lua](https://github.com/TencentCloud/CubeSandbox/blob/master/CubeEgress/lua/cert_signer.lua)
access_phase.lua	Policy enforcement, credential injection, and request context setup	[CubeEgress/lua/access_phase.lua](https://github.com/TencentCloud/CubeSandbox/blob/master/CubeEgress/lua/access_phase.lua)
policy.lua	Security policy DSL and rule matching engine	[CubeEgress/lua/policy.lua](https://github.com/TencentCloud/CubeSandbox/blob/master/CubeEgress/lua/policy.lua)
audit.lua	Audit logging and event recording	[CubeEgress/lua/audit.lua](https://github.com/TencentCloud/CubeSandbox/blob/master/CubeEgress/lua/audit.lua)
cube-proxy-iptables-init.sh	iptables rules for traffic redirection to proxy ports	[CubeEgress/scripts/cube-proxy-iptables-init.sh](https://github.com/TencentCloud/CubeSandbox/blob/master/CubeEgress/scripts/cube-proxy-iptables-init.sh)

Summary

• CubeEgress implements transparent HTTPS MITM inspection using OpenResty's ssl_certificate_by_lua_block hook and Linux transparent proxying.
• The certificate signer (cert_signer.lua) caches generated leaf certificates in a shared memory dictionary to optimize performance and uses lua-resty-lock to prevent generation stampedes.
• ECDSA P-256 keys and SHA-256 signing provide modern cryptographic standards for dynamically generated certificates.
• The placeholder certificate in nginx.conf is required for configuration parsing but is replaced during each TLS handshake via the ngx.ssl API.
• Original SNI is preserved through proxy_ssl_name, ensuring the upstream connection presents the correct certificate while the sandboxed client sees the intercepted leaf.

Frequently Asked Questions

What is transparent proxy mode in CubeEgress?

Transparent proxy mode allows CubeEgress to intercept outbound TCP connections without requiring IP address or port changes on the client. By using the transparent option in NGINX's listen directive and iptables redirection rules (configured in cube-proxy-iptables-init.sh), the proxy receives packets destined for external IPs while maintaining the original destination address for upstream forwarding.

How does CubeEgress generate certificates on-the-fly?

During each TLS handshake, the ssl_certificate_by_lua_block retrieves the SNI and calls cert_signer.serve(). If no cached certificate exists for that SNI, the module generates a new ECDSA P-256 private key, constructs an X.509 certificate with appropriate extensions (Subject Alternative Name, Key Usage, Extended Key Usage), and signs it with the internal CA private key using SHA-256. The resulting certificate is cached and injected into the handshake.

Why does CubeEgress require a placeholder certificate?

NGINX requires valid certificate and key files at configuration load time to start the SSL server. CubeEgress uses a pre-generated placeholder certificate solely to satisfy this requirement. Immediately after NGINX starts, the ssl_certificate_by_lua_block replaces this placeholder with dynamically generated, SNI-specific certificates for each incoming connection.

How does CubeEgress prevent certificate generation stampedes?

The certificate signer uses a two-phase locking strategy. When a cache miss occurs, the requesting worker acquires a lock via lua-resty-lock on the SNI key. After acquiring the lock, it checks the cache again (double-checked locking pattern) before generating the certificate. This ensures that only one worker generates the certificate for a given SNI during high-concurrency scenarios, while others wait and use the cached result.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}