Architecture of Hysteria Core: How the QUIC-Based Proxy Stack Works

The architecture of Hysteria core is a modular, high-performance networking stack built on QUIC and HTTP/3 that layers configuration management, custom authentication, pluggable congestion control, and TCP/UDP proxy logic into a thin, extensible core located in the core/ package.

The apernet/hysteria repository implements Hysteria 2 as a production-grade proxy focused on maximizing throughput over restrictive networks. The architecture of Hysteria core revolves around a thin glue layer that binds together QUIC transport, HTTP/3 signaling, and optional UDP forwarding while remaining agnostic to authentication and outbound routing implementations.

Architectural Layers of Hysteria Core

The core divides responsibilities into seven distinct layers, each implemented as a focused package or module within core/.

Configuration Management

At the base lies configuration, defined in core/server/config.go and core/client/config.go. The Config struct aggregates user-provided settings including TLS certificates, QUIC parameters, and connection handles. The fill() function validates inputs and injects sensible defaults for stream windows, idle timeouts, and MTU discovery settings before the server or client initializes.

Transport Layer (QUIC and HTTP/3)

The transport layer wraps raw net.PacketConn instances into QUIC-capable endpoints. In core/server/server.go, the NewServer function constructs a quic.Transport and binds an http3.Server, while core/client/client.go provides NewClient to build the client-side HTTP/3 transport. Both implementations enable QUIC datagrams (EnableDatagrams: true) to support UDP forwarding over a single connection.

Authentication Handshake

Hysteria uses a custom authentication protocol tunneled over HTTP/3 POST requests. The server-side handler h3sHandler.ServeHTTP in core/server/server.go intercepts requests to protocol.URLHost/URLPath, extracts the AuthRequest from HTTP headers, and validates credentials via the Authenticator interface. The client-side clientImpl.connect in core/client/client.go initiates this handshake and parses the AuthResponse, which signals whether UDP is enabled and advertises the server's receive bandwidth limit.

Congestion Control

After authentication, the congestion control layer selects between BBR, Cubic, or a "brutal" bandwidth limiter based on handshake results and configuration. Both server and client invoke congestion.UseConfigured or congestion.UseBrutal after the initial exchange (visible in h3sHandler lines 61-78 and clientImpl.connect lines 42-55) to tune the QUIC connection's pacing and window scaling.

Proxy Logic (TCP and UDP)

The proxy logic layer handles bidirectional data forwarding. For TCP, streams are wrapped in utils.QStream (defined in core/internal/utils/qstream.go) to ensure proper Close semantics, then forwarded via the handleTCPRequest method in core/server/server.go to the target address using the configured Outbound implementation. For UDP, the udpSessionManager multiplexes datagrams over the single QUIC connection, reassembling fragmented packets for delivery to remote hosts via the configured Outbound.

Extensibility Hooks

The architecture exposes interfaces for extensibility defined in core/server/config.go: Authenticator, RequestHook, Outbound, TrafficLogger, and EventLogger. These hooks allow operators to inject custom credential validators, inspect or modify requests before proxying, redirect traffic through upstream proxies, and emit structured telemetry without modifying core internals.

Utilities

Supporting these layers are utility packages including core/internal/protocol for binary frame encoding/decoding and core/internal/utils for atomic counters and stream wrappers.

Data Flow Through the Core

Understanding how these layers interact clarifies the request lifecycle from initialization to shutdown.

1. Initialization: The user constructs a server.Config or client.Config, then invokes NewServer or NewClient. Config.fill() sanitizes inputs and applies defaults.

2. Transport Setup: The system converts user TLS settings to a standard tls.Config via convertToStdTLSConfig, then initializes the QUIC transport on the supplied UDP socket.

3. Handshake: The client sends an HTTP/3 POST containing authentication credentials. The server validates via Authenticator.Authenticate and returns an AuthResponse with UDP capability flags and bandwidth limits.

4. Flow Control Configuration: Based on AuthResponse.RxAuto and local policy, the connection applies either BBR/Cubic via congestion.UseConfigured or the rate-limited congestion.UseBrutal.

5. Proxy Operations:

   • TCP: Client calls TCP(), opening a QUIC stream and writing a TCPRequest frame. The server's ProxyStreamHijacker delegates to handleTCPRequest, which dials the destination via the Outbound interface.
   • UDP: If enabled, both sides instantiate udpSessionManager, which routes datagrams between the QUIC connection and local/remote UDP sockets using the Outbound implementation.

6. Lifecycle Events: Throughout, TrafficLogger records per-stream statistics, while EventLogger captures connect and disconnect callbacks.

7. Shutdown: Calling Server.Close gracefully terminates the QUIC listener, underlying transport, and executes any Config.Cleanup functions.

Implementation Examples

Starting a Minimal Server

import (
    "net"
    "crypto/tls"
    "log"
    "github.com/apernet/hysteria/core/v2/server"
)

func main() {
    // Create the underlying UDP socket
    udpConn, err := net.ListenPacket("udp", ":443")
    if err != nil {
        log.Fatal(err)
    }
    
    // Configure with mandatory TLS and authenticator
    cfg := &server.Config{
        TLSConfig: server.TLSConfig{
            Certificates: []tls.Certificate{loadCert()},
        },
        Conn:          udpConn,
        Authenticator: myAuth, // implements server.Authenticator
    }
    
    // Initialize and serve
    s, err := server.NewServer(cfg)
    if err != nil {
        log.Fatal(err)
    }
    s.Serve()
}

See the Config definition in [core/server/config.go](https://github.com/apernet/hysteria/blob/master/core/server/config.go#L27-L44).

Creating a Client

import (
    "net"
    "fmt"
    "log"
    "github.com/apernet/hysteria/core/v2/client"
)

func main() {
    cfg := &client.Config{
        ServerAddr: &net.UDPAddr{
            IP:   net.ParseIP("1.2.3.4"),
            Port: 443,
        },
        TLSConfig: client.TLSConfig{
            InsecureSkipVerify: true,
        },
    }
    
    c, info, err := client.NewClient(cfg)
    if err != nil {
        log.Fatal(err)
    }
    
    fmt.Printf("UDP enabled: %v, bandwidth limit: %d\n", 
        info.UDPEnabled, info.Tx)
    // Use c.TCP("example.com:80") or c.UDP()
}

The client entry point is client.NewClient in [core/client/client.go](https://github.com/apernet/hysteria/blob/master/core/client/client.go#L44-L66).

Implementing a Custom Outbound

type socksOutbound struct{}

func (s *socksOutbound) TCP(addr string) (net.Conn, error) {
    return net.Dial("tcp", "socks5-proxy:1080")
}

func (s *socksOutbound) UDP(addr string) (client.UDPConn, error) {
    // Implement SOCKS5 UDP association
    return client.NewSocks5UDPConn("socks5-proxy:1080")
}

// Register when building the server
cfg.Outbound = &socksOutbound{}

The Outbound interface is defined in [core/server/config.go](https://github.com/apernet/hysteria/blob/master/core/server/config.go#L52-L60).

Key Source Files

File	Responsibility
core/server/config.go	Server Config struct, validation, and extensibility interfaces
core/server/server.go	NewServer, QUIC listener, HTTP/3 handler dispatch
core/server/udp.go	udpSessionManager for datagram multiplexing
core/client/config.go	Client configuration and verifyAndFill
core/client/client.go	NewClient, handshake logic, public API
core/internal/protocol/*	Binary protocol structs and encoding
core/internal/congestion/*	BBR, Cubic, and brutal congestion adapters
core/internal/utils/qstream.go	QStream wrapper for QUIC streams
core/internal/pmtud/*	Path-MTU discovery helpers

Summary

• The architecture of Hysteria core organizes functionality into seven layers: Configuration, Transport, Authentication, Congestion Control, Proxy Logic, Extensibility, and Utilities.
• The stack builds on QUIC and HTTP/3, using HTTP/3 POST requests for the initial authentication handshake defined in protocol.URLHost/URLPath.
• TCP proxying uses wrapped QUIC streams (utils.QStream) while UDP proxying relies on udpSessionManager to multiplex datagrams.
• Pluggable interfaces (Authenticator, Outbound, TrafficLogger) allow customization without core modifications.
• Congestion control supports BBR, Cubic, and brutal rate limiting, selected dynamically based on handshake results.

Frequently Asked Questions

What transport protocols does Hysteria core use?

Hysteria core is built exclusively on QUIC over UDP, leveraging HTTP/3 for control-plane signaling including authentication. The transport layer wraps standard net.PacketConn interfaces into QUIC connections with datagram support enabled, allowing both reliable stream-based TCP proxying and unreliable UDP forwarding over the same connection.

How does the authentication handshake work in Hysteria?

The client sends an HTTP/3 POST request to a specific path (protocol.URLHost/URLPath) containing credentials in the headers. The server's h3sHandler.ServeHTTP validates these against the configured Authenticator interface and returns an AuthResponse containing UDP capability flags and bandwidth limits. This exchange occurs before any proxy traffic flows.

Which congestion control algorithms are available?

The core supports BBR, Cubic, and a custom "brutal" bandwidth limiter. Selection occurs post-handshake via congestion.UseConfigured (for standard algorithms) or congestion.UseBrutal (for hard rate limiting), depending on the server's advertised RxAuto setting and local configuration.

How does Hysteria core handle UDP traffic?

UDP traffic is managed by the udpSessionManager found in core/server/udp.go and core/client/udp.go. This component receives QUIC datagrams, reassembles them into UDP packets, and forwards them through the configured Outbound implementation. The feature is only enabled if both sides signal support during the authentication handshake.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}