# How ttrpc Differs from Standard gRPC in CubeSandbox: Architecture and Implementation

> Discover how CubeSandbox uses ttrpc for efficient internal communication and gRPC for external APIs. Learn their architectural differences for low-latency and streaming support.

- Repository: [Tencent Cloud/CubeSandbox](https://github.com/TencentCloud/CubeSandbox)
- Tags: architecture
- Published: 2026-07-05

---

**CubeSandbox uses ttrpc for lightweight, low-latency internal communication between the Agent and Shim via Unix sockets, while employing standard gRPC for the Network-Agent to support streaming, TLS, and external API consumption.**

The TencentCloud/CubeSandbox repository implements a dual-RPC architecture that strategically separates internal control-plane traffic from external service-plane exposure. By analyzing the source code in [`agent/src/rpc.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/rpc.rs) and [`network-agent/internal/grpcserver/server.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/network-agent/internal/grpcserver/server.go), we can see how each protocol serves distinct performance and functional requirements within the sandboxed container environment.

## What Is ttrpc and Why CubeSandbox Uses It

**ttrpc** (Tiny gRPC) is a minimal remote procedure call protocol designed specifically for intra-host communication where overhead must be minimized. In CubeSandbox, the Agent component—written in Rust—uses ttrpc to handle latency-critical operations without the baggage of HTTP/2.

### Transport and Protocol Design

Unlike standard gRPC, ttrpc operates exclusively over **Unix-domain sockets** and does not use HTTP/2. In [`agent/src/rpc.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/rpc.rs), the Agent imports the Rust ttrpc crate at line 43 and binds to Unix sockets directly:

```rust
use ttrpc::r#async::{Server, TtrpcContext};
use ttrpc::Code;

```

This design eliminates HTTP/2 framing overhead and connection management complexity, making it ideal for the sandboxed Agent that must remain small and fast. The protocol encodes Protobuf messages directly into binary frames without the additional wrapping layers required by HTTP/2.

### Unary-Only Communication Model

ttrpc in CubeSandbox supports only **unary calls**—each request receives exactly one response. The `AgentService` implementation in the Agent source handles single-request-single-response patterns for operations like container lifecycle management. This constraint keeps the implementation lightweight but means streaming workflows must use alternative mechanisms.

## Standard gRPC Implementation in CubeSandbox

The **Network-Agent** component utilizes standard gRPC to expose APIs to external callers and the Cubelet. Written in Go, this implementation leverages the full feature set of `google.golang.org/grpc`.

### HTTP/2 and Streaming Capabilities

Standard gRPC in CubeSandbox operates over TCP or Unix sockets with full HTTP/2 support, enabling **client-, server-, and bidirectional streaming**. This is critical for the Network-Agent's `EnsureNetwork` and `ReleaseNetwork` operations, which may need to stream status updates or accept continuous configuration streams.

### Network-Agent Service Architecture

In [`network-agent/internal/grpcserver/server.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/network-agent/internal/grpcserver/server.go) (lines 72-78), the server initializes using the standard gRPC constructor:

```go
grpcSrv := grpc.NewServer()
pb.RegisterNetworkAgentServer(grpcSrv, &server{})

```

The generated service definitions in [`network-agent/api/v1/network_agent_grpc.pb.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/network-agent/api/v1/network_agent_grpc.pb.go) (lines 174-185) provide the complete client and server stubs, supporting the rich metadata and interceptor patterns typical of enterprise gRPC deployments.

## Key Differences Between ttrpc and gRPC in CubeSandbox

| Feature | **ttrpc** (Agent) | **gRPC** (Network-Agent) |
|---------|------------------|--------------------------|
| **Transport** | Unix-domain socket only | TCP/Unix socket with HTTP/2 |
| **Message Format** | Protobuf binary frames | Protobuf wrapped in HTTP/2 frames |
| **Streaming** | Unary calls only | Full client/server/bidirectional streaming |
| **TLS/Authentication** | Not built-in; relies on host socket permissions | Optional TLS and gRPC metadata auth |
| **Implementation** | Rust (`ttrpc = "0.x"`) | Go (`google.golang.org/grpc`) |
| **Error Handling** | `ttrpc::Code` enum | `grpc/codes` with rich status details |

### Dependency Footprint and Performance

The ttrpc implementation carries minimal runtime overhead, using the lightweight `ttrpc` Rust crate ideal for the resource-constrained Agent. Conversely, the Network-Agent accepts the heavier dependency footprint of Go's gRPC implementation to gain access to connection pooling, health checking, and comprehensive middleware support.

### Error Handling Semantics

Error handling differs significantly between the two protocols. The Agent wraps errors using a custom macro in [`agent/src/rpc.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/rpc.rs) (lines 128-136):

```rust
macro_rules! ttrpc_error {
    ($code:expr, $msg:expr) => {
        Err(ttrpc::Error::Others(format!("{}: {}", $code, $msg)))
    };
}

```

This generates `ttrpc::Code` errors similar to gRPC codes but without the detailed status metadata and rich error semantics available in `grpc/status`.

## Implementation Details and Code Examples

### ttrpc Server Implementation (Agent)

The Agent implements the `AgentService` trait using ttrpc's async runtime. In [`agent/src/rpc.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/rpc.rs), the service processes requests through `TtrpcContext` extraction (also referenced in [`agent/src/tracer.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/tracer.rs) for telemetry propagation):

```rust
use ttrpc::r#async::{Server, TtrpcContext};
use crate::protocols::agent_ttrpc::{AgentService, Empty};

#[derive(Default)]
struct MyAgent;

#[ttrpc::async_trait]
impl AgentService for MyAgent {
    async fn ping(&self, _: TtrpcContext, _: Empty) -> ttrpc::Result<Empty> {
        Ok(Empty {})
    }
}

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let server = Server::new()
        .bind("unix:///tmp/cube/agent-ttrpc.sock")?
        .register_service(AgentServiceServer::new(MyAgent::default()));
    server.start().await?;
    Ok(())
}

```

This example demonstrates the Unix socket binding and unary-only service pattern used throughout the CubeSandbox Agent.

### gRPC Server Implementation (Network-Agent)

The Network-Agent follows standard Go gRPC patterns, as seen in [`network-agent/internal/grpcserver/server.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/network-agent/internal/grpcserver/server.go):

```go
package main

import (
    "net"
    "google.golang.org/grpc"
    pb "github.com/TencentCloud/CubeSandbox/network-agent/api/v1"
)

type server struct {
    pb.UnimplementedNetworkAgentServer
}

func (s *server) Ping(ctx context.Context, req *pb.Empty) (*pb.Empty, error) {
    return &pb.Empty{}, nil
}

func main() {
    lis, _ := net.Listen("unix", "/tmp/cube/network-agent-grpc.sock")
    grpcSrv := grpc.NewServer()
    pb.RegisterNetworkAgentServer(grpcSrv, &server{})
    grpcSrv.Serve(lis)
}

```

The generated bindings in [`network-agent/api/v1/network_agent_grpc.pb.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/network-agent/api/v1/network_agent_grpc.pb.go) provide the interface definitions that support streaming methods and rich metadata handling.

## Summary

- **CubeSandbox employs ttrpc** for the Agent component to minimize latency and resource usage through Unix-domain sockets and unary-only communication.
- **Standard gRPC powers the Network-Agent**, providing necessary features like bidirectional streaming, TLS support, and comprehensive error handling for external API consumers.
- **Implementation differs by language**: Rust uses the `ttrpc` crate while Go uses `google.golang.org/grpc`, reflecting the distinct requirements of each component.
- **Error handling varies** between the lightweight `ttrpc::Code` enum and the richer gRPC status codes with metadata support.
- **Key source files** include [`agent/src/rpc.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/rpc.rs) for ttrpc logic and [`network-agent/internal/grpcserver/server.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/network-agent/internal/grpcserver/server.go) for gRPC server initialization.

## Frequently Asked Questions

### Why does CubeSandbox use ttrpc instead of gRPC for the Agent?

CubeSandbox selects ttrpc for the Agent because it requires minimal overhead and maximum performance for intra-host communication. The Agent runs in a constrained sandbox environment where the HTTP/2 stack and connection management of standard gRPC would consume unnecessary resources. According to the source code in [`agent/src/rpc.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/rpc.rs), the Rust-based ttrpc implementation provides the necessary Protobuf serialization without the latency penalty of HTTP/2 framing.

### Can ttrpc handle bidirectional streaming like gRPC?

No, ttrpc in CubeSandbox supports only unary calls where each request receives a single response. This limitation is intentional to keep the protocol lightweight and is evident in the Agent's service implementations in [`agent/src/rpc.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/rpc.rs). For streaming operations—such as log follow or event streaming—CubeSandbox relies on the Network-Agent's gRPC implementation, which supports full client-, server-, and bidirectional streaming capabilities.

### How does error handling differ between ttrpc and gRPC in CubeSandbox?

Error handling in ttrpc uses the `ttrpc::Code` enum, which provides basic error categorization similar to gRPC codes but without rich status details. The Agent implements error wrapping through a custom macro in [`agent/src/rpc.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/rpc.rs) (lines 128-136). In contrast, the Network-Agent's gRPC implementation uses `grpc/codes` and `grpc/status` packages, offering detailed error messages, metadata propagation, and structured status details that are essential for external API error reporting.

### Is ttrpc specific to CubeSandbox or a standard protocol?

ttrpc is an open standard maintained by the containerd project, not specific to CubeSandbox. CubeSandbox adopted ttrpc from the containerd ecosystem to align with industry standards for low-overhead container runtime communication. The repository references this in [`docs/changelog/v0.3.0.md`](https://github.com/TencentCloud/CubeSandbox/blob/main/docs/changelog/v0.3.0.md), which documents bug fixes involving "ttrpc errors" causing state drift, demonstrating that CubeSandbox relies on the upstream ttrpc specification rather than a custom implementation.