# How Cube-Agent Forwards I/O Streams in TencentCloud CubeSandbox

> Learn how Cube-agent forwards I/O streams in CubeSandbox using ttrpc, vsock channels, and async copiers for seamless host-guest communication.

- Repository: [Tencent Cloud/CubeSandbox](https://github.com/TencentCloud/CubeSandbox)
- Tags: internals
- Published: 2026-07-05

---

**Cube-agent forwards I/O streams by exposing a ttrpc service over a VM-side vsock channel, implementing RPC handlers that read from and write to container pseudo-terminals and pipes, and using an async copier to bridge low-level vsock traffic between the host shim and guest processes.**

Cube-agent runs as PID 1 inside each MicroVM in the TencentCloud/CubeSandbox repository. Acting as the guest-side agent, it receives standard I/O (stdin, stdout, stderr) forwarded from the host-side `containerd-shim-cube-rs` over virtio-vsock, routing data to the appropriate container processes stored in the sandbox state.

## Architecture Overview

The forwarding architecture relies on a **vsock** channel between the host shim and the guest agent. The host opens a connection to the guest’s vsock listener (`VsockListener` in [`agent/src/util.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/util.rs)), which the agent wraps with a **ttrpc** server (`AgentService` in [`agent/src/rpc.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/rpc.rs)).

The flow follows this path:

```

Host (shim) ──► vsock (ttrpc) ──► cube-agent (inside VM)
                                 │
          ┌──────────────────────┼───────────────────────┐
          │                      │                       │
          ▼                      ▼                       ▼
   WriteStream RPC      ReadStdout RPC          ReadStderr RPC
   (container stdin)    (container stdout)      (container stderr)

```

All stream objects reside inside the `Process` struct (defined in [`rustjail/src/process.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/rustjail/src/process.rs)) and are wrapped in `Arc<Mutex<_>>` to allow concurrent RPC calls to safely share the underlying file descriptors.

## RPC-Based Stream Forwarding

The `AgentService` implementation in [`agent/src/rpc.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/rpc.rs) exposes three primary RPC handlers for I/O operations. These methods map incoming ttrpc requests to the appropriate file descriptors or PTY masters belonging to the target container process.

### Writing to Standard Input (do_write_stream)

When the shim sends data to a container’s stdin, it invokes the `write_stdin` RPC, handled by `AgentService::do_write_stream` (lines ≈ 616‑633 in [`agent/src/rpc.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/rpc.rs)).

The handler performs the following steps:

1. Looks up the target process using `sandbox.find_container_process`.
2. Selects the appropriate writer:
   - If a PTY exists (`term_master`), it uses the PTY master file descriptor.
   - Otherwise, it falls back to the parent stdin pipe.
3. Writes the payload asynchronously: `writer.lock().await.write_all(req.data.as_slice()).await`.

### Reading from Standard Output and Standard Error (do_read_stream)

For outbound data, the shim issues `read_stdout` or `read_stderr` RPCs, both handled by `AgentService::do_read_stream` (lines ≈ 644‑688 in [`agent/src/rpc.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/rpc.rs)).

The logic mirrors the write path:

1. Locates the process via the sandbox.
2. Selects the reader:
   - **PTY mode**: Returns the PTY master stream.
   - **Pipe mode**: Returns the parent stdout or parent stderr pipe.
3. Invokes `read_stream(reader, req.len as usize)` to read exactly `len` bytes from the async reader.
4. Returns the bytes in a `ReadStreamResponse` to the host.

### Terminal Resize Operations (tty_win_resize)

When containers run with a pseudo-terminal, the host can resize the terminal window via the `tty_win_resize` RPC (lines ≈ 1550‑1670 in [`agent/src/rpc.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/rpc.rs)). This handler performs an `ioctl` on the PTY master file descriptor to adjust the window dimensions without interrupting the I/O stream.

## Low-Level I/O Bridging with interruptable_io_copier

Beyond the ttrpc service, cube-agent handles raw vsock traffic for dedicated channels (such as metric exporting) using the `interruptable_io_copier` utility in [`agent/src/util.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/util.rs) (lines ≈ 21‑57).

This async helper function performs a read-write loop between any `AsyncRead` source and `AsyncWrite` destination. It accepts a `watch::Receiver<bool>` shutdown signal, allowing the agent to abort the copy cleanly when the VM terminates.

From [`agent/src/main.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/main.rs) (line ≈ 136), the agent initializes the copier at startup:

```rust
let _ = util::interruptable_io_copier(&mut reader, &mut writer, shutdown).await;

```

Here, `reader` and `writer` represent the vsock stream ends, creating a bridge between the low-level vsock pipe and the higher-level ttrpc service while respecting lifecycle signals.

## End-to-End I/O Flow

The complete forwarding sequence operates as follows:

1. **Sandbox Creation** – The shim calls `create_sandbox`, and the agent initializes namespaces, mounts, and the `Sandbox` state.
2. **Process Launch** – During `create_container`, the agent constructs a `Process` (via `rustjail`), storing stdio pipes or the PTY master in the sandbox.
3. **Stdin Write** – The shim sends a `WriteStreamRequest` payload; `do_write_stream` writes bytes into the container’s stdin pipe or PTY master.
4. **Stdout/Stderr Read** – The shim issues `ReadStreamRequest`; `do_read_stream` pulls the requested byte count from the appropriate reader and returns a `ReadStreamResponse`.
5. **Resizing** – If using a PTY, the shim invokes `tty_win_resize`, which performs an `ioctl` on the PTY master to update terminal geometry.

This design ensures the host never requires direct filesystem or socket access to the guest; cube-agent acts as the sole authority for byte stream forwarding across the vsock boundary.

## Summary

- Cube-agent exposes a **ttrpc** service over **vsock** to receive I/O commands from the host shim.
- **`do_write_stream`** (lines ≈ 616‑633 in [`agent/src/rpc.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/rpc.rs)) writes incoming data to container stdin or PTY masters.
- **`do_read_stream`** (lines ≈ 644‑688 in [`agent/src/rpc.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/rpc.rs)) reads requested byte counts from stdout/stderr pipes or PTY streams.
- **`interruptable_io_copier`** (lines ≈ 21‑57 in [`agent/src/util.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/util.rs)) bridges raw vsock traffic with abortable async copy loops.
- Stream objects are stored in the **`Process`** struct ([`rustjail/src/process.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/rustjail/src/process.rs)) and protected by `Arc<Mutex<_>>` for thread-safe concurrent access.

## Frequently Asked Questions

### What transport protocol does cube-agent use for I/O forwarding?

Cube-agent uses **ttrpc** (a lightweight RPC framework) over **virtio-vsock** channels. The host-side shim opens a vsock connection to the guest agent, and all I/O operations are encapsulated in ttrpc requests such as `write_stdin`, `read_stdout`, and `read_stderr`.

### How does cube-agent handle PTY I/O differently from pipe-based I/O?

When a container allocates a pseudo-terminal, cube-agent stores the `term_master` file descriptor in the `Process` struct. During I/O operations, `do_write_stream` and `do_read_stream` check for the presence of this PTY master first; if found, they use it as the reader or writer. If no PTY exists, they fall back to the standard `parent_stdin`, `parent_stdout`, or `parent_stderr` pipes.

### What mechanism allows cube-agent to shut down I/O streams cleanly when the VM terminates?

The **`interruptable_io_copier`** function in [`agent/src/util.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/src/util.rs) accepts a `watch::Receiver<bool>` shutdown signal. When the agent receives a termination signal, it triggers this watcher, causing the copier to abort its async read-write loop and close the vsock connection gracefully without dropping inflight data.

### Where are the container I/O streams stored in cube-agent's memory?

Stream objects reside inside the **`Process`** struct defined in [`rustjail/src/process.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/rustjail/src/process.rs). The agent stores `parent_stdin`, `parent_stdout`, `parent_stderr`, and optionally `term_master` as `Arc<Mutex<>>` wrapped types, allowing the `AgentService` RPC handlers to safely access these file descriptors concurrently from multiple async tasks.