# CubeSandbox Snapshot Health Check Notifications and App Snapshot Annotations: A Technical Guide

> Learn about CubeSandbox snapshot health check notifications and app snapshot annotations. Understand how CubeMaster and Cubelet manage snapshot workflows for your applications.

- Repository: [Tencent Cloud/CubeSandbox](https://github.com/TencentCloud/CubeSandbox)
- Tags: technical-guide
- Published: 2026-07-15

---

**CubeSandbox uses HTTP notification endpoints in CubeMaster to track snapshot health status updates and relies on specific Kubernetes-style annotations in Cubelet to control application snapshot creation workflows.**

CubeSandbox is an open-source sandbox platform developed by Tencent Cloud that manages containerized workloads with snapshot capabilities. Understanding how snapshot health check notifications propagate through the system and how app snapshot annotations configure creation workflows is essential for operators managing persistent container states. This guide examines the source code implementation in the TencentCloud/CubeSandbox repository to explain these mechanisms.

## Snapshot Health Check Notifications in CubeMaster

The CubeMaster component exposes an HTTP notification service that receives health status updates from sandbox nodes. These notifications persist snapshot state in the database and trigger timeout handling when snapshot operations exceed expected durations.

### Notification Endpoints and Payloads

The notification handlers are registered in [`CubeMaster/pkg/service/httpservice/notify/notify.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeMaster/pkg/service/httpservice/notify/notify.go). The service exposes three distinct endpoints under the `/notify` path:

- **`/notify/health`**: A simple liveness probe that returns HTTP 200 with body "OK"
- **`/notify/snapnotify`**: Receives standard health status updates for snapshots
- **`/notify/snapnotifytimeout`**: Handles timeout notifications when snapshot creation exceeds time limits

The handler function `HttpHandler` routes requests based on the URL path and parses JSON payloads using `util.DecodeJSONBody`.

For standard health updates, the endpoint expects a JSON payload with `snapshot_id` and `status` fields:

```go
var payload struct {
    SnapshotID string `json:"snapshot_id"`
    Status     string `json:"status"`
}

```

Upon successful decoding, the handler invokes `base.UpdateSnapshotHealth(payload.SnapshotID, payload.Status)` to persist the state.

For timeout scenarios, the payload structure includes a boolean `timeout` flag:

```go
var payload struct {
    SnapshotID string `json:"snapshot_id"`
    Timeout    bool   `json:"timeout"`
}

```

This triggers `base.UpdateSnapshotTimeout(payload.SnapshotID, payload.Timeout)`, marking the snapshot operation as timed out in the database.

### Database Persistence and Health Status

The health status is stored in the `snapshot_runtime_refs` table, defined in [`CubeMaster/pkg/base/db/models/snapshot_runtime_ref.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeMaster/pkg/base/db/models/snapshot_runtime_ref.go). The `SnapshotRuntimeRef` struct includes a `HealthStatus` field that tracks the current state of the snapshot:

```go
type SnapshotRuntimeRef struct {
    ID            string    `gorm:"primary_key;column:id" json:"id"`
    SnapshotID    string    `gorm:"column:snapshot_id" json:"snapshot_id"`
    NodeID        string    `gorm:"column:node_id" json:"node_id"`
    NodeIP        string    `gorm:"column:node_ip" json:"node_ip"`
    AttachedAt    time.Time `gorm:"column:attached_at" json:"attached_at"`
    HealthStatus  string    `gorm:"column:health_status" json:"health_status"`
}

```

The model provides an `UpdateHealthStatus` method that wraps the GORM `Save` operation, ensuring atomic updates to the health status column when notifications arrive from the Cubelet.

## App Snapshot Annotations in Cubelet

Cubelet processes application snapshot creation requests by inspecting pod annotations. These annotations act as declarative configuration flags that determine whether to create a snapshot, which template to use, and which resources to include.

### Required Annotations for Snapshot Creation

The `CreateAppSnapshot` function in [`Cubelet/services/cubebox/appsnapshot.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/services/cubebox/appsnapshot.go) validates two mandatory annotations before proceeding with snapshot creation. These constants are defined in [`Cubelet/pkg/constants/const.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/pkg/constants/const.go):

- **`cube.app.snapshot.create`**: Must be set to `"true"` (case-insensitive) to trigger snapshot creation
- **`cube.app.snapshot.template.id`**: Must contain a non-empty string identifying the snapshot template

The validation logic explicitly checks for these annotations:

```go
createFlag, ok := annotations[constants.MasterAnnotationsAppSnapshotCreate]
if !ok || strings.ToLower(createFlag) != "true" {
    return nil, fmt.Errorf("annotation %s must be set to \"true\"", 
        constants.MasterAnnotationsAppSnapshotCreate)
}

templateID, ok := annotations[constants.MasterAnnotationAppSnapshotTemplateID]
if !ok || strings.TrimSpace(templateID) == "" {
    return nil, fmt.Errorf("annotation %s is required and must not be empty", 
        constants.MasterAnnotationAppSnapshotTemplateID)
}

```

### Container Identification and Resource Annotations

Additional annotations control which container to snapshot and what resources to include:

- **`cube.app.snapshot.container.id`**: Specifies the target container ID. If omitted, the function `snapshotContainerIDFromAnnotations` falls back to the sandbox ID
- **`cube.vm.kernel.path`**: Defines the VM kernel image path for VM-based snapshots (`AnnotationsVMKernelPath`)
- **`cube.vmmres`**: Contains JSON-encoded VM resource specifications (`AnnotationsVMSpecKey`)
- **`cube.disk`**: Describes disk mount configurations (`AnnotationsMountListKey`)
- **`cube.pmem`**: Configures persistent memory settings (`AnnotationsPmemKey` and `AnnotationPmem`)

The `snapshotContainerIDFromAnnotations` helper function implements the fallback logic:

```go
func snapshotContainerIDFromAnnotations(annotations map[string]string, sandboxID string) string {
    if len(annotations) == 0 {
        return ""
    }
    if value := strings.TrimSpace(annotations[constants.AnnotationAppSnapshotContainerID]); 
       value != "" {
        return value
    }
    return sandboxID
}

```

## Notification and Annotation Integration Flow

When a sandbox initiates snapshot creation, Cubelet reads the pod annotations defined in [`const.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/const.go) to configure the operation. Upon completion or failure, the sandbox sends an HTTP POST request to CubeMaster's `/notify/snapnotify` endpoint with the snapshot ID and status. If the operation times out, the sandbox sends a notification to `/notify/snapnotifytimeout` instead. CubeMaster persists these updates via `UpdateSnapshotHealth` or `UpdateSnapshotTimeout`, modifying the `HealthStatus` field in the `SnapshotRuntimeRef` database record.

This decoupled architecture ensures that snapshot creation logic remains stateless in Cubelet while CubeMaster maintains the authoritative state in `snapshot_runtime_refs`, enabling reliable health monitoring across distributed sandbox nodes.

## Summary

- **CubeMaster notification endpoints** (`/notify/snapnotify` and `/notify/snapnotifytimeout`) receive HTTP POST requests containing `snapshot_id` and health status updates from sandbox nodes.
- **Health status persistence** occurs in the `SnapshotRuntimeRef` model via `UpdateSnapshotHealth`, storing state in the `health_status` column of the `snapshot_runtime_refs` table.
- **Required annotations** `cube.app.snapshot.create` and `cube.app.snapshot.template.id` must be present and valid for `CreateAppSnapshot` to proceed.
- **Container identification** uses the `cube.app.snapshot.container.id` annotation with automatic fallback to the sandbox ID when unspecified.
- **Resource annotations** (`cube.vm.kernel.path`, `cube.vmmres`, `cube.disk`, `cube.pmem`) provide VM and storage configuration for comprehensive snapshot capture.

## Frequently Asked Questions

### What triggers a snapshot health check notification in CubeSandbox?

A snapshot health check notification triggers when a sandbox completes, fails, or times out during snapshot creation. The sandbox process sends an HTTP POST request to CubeMaster's `/notify/snapnotify` endpoint with the `snapshot_id` and current `status`, or to `/notify/snapnotifytimeout` if the operation exceeded its time limit.

### Which annotations are mandatory for creating an app snapshot?

CubeSandbox requires two specific annotations: `cube.app.snapshot.create` must be set to `"true"` (case-insensitive), and `cube.app.snapshot.template.id` must contain a non-empty template identifier string. The `CreateAppSnapshot` function in [`Cubelet/services/cubebox/appsnapshot.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/services/cubebox/appsnapshot.go) validates both fields before initiating the snapshot process.

### How does CubeSandbox handle timeout scenarios during snapshot creation?

When snapshot creation exceeds the expected duration, the sandbox sends a notification to the `/notify/snapnotifytimeout` endpoint with a JSON payload containing `"snapshot_id"` and `"timeout": true`. The CubeMaster handler calls `base.UpdateSnapshotTimeout`, which marks the snapshot as timed out in the `SnapshotRuntimeRef` database record.

### Where is the snapshot health status stored in the CubeSandbox architecture?

The health status is stored in the `snapshot_runtime_refs` database table, managed through the `SnapshotRuntimeRef` model defined in [`CubeMaster/pkg/base/db/models/snapshot_runtime_ref.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeMaster/pkg/base/db/models/snapshot_runtime_ref.go). The `HealthStatus` column receives updates via the `UpdateHealthStatus` method when health check notifications arrive from Cubelet nodes.