# Understanding the croc Stored Transfer Protocol and Client-Side Encryption

> Learn about the croc stored transfer protocol and client-side encryption. Securely upload files using AES-GCM encryption for ultimate privacy.

- Repository: [Zack/croc](https://github.com/schollz/croc)
- Tags: deep-dive
- Published: 2026-07-30

---

**The croc stored transfer protocol enables secure file uploads to remote storage services (S3, GCS, etc.) using client-side AES-GCM encryption, ensuring the storage provider never sees file names, sizes, hashes, or encryption keys.**

The **croc** file transfer tool from `schollz/croc` supports a **stored-transfer** mode that uploads encrypted files to cloud storage rather than using direct peer-to-peer connections. This architecture moves all cryptographic operations—including key generation, derivation, and authenticated encryption—to the client, maintaining zero-knowledge privacy guarantees even when using third-party storage backends.

## Protocol Identity and Architecture

The stored-transfer protocol is formally identified by the constant `croc-store-v1` defined in [`src/storecrypto/storecrypto.go`](https://github.com/schollz/croc/blob/main/src/storecrypto/storecrypto.go):

```go
const Protocol = "croc-store-v1"

```

When operating in stored mode, croc splits files into 4 MiB chunks (`ChunkSize`), encrypts them with AES-256-GCM, and uploads the ciphertext to a configurable storage backend. The storage service only handles opaque encrypted blobs and never receives the master encryption key or the transfer identifier required for decryption.

## Key Generation and Derivation

All cryptographic material originates from two client-generated values:

- **Master Key** – A fresh 256-bit random key created via `GenerateKey()` for each transfer.
- **Transfer ID** – A public, URL-safe 128-bit identifier generated by `GenerateTransferID()`.

Neither value leaves the client unencrypted. From the master key, croc derives specialized keys using HKDF-SHA256 via the internal `derive` function:

```go
key, err := hkdf.Key(sha256.New, master, nil,
    Protocol+"/"+label, KeySize)

```

In [`src/storecrypto/storecrypto.go`](https://github.com/schollz/croc/blob/main/src/storecrypto/storecrypto.go), two labels determine the key purpose:
- `"manifest"` – Encrypts the JSON manifest describing the transfer.
- `"data"` – Encrypts individual file chunks.

## Authenticated Encryption with Associated Data

croc implements **AES-GCM** through the `aeadFor` helper, using randomly generated nonces for each encryption operation. Critical to the security model is the **Associated Data (AAD)** that binds ciphertext to the specific transfer and data type, preventing substitution attacks.

The AAD format varies by content type as defined in [`storecrypto.go`](https://github.com/schollz/croc/blob/main/storecrypto.go):

```go
// Manifest AAD
[]byte(Protocol + "\x00" + id + "\x00manifest")

// Chunk AAD  
[]byte(Protocol + "\x00" + id + "\x00chunk\x00" + …)

```

This construction ensures that chunks cannot be swapped between transfers or confused with manifest data.

## Manifest Encryption and Validation

Before upload, croc constructs a `Manifest` structure describing the entire transfer—version, chunk size, and a slice of `ManifestFile` entries containing file names, sizes, and SHA-256 hashes. The sender validates the manifest with `ValidateManifest()`, then encrypts it using `SealManifest()` (which chains to `SealManifestJSON` and the internal `seal` function).

Receivers download the encrypted manifest and decrypt it via `OpenManifest()` → `OpenManifestJSON` → `open`, which verifies authenticity before returning the plaintext JSON description of the file collection.

## Chunk Encryption and File Splitting

Files are divided into 4 MiB chunks. For each chunk, croc creates a `ChunkRef` recording its object index, file index, chunk number, and plaintext size. The `SealChunk` function encrypts chunk data using the "data" label and chunk-specific AAD, while `OpenChunk` performs decryption and integrity verification, confirming the output length matches `PlainSize`.

This chunking strategy allows resumable downloads and streaming decryption without requiring the receiver to download entire files before verification.

## Share Generation and Transfer Tokens

To initiate a download, the sender creates a `Share` structure bundling three fields:
- `Origin` – Base URL of the storage service (e.g., `https://s3.amazonaws.com`)
- `ID` – Public transfer identifier
- `MasterKey` – 32-byte secret for key derivation

croc encodes shares in two formats via methods in [`storecrypto.go`](https://github.com/schollz/croc/blob/main/storecrypto.go):
- **Browser URL**: `https://<origin>/s/<id>#v1.<key>` (`BrowserURL()`)
- **CLI Token**: `croc-store-v1.<origin-base64>.<id>.<key>` (`CLIToken()`)

Receivers parse either format using `ParseShare()`, which validates and normalizes the components before returning a `Share` struct for decryption operations.

## End-to-End Implementation Example

The following demonstrates the complete sender and receiver flow using the `storecrypto` and `storeclient` packages:

```go
// Sender implementation
import (
    "github.com/schollz/croc/src/storecrypto"
    "github.com/schollz/croc/src/storeclient"
)

// Generate fresh cryptographic material
masterKey, _ := storecrypto.GenerateKey()
transferID, _ := storecrypto.GenerateTransferID()

// Build the manifest describing files
manifest := storecrypto.Manifest{
    Version:   storecrypto.Version,
    ChunkSize: storecrypto.ChunkSize,
    Files: []storecrypto.ManifestFile{
        {
            Name:       "document.pdf",
            Size:       1048576,
            Modified:   time.Now(),
            SHA256:     storecrypto.EncodedSHA256(fileBytes),
            FirstChunk: 0,
            ChunkCount: 1,
        },
    },
}

// Encrypt and upload
encManifest, _ := storecrypto.SealManifest(masterKey, transferID, manifest)
ref := storecrypto.ChunkRef{ObjectIndex: 0, FileIndex: 0, FileChunk: 0, PlainSize: len(fileBytes)}
encChunk, _ := storecrypto.SealChunk(masterKey, transferID, ref, fileBytes)

client := storeclient.New("https://my-storage.example")
client.PutObject(transferID, encManifest)
client.PutObject(transferID, encChunk)

// Generate share for receiver
share := storecrypto.Share{Origin: "https://my-storage.example", ID: transferID, MasterKey: masterKey}
url, _ := share.BrowserURL()   // https://my-storage.example/s/<id>#v1.<key>
token, _ := share.CLIToken()     // croc-store-v1.<b64origin>.<id>.<key>

```

```go
// Receiver implementation
import "github.com/schollz/croc/src/storecrypto"

// Parse CLI token or browser URL
share, err := storecrypto.ParseShare("croc-store-v1.aHR0cHM6Ly9teS1zdG9yYWdlLmV4YW1wbGU=.transferID.key")
if err != nil {
    log.Fatal(err)
}

// Download and decrypt manifest
encManifest := downloadObject(share.ID) // platform-specific download
manifest, err := storecrypto.OpenManifest(share.MasterKey, share.ID, encManifest, 1<<40)

// Stream and decrypt chunks
for _, ref := range storecrypto.ChunkRefs(manifest) {
    encChunk := downloadChunk(ref.ObjectIndex)
    plain, err := storecrypto.OpenChunk(share.MasterKey, share.ID, ref, encChunk)
    // Write plain to manifest.Files[ref.FileIndex].Name
}

```

## Summary

- The **croc stored transfer protocol** (`croc-store-v1`) enables secure file hosting on untrusted storage by performing all encryption client-side in [`src/storecrypto/storecrypto.go`](https://github.com/schollz/croc/blob/main/src/storecrypto/storecrypto.go).
- **AES-256-GCM** protects both manifests and file chunks, with HKDF-SHA256 deriving separate keys for each data type from a client-generated master key.
- **Associated Data (AAD)** binds ciphertext to specific transfers, preventing cross-transfer substitution attacks.
- **Share tokens** encode the storage origin, public transfer ID, and master key in URL-safe formats parseable by `ParseShare()`.
- The storage backend only handles opaque encrypted blobs and cannot determine file names, sizes, content hashes, or encryption keys.

## Frequently Asked Questions

### How is the encryption key generated and managed in croc stored transfers?

Each transfer generates a unique 256-bit master key via `storecrypto.GenerateKey()` and a 128-bit transfer ID via `GenerateTransferID()`. Both are created on the sender's client using cryptographically secure random number generation. The master key never leaves the client unencrypted—it is shared with receivers only through the URL fragment (browser mode) or CLI token, ensuring the storage service remains unaware of the decryption credentials.

### What prevents the cloud storage provider from accessing my files?

All encryption occurs before data reaches the storage backend. Files are split into 4 MiB chunks and encrypted with AES-256-GCM using keys derived from the master secret via HKDF. The storage provider receives only ciphertext chunks and an encrypted manifest, lacking both the encryption keys and the transfer ID required for decryption. This zero-knowledge architecture ensures the provider cannot determine file contents, names, sizes, or hashes.

### How does croc verify the integrity of transferred chunks?

croc uses **authenticated encryption with associated data (AEAD)** via AES-GCM. Each chunk includes a cryptographic authentication tag generated during `SealChunk()` that is verified during `OpenChunk()`. Additionally, the `ChunkRef` structure records the expected `PlainSize`, and the decryption process validates that the output matches this length. The manifest contains SHA-256 hashes of the original files, allowing receivers to verify integrity after reconstruction.

### What is the difference between croc's live transfer and stored transfer modes?

Live transfers establish direct peer-to-peer connections using the `webrtc` or `tcp` relays defined in [`src/message/message.go`](https://github.com/schollz/croc/blob/main/src/message/message.go), streaming data in real-time without intermediate storage. Stored transfers upload encrypted chunks to persistent cloud storage (S3, GCS, etc.) using [`src/storeclient/client.go`](https://github.com/schollz/croc/blob/main/src/storeclient/client.go) and [`src/store/service.go`](https://github.com/schollz/croc/blob/main/src/store/service.go), allowing asynchronous retrieval. While live transfers require both parties to be simultaneously online, stored transfers enable "send now, download later" workflows while maintaining the same client-side encryption guarantees.