# CasaOS File Upload Service Architecture: A Deep Dive into the Chunked Upload Pipeline > Discover the CasaOS file upload service architecture. Learn how its chunked upload pipeline manages state, temporary storage, and routing for reliable file transfers. - Repository: [IceWhale/CasaOS](https://github.com/IceWhaleTech/CasaOS) - Tags: architecture - Published: 2026-06-28 --- **CasaOS implements a layered, chunk-aware file upload pipeline that separates upload state management, temporary storage handling, and HTTP routing to support resumable, multi-chunk file transfers.** The architecture of the file upload service in CasaOS follows a modular design that enables reliable handling of large files through chunked transfers. This system separates concerns between state management, filesystem operations, and HTTP request handling to ensure uploads can resume after interruptions. Understanding these layers helps developers integrate with or extend CasaOS's storage capabilities. ## Core Components of the Upload Architecture The file upload service is organized into four distinct layers that work together to process multipart uploads. ### Service Layer (service/file_upload.go) The `FileUploadService` struct serves as the central coordinator for upload state. It maintains a `sync.Map` of `FileInfo` objects that track initialization flags, uploaded bitmaps, and chunk counts for each active transfer. The service uses a `sync.RWMutex` to ensure thread-safe concurrent access to upload metadata. Key methods include: - **`UploadFile`** – Writes individual chunks to temporary `.tmp` files and updates the bitmap tracking which segments have been received. - **`TestChunk`** – Provides a lightweight mechanism to probe whether a specific chunk already exists in the temporary storage. ### Utility Layer (pkg/utils/file/file.go) Low-level filesystem operations live in the utility package, decoupled from HTTP concerns. The **`SpliceFiles`** function concatenates numbered chunk files into the final destination file by opening each chunk sequentially and writing them via a buffered writer. Supporting functions like `MkDir`, `CheckNotExist`, and `RMDir` handle directory creation and cleanup operations. ### HTTP API Layer (route/v1/file.go) The router exposes two primary endpoints that translate HTTP parameters into service calls: - **`GetFileUpload`** (GET) – Acts as a "chunk exists" probe. It calculates a content hash using `file.GetHashByContent`, creates the temporary directory structure, and returns HTTP **200** if the chunk exists or **204** if the chunk is ready for upload. - **`PostFileUpload`** (POST) – Receives multipart form data, writes chunks to the temporary folder, and triggers finalization when the last chunk arrives. ### Server Wiring (route/v2/route.go) During application startup, the `NewCasaOS` function instantiates `FileUploadService` and injects it into the generated server interface. This dependency injection pattern ensures the upload service is available throughout the request lifecycle. ## How the Chunked Upload Pipeline Works The upload process follows a stateless, file-system-backed protocol where progress is inferred from the presence of chunk files on disk rather than persistent database records. ### Step 1: Chunk Probing (Optional) Before uploading data, clients may probe the server to check which chunks already exist. The client sends a GET request to `/file/upload` with parameters including `path`, `filename`, `totalChunks`, and `chunkNumber`. The handler calculates a hash of the filename and checks for the existence of `/.temp//`. If found, the server returns HTTP 200; otherwise, it returns HTTP 204. ### Step 2: Temporary Directory Creation For multi-chunk uploads, the system creates a temporary directory under the target path following the pattern `.temp//`. The hash is derived from the filename content, ensuring unique isolation for each upload operation. ### Step 3: Chunk Reception and Storage When receiving a POST request, the handler extracts the multipart file using `ctx.Request().FormFile("file")`. It writes the chunk data to a file named with its chunk number inside the temporary directory. After each write, the handler reads the directory contents to count existing files, comparing this against `totalChunks` to determine completion status. ### Step 4: Finalization and Splicing When the number of received chunks equals `totalChunks`, the handler invokes `SpliceFiles(tempDir, path, totalChunks, 1)`. This function opens the final destination file and sequentially appends each numbered chunk (1 through N) to construct the complete file. The function uses buffered I/O to optimize filesystem performance during concatenation. ### Step 5: Cleanup After successful splicing, the system schedules asynchronous cleanup using `time.Sleep(11s)` followed by `file.RMDir(tempDir)`. This delay ensures file handles are released and any pending operations complete before removing the temporary structure. ### Single-File Optimization When `totalChunks` equals 1, the pipeline bypasses the temporary folder entirely. The handler writes directly to `/`, eliminating overhead for small files. ## Implementation Examples ### Single-File Upload via cURL For files that don't require chunking, upload directly to the destination: ```bash curl -X POST http://localhost:80/file/upload \ -F "path=/share/downloads" \ -F "filename=myphoto.jpg" \ -F "relativePath=myphoto.jpg" \ -F "totalChunks=1" \ -F "chunkNumber=1" \ -F "file=@myphoto.jpg" ``` ### Chunked Upload Workflow For large files, implement the probe-then-upload pattern: ```bash # Step 1: Probe existing chunks (optional resumption check) for i in 1 2 3; do curl -G "http://localhost:80/file/upload" \ -d "path=/share/downloads" \ -d "filename=video.mp4" \ -d "relativePath=video.mp4" \ -d "totalChunks=3" \ -d "chunkNumber=$i" done # Step 2: Upload each chunk for i in 1 2 3; do curl -X POST http://localhost:80/file/upload \ -F "path=/share/downloads" \ -F "filename=video.mp4" \ -F "relativePath=video.mp4" \ -F "totalChunks=3" \ -F "chunkNumber=$i" \ -F "file=@chunk_$i.bin" done ``` When the third chunk arrives, CasaOS automatically splices the temporary files into `/share/downloads/video.mp4` and schedules the temp folder for deletion. ### Programmatic Upload with Go The repository includes generated client code under `codegen/client/` for programmatic integration: ```go import ( "context" "github.com/IceWhaleTech/CasaOS/codegen" "github.com/IceWhaleTech/CasaOS/codegen/client" ) func uploadChunk(apiClient *client.APIClient, path, filename string, chunkNum, totalChunks int, data []byte) error { opts := codegen.PostFileUploadOpts{ Path: path, Filename: filename, RelativePath: filename, TotalChunks: strconv.Itoa(totalChunks), ChunkNumber: strconv.Itoa(chunkNum), } _, err := apiClient.FileApi.PostFileUpload( context.Background(), data, &opts, ) return err } ``` ## Summary - **CasaOS file upload service architecture** separates concerns across four layers: service state management, filesystem utilities, HTTP routing, and server wiring. - **Chunked uploads** use temporary directories under `.temp//` to store individual segments until finalization. - **State tracking** occurs in memory via `sync.Map` and on disk via file presence, eliminating the need for persistent metadata storage during transfers. - **Finalization** uses the `SpliceFiles` function to concatenate chunks sequentially, followed by asynchronous cleanup after 11 seconds. - **Single-file uploads** bypass the temporary storage system entirely when `totalChunks` equals 1. ## Frequently Asked Questions ### How does CasaOS handle interrupted uploads? CasaOS supports resumable uploads through its chunk existence probe mechanism. Clients can query the `GetFileUpload` endpoint to check which chunks already exist in the temporary directory. Since chunk state persists as physical files in `.temp//`, interrupted uploads can resume by re-sending only missing chunks. The server reconstructs the final file only when all chunks are present. ### What is the temporary directory structure for chunked uploads? The system creates temporary directories under the target path using the pattern `.temp//`, where `hash` is derived from `file.GetHashByContent(filename)` and `totalChunks` represents the total number of expected segments. Individual chunks are stored as numbered files (1, 2, 3, etc.) within this directory until `SpliceFiles` concatenates them into the final destination. ### How does the service track which chunks have been received? The `FileUploadService` maintains upload metadata in a `sync.Map` of `FileInfo` objects for internal callers, but the HTTP layer primarily tracks progress by inspecting the filesystem. After each chunk write in `PostFileUpload`, the handler reads the temporary directory using `ioutil.ReadDir` and compares the file count against `totalChunks`. This stateless approach ensures durability across process restarts, as progress is stored on disk rather than in volatile memory. ### Can I upload files without using the chunked protocol? Yes. When `totalChunks` is set to 1, the `PostFileUpload` handler in [`route/v1/file.go`](https://github.com/IceWhaleTech/CasaOS/blob/main/route/v1/file.go) bypasses the temporary folder creation and writes directly to `/`. This optimization eliminates the overhead of directory creation, chunk splicing, and cleanup for small files or single-part uploads.