# How WeKnora Handles Multi-Instance Storage Backends: Architecture and Implementation > Discover how WeKnora manages multi-instance storage backends using UUIDs, StorageBackendID, and runtime resolution via FileServiceResolver. Learn its architecture and implementation. - Repository: [Tencent/WeKnora](https://github.com/tencent/WeKnora) - Tags: architecture - Published: 2026-09-13 --- **WeKnora handles multi-instance storage backends by storing each storage configuration as a distinct `StorageBackend` object with a UUID, binding knowledge bases to specific instances via the `StorageBackendID` field, and resolving the correct backend at runtime using the `storage://` URI wrapper parsed by the `FileServiceResolver`.** Tencent's WeKnora is a knowledge-engineering platform that decouples storage configuration from knowledge base definitions. Unlike systems limited to a single storage engine per deployment, WeKnora's multi-instance storage backend architecture allows a single workspace to manage multiple independent object stores—such as MinIO, Amazon S3, and Tencent COS—simultaneously, with per-knowledge-base granularity. According to the Tencent/WeKnora source code, this flexibility is achieved through a combination of database-backed storage definitions, tenant-level defaults, and runtime URI resolution. ## The StorageBackend Data Model WeKnora's storage layer treats each backend as an independent database entity. The `StorageBackend` struct defined in [[`internal/types/storagebackend.go`](https://github.com/Tencent/WeKnora/blob/main/internal/types/storagebackend.go)](https://github.com/Tencent/WeKnora/blob/main/internal/types/storagebackend.go) contains the fields necessary to isolate instances: - **ID**: A UUID generated automatically on `BeforeCreate` that uniquely identifies the backend. - **TenantID**: The workspace (tenant) that owns this backend. - **Provider**: The underlying implementation (`local`, `minio`, `s3`, `cos`). - **Config**: Provider-specific connection details including endpoints, credentials, and bucket names. - **Source**: Indicates creation method (`"user"` for UI-created, `"env"` for process-wide defaults). - **Status**: Soft-deletion control (`"active"` or `"disabled"`). Additionally, the `Tenant` struct in [[`internal/types/tenant.go`](https://github.com/Tencent/WeKnora/blob/main/internal/types/tenant.go)](https://github.com/Tencent/WeKnora/blob/main/internal/types/tenant.go) declares `DefaultStorageBackendID`. This field establishes which backend to use when a knowledge base does not specify its own, creating a fallback chain from explicit binding to tenant default to environment variable. ## Binding Knowledge Bases to Storage Instances The binding mechanism centers on an optional field in the `KnowledgeBase` struct located in [[`internal/types/knowledgebase.go`](https://github.com/Tencent/WeKnora/blob/main/internal/types/knowledgebase.go)](https://github.com/Tencent/WeKnora/blob/main/internal/types/knowledgebase.go): ```go StorageBackendID *string `json:"storage_backend_id,omitempty"` ``` When `StorageBackendID` is set, the knowledge base is explicitly bound to that specific storage instance. If the field is `nil`, WeKnora falls back to the tenant's `DefaultStorageBackendID`, and ultimately to the process-wide default defined by the `STORAGE_TYPE` environment variable. The helper method `SharesStorageBackendWith` in the same file compares two knowledge bases for backend compatibility: ```go func (kb *KnowledgeBase) SharesStorageBackendWith(other *KnowledgeBase, defaultBackendID, defaultProvider string) bool ``` This method first checks for explicit `StorageBackendID` matches on both sides. Only when neither knowledge base has an explicit ID does it compare the effective storage provider, ensuring accurate conflict detection during import and export operations. ## Runtime Resolution with the storage:// URI To address a specific backend instance at runtime, WeKnora uses a URI wrapper scheme. File references stored in the database use standard provider schemes (`minio://`, `s3://`), but to target a specific backend instance, the system wraps them as follows: ``` storage:///:// ``` Utility functions in [[`internal/types/knowledgebase.go`](https://github.com/Tencent/WeKnora/blob/main/internal/types/knowledgebase.go)](https://github.com/Tencent/WeKnora/blob/main/internal/types/knowledgebase.go) handle this conversion: ```go func BuildStorageBackendPath(backendID, providerPath string) string { return "storage://" + strings.TrimSpace(backendID) + "/" + providerPath } func ParseStorageBackendPath(path string) (backendID, providerPath string, ok bool) ``` When the server needs to retrieve a file, the `FileServiceResolver` in [[`internal/storageurl/resolver.go`](https://github.com/Tencent/WeKnora/blob/main/internal/storageurl/resolver.go)](https://github.com/Tencent/WeKnora/blob/main/internal/storageurl/resolver.go) performs the following steps: 1. **Detect** `resource://` references and route to the tenant-wide default. 2. **Parse** the `storage://` wrapper using `ParseStorageBackendPath` to extract the `backendID`. 3. **Determine** the provider by inspecting the inner URI scheme via `ParseProviderScheme`, falling back to `StorageEngineConfig.DefaultProvider` if absent. 4. **Cache** the resolved client per request using a map keyed by `backendID:provider` to avoid SDK client recreation. 5. **Resolve** the concrete backend by loading the `StorageBackend` row from the database and constructing a `FileService` via `filesvc.NewFileServiceFromStorageConfig`. 6. **Fallback** to a local file service if resolution fails or the provider is `local`. ## Implementing Multi-Instance Storage in Practice ### Creating a Storage Backend Programmatically To create a new backend instance in code, use the `StorageBackend` type and repository pattern: ```go package main import ( "context" "github.com/Tencent/WeKnora/internal/types" "github.com/Tencent/WeKnora/internal/types/interfaces" ) func createS3Backend(ctx context.Context, repo interfaces.StorageBackendRepository) error { backend := &types.StorageBackend{ TenantID: 12345, Name: "S3 Production", Provider: "s3", Config: types.StorageBackendConfig{ Endpoint: "https://s3.amazonaws.com", AccessKeyID: "AKIAIOSFODNN7EXAMPLE", SecretAccessKey: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", BucketName: "prod-knowledge-base", UseSSL: true, }, } if err := backend.Validate(); err != nil { return err } return repo.Create(ctx, backend) } ``` The `Validate()` method enforces provider-specific constraints, while `BeforeCreate` automatically generates the UUID that serves as the `backendID` for URI construction. ### Generating Storage-Wrapped File References When storing file metadata that must reference a specific backend, use the utility functions to build the wrapper: ```go import "github.com/Tencent/WeKnora/internal/types" func makeFileReference(backendID, bucketPath string) string { // bucketPath format: "provider://bucket/key" return types.BuildStorageBackendPath(backendID, bucketPath) } // Example usage: ref := makeFileReference("c3f5e1d2-9a4b-4d1a-a5f3-e7b9c2d5f6a7", "minio://documents/report.pdf") // Result: storage://c3f5e1d2-9a4b-4d1a-a5f3-e7b9c2d5f6a7/minio://documents/report.pdf ``` ### Resolving File Services at Runtime To stream or presign URLs for files stored across different backends, instantiate the resolver: ```go import ( "github.com/Tencent/WeKnora/internal/storageurl" "github.com/Tencent/WeKnora/internal/types" ) func getFileService(tenant *types.Tenant, storagePath string) (interfaces.FileService, error) { resolver := storageurl.NewFileServiceResolver(tenant, file.DefaultFileService) return resolver.ResolveFileService(storagePath) } // Usage: svc, err := getFileService(tenant, "storage://abc123/cos://bucket/image.png") if err != nil { // handle resolution failure } url, err := svc.Presign("cos://bucket/image.png") // Returns presigned URL ``` The resolver handles credential retrieval, client initialization, and caching automatically based on the `backendID` embedded in the path. ## Security and Legacy Compatibility Multi-instance storage requires careful credential management. In [[`internal/types/storagebackend.go`](https://github.com/Tencent/WeKnora/blob/main/internal/types/storagebackend.go)](https://github.com/Tencent/WeKnora/blob/main/internal/types/storagebackend.go), the `StorageBackendConfig` encrypts sensitive fields like `SecretAccessKey` using AES encryption via `utils.GetAESKey`. When the API returns a backend configuration to clients, the `NewStorageBackendResponse` function masks secrets with a redacted placeholder to prevent credential leakage. For existing deployments using the legacy single-storage configuration, WeKnora provides migration helpers. The `StorageBackendFromLegacy` function converts a `StorageEngineConfig` into a concrete `StorageBackend` row, while `StorageBackendFromEnvironment` creates backend definitions derived from environment variables, ensuring backward compatibility during upgrades. ## Summary - **Database-backed instances**: Each storage backend is a distinct row with a UUID, enabling unlimited instances per tenant. - **Explicit binding**: Knowledge bases link to backends via `StorageBackendID`, with automatic fallback to tenant defaults. - **URI-based resolution**: The `storage:///://` wrapper allows per-file backend selection parsed by `FileServiceResolver`. - **Runtime efficiency**: Per-request caching of SDK clients prevents connection overhead when accessing multiple backends. - **Secure by default**: Credentials are encrypted at rest and redacted in API responses. ## Frequently Asked Questions ### Can a single knowledge base use multiple storage backends simultaneously? No, a knowledge base binds to exactly one storage backend via its `StorageBackendID` field. However, different knowledge bases within the same workspace can each bind to different backends, and file references within a knowledge base can technically point to other backends via explicit `storage://` URIs, though this bypasses the knowledge base's default binding. ### How does WeKnora handle storage backend credentials securely? According to the source code in [`internal/types/storagebackend.go`](https://github.com/Tencent/WeKnora/blob/main/internal/types/storagebackend.go), secrets within `StorageBackendConfig` are encrypted using AES with a key retrieved from `utils.GetAESKey`. When the backend configuration is serialized for API responses, the `NewStorageBackendResponse` utility replaces sensitive values with a `RedactedSecretPlaceholder`, ensuring credentials never travel over the wire in plaintext. ### What happens if a knowledge base doesn't specify a storage backend? If `StorageBackendID` is nil, WeKnora checks the tenant's `DefaultStorageBackendID` field. If that is also unset, the system falls back to the process-wide default derived from the `STORAGE_TYPE` environment variable. This three-tier fallback ensures that legacy configurations continue to function while allowing gradual adoption of explicit multi-instance bindings. ### Is it possible to migrate from a legacy single-storage setup to multi-instance? Yes, WeKnora includes migration utilities in [`internal/types/storagebackend.go`](https://github.com/Tencent/WeKnora/blob/main/internal/types/storagebackend.go). The `StorageBackendFromLegacy` function converts a legacy `StorageEngineConfig` into a concrete `StorageBackend` database row, preserving existing connections. Additionally, `StorageBackendFromEnvironment` creates backend instances from environment variables for deployments that prefer infrastructure-as-code over UI management.