Cloud Storage Implementation
Technical documentation of the cloud storage architecture and implementation details
On this page
This page is the adapter. Turning a bucket on is the Cloud storage guide.
Overview
Media can be stored on local disk or in S3/R2/Cloudinary. Both cases are served by one adapter module and one storage type setting, so callers never branch on the backend:
| Concern | Module |
|---|---|
| Adapter contracts + factory | src/utils/media/storage-adapters.ts |
| Filesystem/driver wrappers | src/utils/media/media-storage.server.ts (saveFile, deleteFile, fileExists, getFile, saveResized) |
| Storage type & URL shapes | src/utils/media/media-models.ts (STORAGE_TYPES, StorageType) |
| Public file serving / redirect | src/routes/files/[...path]/+server.ts |
| Signed media URLs (optional) | src/utils/media/signed-urls.ts |
| Operator configuration steps | Cloud Storage & CDN guide |
Adjacent modules that share this pipeline:
src/utils/media/streaming-upload.ts—parseMultipartStream()hands each file part to a callback while it streams, so large uploads are never fully buffered.src/utils/media/storage-analytics.ts—analyze(),generateInsights(),trends(),top(),quota()power the media-storage analytics dashboard widget.src/utils/media/media-service.server.ts—sanitizeSvg()and the remote-upload path (validateEgressUrl+safeFetch) sit in front of the adapters.
There is no separate cloudStorage.ts module: cloud-upload logic lives in the adapter classes below.
Architecture Decisions
1. One StorageAdapter Interface
Every backend implements the same contract (src/utils/media/storage-adapters.ts:86):
export interface StorageAdapter {
upload(data: Buffer | ReadableStream | Readable, relativePath: string): Promise<string>;
download(relativePath: string): Promise<Buffer>;
remove(relativePath: string): Promise<void>;
exists(relativePath: string): Promise<boolean>;
getMetadata(relativePath: string): Promise<StorageMetadata | null>;
getUrl(relativePath: string, prefix?: string): string;
/** Optional cleanup when the adapter is being replaced or shut down. */
dispose?(): void | Promise<void>;
}
Implementations: LocalStorageAdapter (:158), S3StorageAdapter (:261, used for both s3 and r2), CloudinaryStorageAdapter (:423).
Rationale: business code calls saveFile() / deleteFile() / fileExists() and never imports a vendor SDK. SDKs are still lazy-loaded inside the adapter that needs them, so a local-only deployment pays nothing for them.
2. Adapter Factory & Lifecycle
getStorageAdapter() (:547) caches one instance per process and re-creates it only when MEDIA_STORAGE_TYPE changes; the previous adapter’s dispose() is invoked first, which is what keeps SDK connection pools from leaking across a settings change.
if (config.storageType === "s3" || config.storageType === "r2") {
currentAdapter = new S3StorageAdapter(config);
} else if (config.storageType === "cloudinary") {
currentAdapter = new CloudinaryStorageAdapter(config);
} else {
currentAdapter = new LocalStorageAdapter();
}
3. Path & URL Normalization
Paths are stored relative to the media root and URLs are always expressed as /files/..., whatever the backend:
getConfig()(:126) reads the storage settings withgetPublicSettingSync()and normalizes the media folder (leading./,/and trailing/stripped).getPath(config, relativePath, prefix?)(:149) prependsconfig.mediaFolder;getCloudPath(relativePath, prefix?)(:584) is the config-less shorthand.getUrl(relativePath, prefix?)(:594) resolves through the active adapter and memoizes the result in a 5-minute process-local cache (URL_CACHE_TTL = 300_000, oldest-entry eviction above 10 000 keys).resolveConfiguredMediaFolder()(:114) intentionally letsprocess.env.MEDIA_FOLDERwin over the DB setting inside benchmark/test harnesses, so an isolated run can never read or write the live media directory.
Rationale: the database schema stays backend-agnostic and a storage migration does not require rewriting stored URLs.
4. Redirect vs Stream
src/routes/files/[...path]/+server.ts decides from MEDIA_STORAGE_TYPE:
- Cloud (
storageType !== "local"): the route asks the adapter for metadata and answers 302 withLocationbuilt fromMEDIA_CLOUD_PUBLIC_URL(orMEDIASERVER_URL) + media folder + path. When the adapter exposes anETagand the request’sIf-None-Matchmatches, it short-circuits to 304 instead. A missing public URL fails closed withCLOUD_CONFIG_ERROR(500). - Local: the file is streamed from disk with
Content-Length,ETag,Last-Modified,Cache-Control: public, max-age=31536000, immutable, andAccept-Ranges: bytes;Rangerequests get a real 206 (with416for unsatisfiable ranges) and the filesystem stream is destroyed on client abort.
Rationale: redirects keep bytes off the CMS process and let the CDN/bucket do range handling and caching, while the local path stays a first-class citizen for self-hosting.
Configuration
All storage settings are public settings (schema: src/databases/public-config-schema.ts, editable in System Settings) — never in config/private.ts:
| Setting | Purpose |
|---|---|
MEDIA_STORAGE_TYPE |
local | s3 | r2 | cloudinary (STORAGE_TYPES) |
MEDIA_FOLDER |
Path prefix inside the bucket/disk root |
MEDIA_BUCKET_NAME |
S3/R2 bucket |
MEDIA_CLOUD_REGION |
S3/R2 region |
MEDIA_CLOUD_ENDPOINT |
Custom endpoint (R2, MinIO) — SSRF-validated |
MEDIA_CLOUD_PUBLIC_URL |
Public CDN/bucket origin used for the 302 redirect |
MEDIA_ACCESS_KEY_ID / MEDIA_SECRET_ACCESS_KEY |
S3/R2 credentials |
CLOUDINARY_CLOUD_NAME (plus key/secret) |
Cloudinary credentials |
MEDIASERVER_URL |
Legacy fallback for MEDIA_CLOUD_PUBLIC_URL |
MEDIA_SIGNED_URL_ENABLED |
Require ?sig=&exp= on tenant-scoped /files/ reads |
Step-by-step setup for S3, R2, Cloudinary and a CDN front end: Cloud Storage & CDN guide.
Data Flow
Upload
- The upload pipeline builds a hashed relative path and calls
saveFile(data, relPath)(media-storage.server.ts:79). saveFiledelegates togetStorageAdapter().upload(...)— no branching at the call site.- The adapter returns the stored path; the media record keeps a
/files/...URL.
Access
- The browser requests
/files/<tenant>/<hash>/original/file.jpg. - Local storage: traversal guard (
path.relativemust stay inside the media root), tenant check against the request’stenantId, and — whenMEDIA_SIGNED_URL_ENABLEDis on — HMAC validation viavalidateSignedMediaUrl()before any filesystem work. - Cloud storage: 304 on a matching
ETag, otherwise 302 to the public URL (bucket/CDN).
Delete
deleteFile(url) (media-storage.server.ts:87) strips the origin, any legacy prefix and the /files/ segment, then calls getStorageAdapter().remove(rel). Resized variants are removed through the same path, so a bucket and a disk behave identically.
Security Considerations
- SSRF-guarded endpoints:
validateS3Endpoint()(storage-adapters.ts:42) rejects non-HTTP protocols, loopback/private IP ranges (IPv4 and IPv6), cloud-metadata hosts and blocked hostnames before any S3 client is created. - No credentials in
private.ts: bucket keys are database settings, read throughgetPublicSettingSync(); only bootstrap secrets live inconfig/private.ts. - Traversal & tenant isolation: the
/files/route rejects..escapes and cross-tenant paths before touching storage. - Fail-closed signed URLs: when signing is enabled, a missing or invalid signature is a 403 (
SIGNATURE_REQUIRED), andgenerateSignedMediaUrl()returns an empty string rather than an unsigned URL when no secret is configured. - SVG hardening:
image/svg+xmlresponses carry their own restrictiveContent-Security-Policy(script-src 'none',sandbox) plusX-Content-Type-Options: nosniff.
Troubleshooting
| Symptom | Check |
|---|---|
CLOUD_CONFIG_ERROR (500) |
MEDIA_STORAGE_TYPE is not local but MEDIA_CLOUD_PUBLIC_URL / MEDIASERVER_URL is unset |
Uploads succeed, /files/... 404s |
MEDIA_FOLDER mismatch between the write path and the served prefix |
S3 endpoint blocked (private IP) |
MEDIA_CLOUD_ENDPOINT points at an internal host — the SSRF guard is doing its job |
| Signature failures after enabling signing | Secret changed, or the URL was generated before MEDIA_SIGNED_URL_ENABLED was turned on |
| Stale URLs after a config change | The URL cache is process-local with a 5-minute TTL — restart the server or wait out the TTL |