Skip to content

Documentation

Cloud Storage Implementation

Technical documentation of the cloud storage architecture and implementation details

9/22/2026
6 min read Edit on GitHub

This page is the adapter. Turning a bucket on is the Cloud storage guide.

Overview

Media can be stored on local disk or in S3/R2/Cloudinary. Both cases are served by one adapter module and one storage type setting, so callers never branch on the backend:

Concern Module
Adapter contracts + factory src/utils/media/storage-adapters.ts
Filesystem/driver wrappers src/utils/media/media-storage.server.ts (saveFile, deleteFile, fileExists, getFile, saveResized)
Storage type & URL shapes src/utils/media/media-models.ts (STORAGE_TYPES, StorageType)
Public file serving / redirect src/routes/files/[...path]/+server.ts
Signed media URLs (optional) src/utils/media/signed-urls.ts
Operator configuration steps Cloud Storage & CDN guide

Adjacent modules that share this pipeline:

  • src/utils/media/streaming-upload.ts — parseMultipartStream() hands each file part to a callback while it streams, so large uploads are never fully buffered.
  • src/utils/media/storage-analytics.ts — analyze(), generateInsights(), trends(), top(), quota() power the media-storage analytics dashboard widget.
  • src/utils/media/media-service.server.ts — sanitizeSvg() and the remote-upload path (validateEgressUrl + safeFetch) sit in front of the adapters.

There is no separate cloudStorage.ts module: cloud-upload logic lives in the adapter classes below.

Architecture Decisions

1. One StorageAdapter Interface

Every backend implements the same contract (src/utils/media/storage-adapters.ts:86):

export interface StorageAdapter {
  upload(data: Buffer | ReadableStream | Readable, relativePath: string): Promise<string>;
  download(relativePath: string): Promise<Buffer>;
  remove(relativePath: string): Promise<void>;
  exists(relativePath: string): Promise<boolean>;
  getMetadata(relativePath: string): Promise<StorageMetadata | null>;
  getUrl(relativePath: string, prefix?: string): string;
  /** Optional cleanup when the adapter is being replaced or shut down. */
  dispose?(): void | Promise<void>;
}

Implementations: LocalStorageAdapter (:158), S3StorageAdapter (:261, used for both s3 and r2), CloudinaryStorageAdapter (:423).

Rationale: business code calls saveFile() / deleteFile() / fileExists() and never imports a vendor SDK. SDKs are still lazy-loaded inside the adapter that needs them, so a local-only deployment pays nothing for them.

2. Adapter Factory & Lifecycle

getStorageAdapter() (:547) caches one instance per process and re-creates it only when MEDIA_STORAGE_TYPE changes; the previous adapter’s dispose() is invoked first, which is what keeps SDK connection pools from leaking across a settings change.

if (config.storageType === "s3" || config.storageType === "r2") {
  currentAdapter = new S3StorageAdapter(config);
} else if (config.storageType === "cloudinary") {
  currentAdapter = new CloudinaryStorageAdapter(config);
} else {
  currentAdapter = new LocalStorageAdapter();
}

3. Path & URL Normalization

Paths are stored relative to the media root and URLs are always expressed as /files/..., whatever the backend:

  • getConfig() (:126) reads the storage settings with getPublicSettingSync() and normalizes the media folder (leading ./, / and trailing / stripped).
  • getPath(config, relativePath, prefix?) (:149) prepends config.mediaFolder; getCloudPath(relativePath, prefix?) (:584) is the config-less shorthand.
  • getUrl(relativePath, prefix?) (:594) resolves through the active adapter and memoizes the result in a 5-minute process-local cache (URL_CACHE_TTL = 300_000, oldest-entry eviction above 10 000 keys).
  • resolveConfiguredMediaFolder() (:114) intentionally lets process.env.MEDIA_FOLDER win over the DB setting inside benchmark/test harnesses, so an isolated run can never read or write the live media directory.

Rationale: the database schema stays backend-agnostic and a storage migration does not require rewriting stored URLs.

4. Redirect vs Stream

src/routes/files/[...path]/+server.ts decides from MEDIA_STORAGE_TYPE:

  • Cloud (storageType !== "local"): the route asks the adapter for metadata and answers 302 with Location built from MEDIA_CLOUD_PUBLIC_URL (or MEDIASERVER_URL) + media folder + path. When the adapter exposes an ETag and the request’s If-None-Match matches, it short-circuits to 304 instead. A missing public URL fails closed with CLOUD_CONFIG_ERROR (500).
  • Local: the file is streamed from disk with Content-Length, ETag, Last-Modified, Cache-Control: public, max-age=31536000, immutable, and Accept-Ranges: bytes; Range requests get a real 206 (with 416 for unsatisfiable ranges) and the filesystem stream is destroyed on client abort.

Rationale: redirects keep bytes off the CMS process and let the CDN/bucket do range handling and caching, while the local path stays a first-class citizen for self-hosting.

Configuration

All storage settings are public settings (schema: src/databases/public-config-schema.ts, editable in System Settings) — never in config/private.ts:

Setting Purpose
MEDIA_STORAGE_TYPE local | s3 | r2 | cloudinary (STORAGE_TYPES)
MEDIA_FOLDER Path prefix inside the bucket/disk root
MEDIA_BUCKET_NAME S3/R2 bucket
MEDIA_CLOUD_REGION S3/R2 region
MEDIA_CLOUD_ENDPOINT Custom endpoint (R2, MinIO) — SSRF-validated
MEDIA_CLOUD_PUBLIC_URL Public CDN/bucket origin used for the 302 redirect
MEDIA_ACCESS_KEY_ID / MEDIA_SECRET_ACCESS_KEY S3/R2 credentials
CLOUDINARY_CLOUD_NAME (plus key/secret) Cloudinary credentials
MEDIASERVER_URL Legacy fallback for MEDIA_CLOUD_PUBLIC_URL
MEDIA_SIGNED_URL_ENABLED Require ?sig=&exp= on tenant-scoped /files/ reads

Step-by-step setup for S3, R2, Cloudinary and a CDN front end: Cloud Storage & CDN guide.

Data Flow

Upload

  1. The upload pipeline builds a hashed relative path and calls saveFile(data, relPath) (media-storage.server.ts:79).
  2. saveFile delegates to getStorageAdapter().upload(...) — no branching at the call site.
  3. The adapter returns the stored path; the media record keeps a /files/... URL.

Access

  1. The browser requests /files/<tenant>/<hash>/original/file.jpg.
  2. Local storage: traversal guard (path.relative must stay inside the media root), tenant check against the request’s tenantId, and — when MEDIA_SIGNED_URL_ENABLED is on — HMAC validation via validateSignedMediaUrl() before any filesystem work.
  3. Cloud storage: 304 on a matching ETag, otherwise 302 to the public URL (bucket/CDN).

Delete

deleteFile(url) (media-storage.server.ts:87) strips the origin, any legacy prefix and the /files/ segment, then calls getStorageAdapter().remove(rel). Resized variants are removed through the same path, so a bucket and a disk behave identically.

Security Considerations

  • SSRF-guarded endpoints: validateS3Endpoint() (storage-adapters.ts:42) rejects non-HTTP protocols, loopback/private IP ranges (IPv4 and IPv6), cloud-metadata hosts and blocked hostnames before any S3 client is created.
  • No credentials in private.ts: bucket keys are database settings, read through getPublicSettingSync(); only bootstrap secrets live in config/private.ts.
  • Traversal & tenant isolation: the /files/ route rejects .. escapes and cross-tenant paths before touching storage.
  • Fail-closed signed URLs: when signing is enabled, a missing or invalid signature is a 403 (SIGNATURE_REQUIRED), and generateSignedMediaUrl() returns an empty string rather than an unsigned URL when no secret is configured.
  • SVG hardening: image/svg+xml responses carry their own restrictive Content-Security-Policy (script-src 'none', sandbox) plus X-Content-Type-Options: nosniff.

Troubleshooting

Symptom Check
CLOUD_CONFIG_ERROR (500) MEDIA_STORAGE_TYPE is not local but MEDIA_CLOUD_PUBLIC_URL / MEDIASERVER_URL is unset
Uploads succeed, /files/... 404s MEDIA_FOLDER mismatch between the write path and the served prefix
S3 endpoint blocked (private IP) MEDIA_CLOUD_ENDPOINT points at an internal host — the SSRF guard is doing its job
Signature failures after enabling signing Secret changed, or the URL was generated before MEDIA_SIGNED_URL_ENABLED was turned on
Stale URLs after a config change The URL cache is process-local with a 5-minute TTL — restart the server or wait out the TTL

Related

architecturecloud-storages3r2cloudinarymediastorage
Was this page helpful?