Cloud Storage Implementation

Technical documentation of the cloud storage architecture and implementation details

3/27/2026

12 min read

Overview

This document provides technical details about the cloud storage implementation in SveltyCMS, including architecture decisions, code changes, and integration points.

Architecture Decisions

1. Abstraction Layer

Decision: Create a unified abstraction layer (cloudStorage.ts) that handles all cloud storage operations.

Rationale:

Single source of truth for cloud storage logic
Easy to add new storage providers
Isolates cloud-specific code from business logic
Lazy-loads SDKs to reduce bundle size

2. URL Normalization

Decision: Store all media URLs in the database using the /files/ format, regardless of storage type.

Rationale:

Storage-agnostic database schema
Easy migration between storage types
Consistent API responses
Centralized URL resolution in the /files/ route

3. MEDIA_FOLDER as Prefix

Decision: Use the MEDIA_FOLDER setting as a path prefix in both local and cloud storage.

Rationale:

Consistent behavior across storage types
Allows organizing files within cloud buckets
Supports multiple CMS instances sharing one bucket
Prevents naming conflicts

4. Redirect vs Proxy

Decision: Use 307 redirects to cloud URLs instead of proxying files through the server.

Rationale:

Reduces server bandwidth and CPU usage
Leverages CDN caching and global distribution
Maintains HTTP method (important for range requests)
Faster content delivery to end users

Implementation Details

Files Created

`src/utils/media/cloudStorage.ts`

Purpose: Core cloud storage abstraction layer

Exports:

// Configuration
export function getCloudStorageConfig(): CloudStorageConfig;
export function isCloudStorage(): boolean;

// Path handling
export function getCloudPath(relativePath: string): string;
export function getCloudUrl(relativePath: string): string;

// Operations
export async function uploadToCloud(buffer: Buffer, relativePath: string): Promise<string>;
export async function deleteFromCloud(relativePath: string): Promise<void>;
export async function cloudFileExists(relativePath: string): Promise<boolean>;

Key Features:

Lazy SDK loading (only loads when needed)
Support for S3, R2, and Cloudinary
Automatic MEDIA_FOLDER prefix handling
Environment variable fallback for credentials
Comprehensive error logging

Implementation Details:

// S3/R2 Implementation
- Uses @aws-sdk/client-s3
- Lazy-loads SDK on first use
- Supports custom endpoints (R2)
- Public-read ACL for uploaded files
- HeadObject for existence checks

// Cloudinary Implementation
- Uses cloudinary SDK
- Lazy-loads on first use
- Automatic format optimization
- Unique public_id generation
- resource_type: 'auto' for all file types

Files Modified

`src/utils/media/mediaModels.ts`

Changes:

// Added
export type StorageType = "local" | "s3" | "r2" | "cloudinary";

`src/routes/files/[...path]/+server.ts`

Changes:

Added cloud storage check at the beginning of GET handler
Performance: Implemented ETag caching. Uses getMetadata to fetch the ETag from cloud storage.
Returns 304 Not Modified if the request’s If-None-Match header matches the cloud asset’s ETag, saving significant bandwidth and latency.
Constructs full cloud URL with MEDIA_FOLDER prefix and returns a 307 redirect if the cache is stale or missing.
Falls through to local file serving if not cloud (also supports ETags via file stats).

Flow:

GET /files/avatars/image.avif (If-None-Match: "etag-value")
↓
if (isCloudStorage())
  ↓
  metadata = await getMetadata('avatars/image.avif')
  ↓
  if (metadata.etag === "etag-value")
    ↓
    return 304 Not Modified (DONE)
  else
    ↓
    MEDIA_FOLDER = getPublicSettingSync('MEDIA_FOLDER') // 'cms-media'
    ↓
    cloudUrl = getCloudUrl('avatars/image.avif')
    // Returns: https://cdn.example.com/cms-media/avatars/image.avif
    ↓
    return new Response(null, {
      status: 307,
      location: cloudUrl,
      headers: { 'ETag': metadata.etag }
    })
else
  ↓
  Serve from local filesystem (with local ETag validation)

`src/utils/media/mediaStorage.ts`

Major Changes:

Imports: Added cloud storage functions

import {
  isCloudStorage,
  uploadToCloud,
  deleteFromCloud,
  getCloudUrl,
  cloudFileExists,
} from "./cloudStorage";


2. **`saveFileToDisk()`**: Now returns URL instead of void

   ```typescript
   // Before
   export async function saveFileToDisk(buffer: Buffer, relativePath: string): Promise<void>;

   // After
   export async function saveFileToDisk(buffer: Buffer, relativePath: string): Promise<string>;

   // Implementation
   if (isCloudStorage()) {
     const publicUrl = await uploadToCloud(buffer, relativePath);
     return publicUrl; // Cloud URL
   }
   // Local save...
   return `/files/${relativePath}`; // Local URL

deleteFile(): Cloud-aware deletion

if (isCloudStorage()) {
  // Extract relative path from URL
  let relativePath = url.replace(/^https?:\/\/[^/]+/i, "").replace(/^\/+/, "");
  if (relativePath.startsWith("files/")) {
    relativePath = relativePath.slice("files/".length);
  }
  await deleteFromCloud(relativePath);
  return;
}
// Local deletion...

getFile(): Cloud file retrieval via fetch

if (isCloudStorage()) {
  // Extract relative path...
  const cloudUrl = getCloudUrl(relativePath);
  const response = await fetch(cloudUrl);
  return Buffer.from(await response.arrayBuffer());
}
// Local read...

fileExists(): Cloud existence check

if (isCloudStorage()) {
  // Extract relative path...
  return await cloudFileExists(relativePath);
}
// Local check...

moveMediaToTrash(): Cloud deletion (no trash)

if (isCloudStorage()) {
  // Extract relative path...
  await deleteFromCloud(relativePath);
  return; // No trash folder in cloud
}
// Local trash logic...

saveAvatarImage(): Captures public URL

// Capture URL from saveFileToDisk
const publicUrl = await saveFileToDisk(svgBuffer, avatarUrl);

// Store relative path in DB
url: avatarUrl; // e.g., 'avatars/hash.avif'

// Return public URL
return publicUrl; // Cloud URL or /files/ URL

`src/routes/api/[...path]/handlers/system.ts`

Changes:

Consolidated save-avatar and delete-avatar into the system handler.
Now trusts saveAvatarImage() to return correct URL
Simplified logic by delegating to the unified dispatcher.

`src/routes/api/[...path]/handlers/media.ts`

Changes: Unified media handling for delete, exists, and get operations.

Changes: None needed - already use updated deleteFile(), fileExists(), and getFile() functions

Documentation Created

`docs/guides/cloud-storage.mdx`

Sections:

Overview and benefits
Architecture diagrams
Storage flow visualization
URL resolution explanation
Configuration for S3, R2, Cloudinary
Installation instructions
Migration guide
Best practices (security, performance, cost)
Troubleshooting
FAQ

`docs/api/Media_API.mdx`

Updates:

Added storage architecture section
Configuration examples
MEDIA_FOLDER usage explanation
URL pattern examples
Updated tags to include cloud storage

Configuration Schema

System Settings (Database)

interface MediaSettings {
  MEDIA_STORAGE_TYPE: "local" | "s3" | "r2" | "cloudinary";
  MEDIA_FOLDER: string; // Local path OR cloud prefix

  // Cloud-specific (optional)
  MEDIA_CLOUD_REGION?: string;
  MEDIA_CLOUD_ENDPOINT?: string;
  MEDIA_CLOUD_PUBLIC_URL?: string;
}

Environment Variables

# S3/R2 Credentials
MEDIA_ACCESS_KEY_ID=
MEDIA_SECRET_ACCESS_KEY=

# Cloudinary Credentials
CLOUDINARY_CLOUD_NAME=
CLOUDINARY_API_KEY=
CLOUDINARY_API_SECRET=

Data Flow

Upload Flow

1. Client uploads file via API
   ↓
2. API calls saveAvatarImage() or saveFileToDisk()
   ↓
3. Functions check isCloudStorage()
   ↓
4a. [Cloud] uploadToCloud() → Returns cloud URL
4b. [Local] fs.writeFile() → Returns /files/ URL
   ↓
5. Database stores relative path (e.g., 'avatars/image.avif')
   ↓
6. API returns public URL to client

Access Flow

1. Client requests /files/avatars/image.avif
   ↓
2. /files/ route handler checks isCloudStorage()
   ↓
3a. [Cloud] 307 Redirect to cloud URL
3b. [Local] Serve from filesystem
   ↓
4. Browser fetches from final location

Delete Flow

1. API calls deleteFile() or moveMediaToTrash()
   ↓
2. Functions check isCloudStorage()
   ↓
3a. [Cloud] deleteFromCloud() → Direct deletion
3b. [Local] Move to .trash folder
   ↓
4. Database record updated/removed

Testing Considerations

Unit Tests

Should test:

getCloudPath() correctly prepends MEDIA_FOLDER
getCloudUrl() constructs valid URLs
isCloudStorage() correctly identifies storage type
Path normalization in all functions

Integration Tests

Should test:

Avatar upload with local storage
Avatar upload with S3 storage
Avatar upload with Cloudinary storage
File deletion with all storage types
URL redirection from /files/ route
File existence checks

Manual Testing

Checklist:

Security Considerations

Credentials Management

Best Practices:

Store credentials in environment variables, not settings
Use IAM roles when possible (AWS)
Rotate keys regularly
Use minimum required permissions

Access Control

Implementation:

Public-read ACL for uploaded files (S3/R2)
Bucket policies restrict write access
API endpoints check user permissions
Tenant isolation maintained

URL Security

Considerations:

Cloud URLs are public (not signed by default)
For private files, implement signed URL generation
Consider presigned URLs for temporary access
CORS configuration on cloud storage

Performance Optimizations

Lazy Loading

Implementation:

let s3Client: S3Client | null = null;

async function getS3Client() {
  if (!s3Client) {
    const { S3Client } = await import('@aws-sdk/client-s3');
    s3Client = new S3Client({...});
  }
  return s3Client;
}

Benefits:

Reduces initial bundle size
Only loads SDKs when needed
Faster app startup

Caching

Strategy:

Cloud URLs cached in /files/ route
Browser caches cloud files (CDN headers)
Settings cached to avoid DB lookups
File existence checks can be cached

CDN Integration

Recommended:

S3: Use CloudFront distribution
R2: Use custom domain (automatic CDN)
Cloudinary: Built-in CDN included

Migration Path

Phase 1: Implementation (Completed)

✅ Create cloud storage abstraction
✅ Update media utilities
✅ Update API endpoints
✅ Update documentation

Phase 2: Testing

Unit tests for cloud storage functions
Integration tests for each storage type
Manual testing with real cloud accounts

Phase 3: Deployment

Deploy with local storage (backward compatible)
Gradual migration to cloud storage
Monitor for issues

Phase 4: Optimization

Implement signed URLs for private files
Add batch upload/delete operations
Optimize for large files (S3 multipart upload)
Add upload progress tracking

Streaming Upload

For multi-gigabyte uploads, src/utils/media/streaming-upload.ts provides a Web Streams-based multipart parser that processes file data without buffering the entire file into memory:

import { parseMultipartStream } from "@utils/media/streaming-upload";

await parseMultipartStream(request, {
  onFile: async (info) => {
    // info contains: name, filename, contentType, stream (ReadableStream)
    await uploadToStorage(info.stream, info.filename, info.contentType);
  },
  onField: (name, value) => {
    console.log(`Field: ${name} = ${value}`);
  },
});

The parser works with Bun, Node.js, and Edge runtimes using the standard Web Streams API. Boundary detection and chunked processing keep memory usage constant regardless of file size.

Storage Analytics

src/utils/media/storage-analytics.ts provides detailed insights into media usage across the entire system:

import { analyze, getInsights, getTrends } from "@utils/media/storage-analytics";

const files = await db.media.findMany({});
const breakdown = analyze(files);
// breakdown.byType   → { "image/png": { count, size, pct }, ... }
// breakdown.byFolder → { "uploads/2026": { count, size, pct }, ... }
// breakdown.byUser   → { userId: { count, size, pct }, ... }
// breakdown.byMonth  → { "2026-06": { count, size }, ... }
// breakdown.total    → { files, size, avgSize }

const insights = getInsights(breakdown);
// Returns actionable Insight[] cards: success/info/warning with descriptions
// Example: "PNG files consume 45% of storage. Consider WebP conversion."

const trends = getTrends(breakdown);
// Returns month-over-month Trend[] with growthPct and totalSize

Planned

Signed URLs — Temporary access tokens for private files with configurable TTL
S3 Multipart Upload — Full S3 multipart API with resume support for files >5MB
Backup Strategy — Automated nightly backups to secondary storage with retention policies
Hybrid Storage — Route recent files to fast storage, archive older files to cold storage

Troubleshooting Guide

Common Issues

Issue: Files upload but URLs are 404

Cause: MEDIA_FOLDER mismatch between upload and URL generation
Solution: Verify MEDIA_FOLDER setting is consistent

Issue: 307 redirects not working

Cause: isCloudStorage() returning false
Solution: Check MEDIA_STORAGE_TYPE setting

Issue: Access denied errors

Cause: Missing or incorrect credentials
Solution: Verify environment variables and IAM permissions

Issue: Files uploaded to wrong bucket path

Cause: getCloudPath() not being used
Solution: All cloud operations should use getCloudPath()

Debug Steps

Check Configuration:

const config = getCloudStorageConfig();
console.log("Storage Type:", config.storageType);
console.log("Media Folder:", config.mediaFolder);


2. **Test Cloud Operations**:

   ```typescript
   const testPath = "test/file.txt";
   console.log("Cloud Path:", getCloudPath(testPath));
   console.log("Cloud URL:", getCloudUrl(testPath));

Verify Credentials:

echo $MEDIA_ACCESS_KEY_ID
echo $MEDIA_SECRET_ACCESS_KEY

Check Logs:

grep "cloud storage" logs/app.log
grep "uploading to cloud" logs/app.log

Code Examples

Custom Upload Handler

import { uploadToCloud, getCloudUrl } from "@utils/media/cloud-storage";

export async function uploadCustomFile(file: File, folder: string) {
  const buffer = Buffer.from(await file.arrayBuffer());
  const filename = `${folder}/${Date.now()}-${file.name}`;

  const publicUrl = await uploadToCloud(buffer, filename);

  return {
    url: filename, // For database
    publicUrl, // For API response
  };
}

Batch Delete

import { deleteFromCloud } from "@utils/media/cloud-storage";

export async function batchDelete(files: string[]) {
  const results = await Promise.allSettled(files.map((file) => deleteFromCloud(file)));

  return results.map((result, i) => ({
    file: files[i],
    success: result.status === "fulfilled",
    error: result.status === "rejected" ? result.reason : null,
  }));
}

Signed URL Generation (S3)

import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import { GetObjectCommand } from "@aws-sdk/client-s3";

export async function getPrivateFileUrl(relativePath: string, expiresIn = 3600) {
  const config = getCloudStorageConfig();
  const s3Client = await getS3Client();

  const command = new GetObjectCommand({
    Bucket: config.bucket,
    Key: getCloudPath(relativePath),
  });

  return await getSignedUrl(s3Client, command, { expiresIn });
}

Conclusion

The cloud storage implementation provides a robust, scalable foundation for media management in SveltyCMS. The abstraction layer ensures consistency across storage types while maintaining flexibility for future enhancements.

Key Achievements:

✅ Multi-provider support (S3, R2, Cloudinary)
✅ Transparent to existing code
✅ MEDIA_FOLDER as universal prefix
✅ Backward compatible with local storage
✅ Performance optimizations (lazy loading, redirects)
✅ Comprehensive documentation

Next Steps:

Add unit and integration tests
Test with real cloud accounts
Monitor performance and costs
Gather user feedback
Implement advanced features (signed URLs, multipart uploads)