Overview

SveltyCMS implements a dual-layer caching system: L1 is an in-memory LRU cache (500K entries, sub-µs access), L2 is optional Redis for distributed deployments. The system includes stampede protection (distributed single-flight coalescing inside cacheService.get), negative caching, predictive warming, and stale-while-revalidate.

HTTP response cache (turbo GET + GraphQL + Zero-Copy Buffers)

Separate from cacheService entity keys: responseCache (src/services/cache/response-cache.ts) stores user-scoped pre-stringified API/GraphQL bodies + ETags for the Request Lane Router (HYPER_TURBO / handler-level GraphQL HIT). Keys are u:{userId}:… with deep-sorted variables; mutations invalidate res:* / GraphQL patterns. The service is a globalThis singleton so invalidations apply across Rolldown chunks / worker threads.

On HTTP mutations (handle-api-requests.ts), the L1 Map walk is synchronous (the next GET in-process cannot see a stale turbo body). L2 cacheService.delete / clearByPattern is fire-and-forget plus a 10ms batched flusher — those scans never sit on the mutation 200. SDK document writes (post-write.ts) clear collection + API patterns only; they do not evict cms:content_structure schema models.

• Zero-Copy Binary Byte Buffers: Pre-encodes and stores Uint8Array binary byte buffers directly in L1 RAM. handleTurboGet and the API dispatcher return new Response(resEntry.buffer) directly, completely bypassing V8 UTF-16 → UTF-8 string re-serialization and dropping cached p99 latency to <0.08ms.
• O(1) ByteLength Resolution: Uses resEntry.buffer.byteLength directly to avoid scanning multi-kilobyte JavaScript strings with Buffer.byteLength(rawBody, "utf-8").
• Zero-Allocation Read-Pipeline Keys: buildFindCacheKey and normalizeRelationshipFilter use dedicated hasExtraQueryKeys and isSingleKeyObject short-circuiting checks with lazy filter cloning, so the single-key fast path never clones the filter or the key input (self-assessed — not instrumented; the analogous zero-work fast paths are pinned by reference-identity assertions in tests/unit/**).
• Pre-Encoded Streaming Chunks: In streamingJsonResponse, static protocol markers ({"success":true,"data":[, ,, ],"error":"Stream interrupted"}) are pre-encoded singletons, eliminating thousands of per-item encoder.encode(",") allocations during dataset streaming.
• Conditional HTTP 304 Not Modified: Requests carrying matching If-None-Match headers short-circuit in handleTurboGet with HTTP 304 responses without body allocation or JSON reconstruction.
• List / GraphQL serve-stale: a write no longer deletes list/res:graphql turbo bodies. invalidateLocal marks them stale and drops only the written doc: keys (sibling findById stays). The next list GET / GraphQL query serves the previous bytes immediately (HIT latency). GET does not find()-refill. Fresh bytes come from a true miss, ?refresh=true, or TTL.
• Point-read L1 (first set admits): /:entryId turbo keys live in a dedicated 2,000-slot FIFO, separate from list/GraphQL L1. First set() admits (if (body) HIT else rebuild). A random id scan cannot FIFO-evict lists. Tests: tests/unit/hooks/collection-read-lane.test.ts, tests/unit/core/request-classifier-and-response-cache.test.ts.

Layout L0: ContentStore.getClientNodes(tenantId) memoizes JSON-sanitized navigation nodes per tenant and invalidates on updateVersion() / clear(). Root +layout.server.ts uses that cache instead of cloning the structure on every load. Schema lookups use limit: 1; CollectionsNamespace and count use a synchronous L1 getSync fast-path. In-memory sessions are a bounded Map (MAX_SESSIONS = 10000).

N+1 batching in GraphQL

GraphQL relation resolvers use a request-scoped BatchLoader (src/utils/server/batch-loader.ts, wired in src/routes/api/graphql/loaders.ts) that coalesces N+1 findByIds calls into single WHERE _id IN (...) / $in queries per tick. Lazy per-request instantiation keeps the loader cost at zero until a relation is actually resolved.

Architecture

Dual-Layer Strategy & Distributed Multi-Node Cluster

Layer 1: In-Memory LRU Cache → <0.001ms (L1 hit, zero allocation)
Layer 2: Redis (Optional)     → <0.5ms (distributed cluster, edge sync, cross-node invalidation)
Layer 3: Database (Persistent) → PostgreSQL, MariaDB, MongoDB, SQLite (0.042-3.261ms)

Next-Gen 2026 Cache Features

1. GraphQL Automatic Persisted Queries (APQ)

SveltyCMS natively implements the Apollo APQ open standard protocol (extensions: { persistedQuery: { sha256Hash: "<hash>" } } & ?extensions=... / ?hash=...):

• Bandwidth Reduction: Clients send compact 64-bit/SHA-256 hashes instead of multi-kilobyte GraphQL query documents.
• Zero-Parse AST Hits: Queries registered in the cacheService APQ registry (apq:${hash}) bypass Yoga query parsing and AST construction entirely.
• Fail-Open Handshake: If an APQ hash is not yet registered, SveltyCMS returns HTTP 200 with standard PERSISTED_QUERY_NOT_FOUND, allowing the client to automatically retry and register the full query.

2. AI-Driven Speculative Pre-Warming

The routeResourceStateMachine connects directly with the Behavioral Learning Engine (predictNextPath):

• When users navigate the CMS, transition probability heatmaps track subsequent navigation patterns.
• speculativePrewarm() selects the highest-scoring Markov transition from the decaying heat map and triggers non-blocking, single-flight background pre-warming of target endpoints before the user even hovers over the next link.
• Zero-Wait Navigation: Response caches are primed in RAM before click events occur.

3. Pre-Compressed Binary Byte Serving (Zero-CPU)

• For API and GraphQL responses $>1\text{ KB}$, responseCache asynchronously computes and caches pre-compressed Brotli and Gzip binary chunks (Uint8Array).
• On TURBO-HIT and GraphQL cache hits, matching Accept-Encoding requests receive pre-compressed byte arrays directly, bypassing on-the-fly CPU compression during traffic spikes.

Key: L1 is In-Memory, L2 is Redis (Optional)

The L1 cache is a local FastLRU instance (not Redis). Redis is the strictly optional L2 layer for distributed deployments. Even without Redis (USE_REDIS=false or absent), the entire cache system is fully operational with sub-µs L1 hits across all 4 database adapters. When enabled, Redis synchronizes cluster nodes and provides distributed caching.

Redis L2 Performance Architecture (October 2026 Enhancements)

• Strict Optionality: Redis is never a boot or runtime prerequisite. If Redis is disabled, down, or does not connect, the system automatically falls back to in-memory L1 operation without degraded functionality.
• TCP Socket Optimization (TCP_NODELAY + Keep-Alive): Sockets are configured with noDelay: true and keepAlive: 5000. This disables Nagle’s algorithm packet coalescing, eliminating artificial 5–40ms tail latency jitter on small JSON payload reads (<1.5KB).
• Non-Blocking Invalidation (UNLINK): Cache evictions and tag clears pipeline Redis UNLINK commands in multi.exec() batches rather than blocking DEL. This frees memory asynchronously on background threads without freezing Redis event loops under bulk tag sweeps.
• Tag-Based $O(1)$ Invalidation: Invalidation operates on tag sets (tag:{tenant}:{tag}) rather than slow wildcard SCAN sweeps.
• Atomic Epoch Versioning (INCRBY): Collection and global version bumps use atomic Redis INCRBY commands instead of race-prone read-modify-write patterns, preventing stale content delivery during concurrent updates across cluster nodes.
• Batch Read Probing (mGet): High-traffic relational pipelines (such as populate-resolver.ts) probe missing IDs in Redis via cacheService.getMany() using a single pipelined mGet(keys) command before falling back to database findMany queries.

L1 Memory Containment & Bounded Footprint

The L1 tier (cacheService.l1) is backed by the repo-native, dependency-free FastLRU (src/utils/native-utils.ts) rather than an external LRU library. FastLRU covers the full L1 surface: byte-budget eviction (maxSize + maxEntrySize + sizeCalculation), per-entry TTL, a dispose hook, and the read hints updateAgeOnGet / updateRecencyOnGet.

• Byte Budget (maxSize): Defaults to 128 MB (configurable via CACHE_L1_MAX_BYTES). Prevents unbounded heap growth regardless of total entry count.
• Entry Cap (max): Defaults to 200,000 (configurable via CACHE_L1_MAX_ENTRIES).
• Single-Entry Ceiling (maxEntrySize): Defaults to 16 MB (configurable via CACHE_L1_MAX_ENTRY_BYTES), preventing oversized individual payloads from evicting hot cache entries.
• $O(1)$ Size Estimator: Accurately sizes pre-stringified response bodies and employs a flat overhead estimate for structured objects without recursive traversal overhead.
• ENTRY read “peek” (updateRecencyOnGet: false): CacheService.get(..., CacheCategory.ENTRY) reads without moving the hit to the most-recently-used position and without refreshing its TTL (updateAgeOnGet: false), so uniformly-random point reads cannot displace the resident hot list/GraphQL working set. CONTENT reads still promote and refresh on every hit; getSync also takes the no-refresh path.

Response-cache L1 tiers: exact byte accounting (2026-09-22)

The HTTP response cache (src/services/cache/response-cache.ts) keeps two L1 tiers above cacheService.l1: a 2,000-slot list/GraphQL FIFO and a 2,000-slot point-read FIFO. Each has an entry cap and a byte budget (SVELTY_L1_MAX_MB, default 64; SVELTY_L1_POINT_MAX_MB, default 16) — an entry cap alone does not bound memory, because a list?limit=100 body is 50–100 KB (2,000 of those is ~200 MB).

Every mutation of a tier now runs through exactly one insert (putEntry) and one delete (removeEntry) primitive, so the maintained byte counters cannot drift from the store:

• The L2-refill paths (get() / getAsync() promoting a cacheService entry into L1) insert with accounting. Before the fix they inserted silently, so a tier filled by refills reported 0 bytes while holding every entry — the byte budget simply did not bind, and only the 2,000-entry cap stopped it (measured on the pre-fix code: 40 refilled entries, 960,000 bytes in the store, counter 0).
• Deletes decrement: invalidate, invalidateAll, invalidateLocal doc-scoped drops, and TTL expiry at read time.
• Enforcement runs after the insert, so size ≤ cap and bytes ≤ budget hold strictly once a tier is at its ceiling (a single entry larger than the whole budget is dropped rather than cached forever).
• Budgets are re-read when their env value changes, and responseCache.getL1ByteStats() returns the counters plus an exact recount for diagnostics and tests.

Tests: tests/unit/core/response-cache-l1-budget.test.ts (insert-above-budget shrinks the tier on both tiers, refill accounting, delete-path accounting, counter == recount after a mixed torture sequence).

Content System: L0 + L1 + L2 (Three Tiers)

The content scanner adds a process-local L0 on top of the dual-layer cache:

L0: Mtime Tree + _schemaCache (engine.server.ts) + _clientNodesCache (contentStore) → skip fs/import and layout serialization
L1: cacheService LRU (schema:*, navigation:tree:*)       → sub-µs metadata hits
L2: Redis MGET/MSET (optional)                            → cross-node schema + nav snapshots

Helpers: src/content/engine.server.ts (setSchemaCacheEntry, invalidateSchemaCache, invalidateNavigationCache, notifyContentUpdate).

Bearer credential auth (website tokens + API keys)

Hash-keyed entries — never store plaintext bearer tokens in L1/L2:

Helpers: src/databases/auth/credential-auth-cache.ts (setWebsiteTokenAuthCache, getWebsiteTokenAuthCacheSync, invalidateWebsiteTokenAuth, recordWebsiteTokenAuthMiss).

Negative misses: cacheService.isNegativeHit() / recordMiss() — Bloom filter per cache-system (not plaintext module bloom).

Auth middleware: handle-authentication.ts hashes once (hashCredentialSha256HexSync for website tokens / hashApiKey for API keys), looks up by hash via getByTokenHash, caches with CacheCategory.SESSION.

Session user snapshots are credential-free: the session cache (L1 LRU + L2 Redis) and the in-memory/Redis session store strip credential material (password, totpSecret, backupCodes, resetToken, googleRefreshToken, twoFactorTrustedDevices) at every write boundary (toSafeSessionUser in src/databases/auth/session-user.ts). The zero-allocation fast path returns the original reference when no sensitive field is present. Password-verifying endpoints fetch a fresh user from the DB instead of reading the snapshot.

Delete propagation (untagged entries): cacheService.delete() publishes the logical key as an invalidation pattern (plus the key as a tag). The pub/sub subscriber applies both channels — tags for the L1 tag index and the pattern for untagged entries (settings cache, session entries, user counts) — so a delete on one node purges the remote L1 immediately instead of lingering until LRU eviction. This closes a cross-node staleness hole for untagged keys in Redis deployments.

L2 contract suite: tests/unit/databases/cache-service-l2-contract.test.ts runs the full distributed path (cross-instance hits, write batching, stampede locks, tenant-scoped tag sets, pattern scans, pub/sub) against an in-memory FakeRedis on every run, and against a real Redis when TEST_REDIS_URL is set — same contract, two drivers.

Permission Cache (RBAC)

hasPermissionWithRoles() consults this cache on every permission check (after the admin fast-path), keyed by {userId}:{permissionId}:{sortedRoleIds}. On a miss it evaluates via the memoized role bitsets and stores the boolean result:

Helpers: src/databases/auth/permissions.ts (hasPermissionWithRoles, invalidatePermissionCache), src/utils/security/permission-cache.ts (PermissionCache).

Invalidation policy:

• Per-user: Called after Auth.updateUser() — clears cached permission checks for the modified user so RBAC changes take effect immediately.
• Global: Called after AuthNamespace.updateRoles() — clears all entries since any role change can affect many users’ cached checks.
• Config import: ConfigService.performImport() (config sync, /api/config/import, GraphQL configApply, LocalCMS performImport) writes roles directly through the DB adapter. It now invalidates the roles list cache and the permission cache whenever a role entity is upserted or deleted, so imported role changes take effect immediately instead of after the 1h roles-cache TTL.
• Turbo auth: Turbo contexts (src/hooks/handle-turbo-get.ts, 60s TTL) cache per-session user/roles/bitset and skip session re-validation on cacheable API GETs, so stale permission state can persist up to 60s after a change. Invalidation coverage:
  • clearTurboAuthCache() — full clear after AuthNamespace.updateRoles() (a role change can affect any user), ConfigService.performImport() role imports/deletes, and testing-API role mutations.
  • invalidateTurboAuthForUser(userId) — per-user clear after Auth.updateUser() (role/status edits) and Auth.deleteUser(); batchAction() block/delete/unblock already clears the affected users.

Roles List Cache (Authorization Hook)

The handleAuthorization hook maintains a tenant-scoped list of all roles (src/hooks/handle-authorization.ts), used to hydrate locals.roles for page loaders and permission checks:

Invalidation triggers: AuthNamespace.updateRoles() (admin UI), the testing API role mutations, and ConfigService.performImport() for imported/deleted role entities. Any role mutation path that bypasses AuthNamespace.updateRoles must call invalidateRolesCache() — otherwise stale role lists can surface as incorrect permission-denied errors for up to 1 hour.

Scheduled Publish Cache Invalidation

The background job scheduler (scheduled-jobs.ts) now invalidates the collection cache after every successful scheduled publish. This prevents stale relation data from being served after entries transition from draft to publish status.

Distributed Rate Limiter & Token-Bucket L2 Integration

The rate limiting middleware (src/hooks/handle-rate-limit.ts) integrates with the Redis L2 layer to coordinate request throttling across clustered instances:

• Atomic Lua Engine: Redis L2 executes an atomic Lua script that computes continuous token replenishment (refillRatePerMs = capacity / windowMs) and decrements available tokens in a single network round-trip.
• Fail-Safe Circuit-Breaker Fallback: Network queries to Redis L2 are bounded by a strict 15 ms timeout. If Redis is unreachable or exceeds latency limits, an internal circuit breaker trips with a 5-second backoff, instantly diverting throttling decisions to process-local in-memory Token Buckets (< 2 µs). This ensures cluster availability without dropping requests or increasing user-perceived latency.
• Zero-Allocation Session Cache Peeking: Privilege-tier multipliers (Admin $10\times$, Authenticated $2\times$, Anonymous $1\times$) are evaluated synchronously against the session cache (peekSessionFromCacheSync) in handle-authentication.ts without triggering async microtask ticks.

Field-Level Encryption & Ciphertext Cache Isolation

In accordance with our zero-trust data protection model (docs/reference/security/field-level-encryption.mdx):

• Ciphertext Storage: L1 RAM and L2 Redis caches store encrypted envelopes (v1:iv:tag:ciphertext) for all fields flagged with encrypt: true. Unencrypted plaintext is never persisted in shared cache infrastructure.
• Read-Path Decryption Cloning: Upon a cache hit, the read pipeline decrypts field values into an isolated shallow clone of the document before returning it to the SDK/API caller, preventing plaintext leaks back into the cache layer.

// After scheduled publish, in scheduled-jobs.ts:
await cacheService.invalidateCollection(collectionName);

Without this invalidation, GraphQL queries using publicationFilter would continue serving cached results that don’t reflect the newly published state.

Related: Schedule Modal Component

Cache Stampede Protection (Single-Flight + Distributed Locks)

When a popular cache key expires, only ONE request rebuilds it. All other concurrent requests wait for the result. This prevents the “thundering herd” problem where 100 simultaneous requests all hit the database for the same missing key.

// Internal: pendingRequests Map coalesces concurrent misses
if (this.pendingRequests.has(fullKey)) {
  return this.pendingRequests.get(fullKey); // Wait for the winner
}

// Internal: lockedKeys Map for distributed coordination (Redis-backed)
lockOwner = await this.acquireLock(fullKey, 500);
if (!lockOwner) {
  await this.waitForCache(fullKey, 1000); // Another node is fetching
}

Negative Caching (Bloom Filter + Exact TTL Map) {#negative-caching-bloom-filter—exact-ttl-map}

A Bloom filter prevents “cache miss storms” — repeated requests for non-existent keys (404s, broken links) bypass the database entirely. Verified 2392x speedup for repeated misses.

The Bloom filter is a prefilter only. A positive is authoritative only while a bounded exact map still holds a live entry for that full key (NegativeCacheManager, 60 s TTL, 20 000 keys, oldest-first eviction). This removes the Bloom false-positive rate from the 404 path: a key that was never recorded as missing can never be rejected.

// Bloom says "maybe"; the exact map decides.
if (!this.negativeInvalidated.has(fullKey) && this.negativeBloom.has(fullKey)) {
  const entry = this.exact.get(fullKey); // TTL-checked, tag-indexed
  if (entry && entry.expiresAt > Date.now()) return null; // known missing
}

Point reads (findById): a confirmed miss is recorded under the exact point-read cache key (including the publication suffix) and tagged doc:<collection>:<id>. A later create/update/delete clears the tag through clearByTags() → invalidateByTags(), so a freshly written entry is never hidden by a stale 404 marker. Only successful adapter queries can record a miss — transient adapter errors do not poison the cache — and system scopes, disableErrors existence probes, and bypassCache callers never use it.

List ID fast path: a bare _id lookup (no status constraint) shares the same negative key. Status-constrained lookups keep their own publication suffix and are not admitted.

Prefix-Bucketed Invalidation (O(1) Clearing)

Instead of scanning all cache keys, the system maintains a prefixMap that groups keys by namespace. Clearing collection:posts:* only iterates over keys in that bucket.

Predictive Cache Warming

On startup, the cacheWarmingService pre-loads frequently-accessed paths based on historical patterns.

Stale-While-Revalidate (SWR)

When a cached entry is stale (TTL expired but within stale window), the stale value is returned immediately while a background refresh updates the cache. Eliminates cache-miss latency entirely for frequently-accessed content.

const result = await cacheService.getOrSetSWR(
  "collection:posts:published",
  async () => await db.findMany("posts", { status: "published" }),
  60_000, // TTL: 1 minute
  300_000, // Stale: 5 minutes (serve stale + refresh)
);

Collection List Queries (entry-list / CollectionService) {#collection-list-queries-entry-list—collectionservice}

Editorial list views are cached through CollectionService.getCollectionData() using getOrSetSWR, not raw get/set.

// CollectionService (simplified)
const queryHash = hashQueryPayload({ filter, search, sort });
const cacheKey = buildCollectionQueryCacheKey({ collectionId, page, pageSize, queryHash, language, tenantId, userId });

return cacheService.getOrSetSWR(
  cacheKey,
  () => loadFromDb(...),
  60_000,   // fresh
  300_000,  // stale-while-revalidate
  tenantId,
  CacheCategory.COLLECTION,
  ["collection", `collection:${collectionId}`],
);

Short-lived crud.count cache (30s)

List badges and pagination totals call dbAdapter.crud.count frequently. A thin proxy wraps every adapter after the tenant guard:

Hits serve L1 (and L2 if Redis is configured) without re-running SQL/Mongo COUNT. Misses still execute the engine path (exact vs estimatedDocumentCount / table statistics). Tenant isolation is enforced by cacheService tenant namespacing — unscoped whole-table estimates are never shared with tenant-scoped keys.

findPage itself is not count-cached; when total: "exact"|"auto" is requested, the parallel count still goes through this wrapper. Prefer total: "none" + hasMore for scroll UIs.

Related: Collection Filtering Platform · Performance Architecture · entry-list · Content API filters · Data Operations

Cache Categories

The CacheCategory enum (src/databases/cache/types.ts) is the single source of truth: API, AUTH, COLLECTION, CONTENT, GENERAL, MEDIA, SCHEMA, SESSION, SYSTEM, THEME, USER, WIDGET. When a caller passes ttl = 0, cacheService.set() applies the adaptive per-category default from CATEGORY_TTL_SECONDS (src/databases/cache/cache-service.ts):

Explicit ttl > 0 always wins over the category profile. Categories in the enum without a profile (e.g. CacheCategory.GENERAL) fall back to general (300s).

Performance Metrics

Cache Hit Rates

• User Sessions: 92%
• Static Content: 95%
• API Responses: 88%
• Database Queries: 85%

Last Updated: 2026-08-09 (RBAC permission cache wired into hasPermissionWithRoles; turbo-auth invalidation on all role/user mutation paths; architecture + invalidation mermaid diagrams; cache categories table aligned to the real CacheCategory enum + CATEGORY_TTL_SECONDS profiles)