Skip to content

Documentation

E2E Coverage Matrix

A comprehensive route-based analysis of E2E coverage and identified gaps in SveltyCMS.

6/19/2026
11 min read Edit on GitHub

Generated: 2026-06-19 (spec count refreshed 2026-07-31) Total routes: 9 (matrix scope below) | Total E2E specs: 43 on disk | CI projects: 17 (+ wizard + auth-setup)

1. /setup — Setup Wizard (24 components, 6 tested)

Feature Component Tested? Test File
DB type selection (sqlite/mongo/maria/postgres) database-config.svelte setup-wizard.spec.ts
DB connection test (success) connection-status.svelte setup-wizard
DB connection test (bad host) connection-status.svelte setup-wizard
Admin account creation (valid) admin-config.svelte setup-wizard (full flow)
Admin password mismatch admin-config.svelte setup-wizard
SMTP invalid host error email-config.svelte setup-wizard
Preset/template selection preset-selector.svelte
System config (site name, media folder) system-config.svelte
Feature Component Tested? Test File
SMTP test email (valid config) email-config.svelte
Review step verification review-config.svelte
Back navigation (return to previous) setup-navigation.svelte
OAuth config during setup setup server
Demo mode detection +page.svelte
Multi-DB provisioning (Mongo/Maria/PG) full flow Commented out
Config write permissions error write-private-config.ts
Cookie consent during setup CookieConsent plugin ⚠️ Flaky

Coverage: 6/16 features (37%)


2. /login — Authentication (4 components, 5 specs)

Feature Test File Status
Login with valid credentials login.spec.ts
Logout login.spec.ts
Sign Up (first user) login/signup.spec.ts ⚠️ Broken
Forgot password flow login/signup.spec.ts ⚠️ Broken
OAuth button visibility login/oauth.spec.ts ⚠️ Skips
OAuth redirect mock login/oauth.spec.ts
OAuth error handling login/oauth.spec.ts
Axe accessibility audit accessibility.spec.ts ⚠️ 1 violation
Keyboard focus trap accessibility.spec.ts ✅ Fixed
RTL layout audit accessibility.spec.ts ⚠️ 1 violation
2FA/TOTP flow
SAML SSO login
Rate limiting (5 failed attempts)
Account lockout (15 min)
Session timeout warning
“Remember me” persistent session

Coverage: 10/16 features (62%)


3. /user — User Profile & Management (5 specs + control map)

Full 1:1 control map: routes/user.mdx (updated 2026-07-18). Product fixes: batch self-delete, CSRF on remotes/GDPR, permissions display, sessions UI.

Feature Test File Status
View profile user/account-smoke.spec.ts
Edit avatar user/profile.spec.ts ✅ (fixture testthumb.png committed + hard-required)
Delete avatar user/profile.spec.ts ✅ serial (button hard-asserted; no soft-skip)
Edit user details user/profile.spec.ts
Appearance settings link user/profile.spec.ts
Registration token workflow user/profile.spec.ts ⚠️
Toggle token/list visibility user/profile.spec.ts
Read/edit user (admin) user/management.spec.ts
Delete/block/unblock (admin) user/management.spec.ts
Invite user via email user/management-invite.spec.ts
Auth preference toggles user/complete-coverage.spec.ts
Admin table search/sort user/complete-coverage.spec.ts
Privacy modal open/close user/complete-coverage.spec.ts ✅ (privacy-data-btn)
Active sessions refresh user/complete-coverage.spec.ts
Password change (full) user/p0-journeys.spec.ts ✅ editor verify→set→re-login
Non-admin profile user/p0-journeys.spec.ts ✅ AdminArea hidden
Token edit/delete (seeded) user/p0-journeys.spec.ts ✅ no soft-skip
2FA setup / enroll user/coverage-100.spec.ts ✅ USE_2FA + TOTP
GDPR export / anonymize user/coverage-100.spec.ts ✅ download + secondary wipe
RTC prefs persistence user/coverage-100.spec.ts ✅ toggle + reload
Permissions badges user/coverage-100.spec.ts ✅ admin list
Pagination (11+ users) user/coverage-100.spec.ts ✅ bulk-create-users
Identity email/role/id user/coverage-100.spec.ts

Coverage: 100% of /user control map — see routes/user.mdx


3b. /dashboard — Shell (widget catalog varies per install)

Full control map: routes/dashboard.mdx.
Never assert a fixed widget list — core + plugins differ by install.

Feature Test File Status
Page shell + title dashboard/dashboard.spec.ts
Empty state or widget grid same ✅ install-dependent
Add widget menu same ✅ if catalog has free widgets
Widget search filter same ✅ nonsense → no results
Reset layout same
AI toggle present same ✅ no live MCP required
Plugin slot same ✅ attached
Dashboard APIs (health/metrics/…) integration dashboard.test.ts
Drag-reorder E2E 🟢 Low (keyboard reorder exists; pointer DnD optional)

Coverage: shell 100% (per-widget UI content = opt-in, not part of shell contract)


4. /config/collectionbuilder (4 specs)

Feature Test File Status
Shell: open builder + new editor builder.spec.ts shell
Golden: schema → entry → API builder.spec.ts golden
Collections list auth integration collection-structure
Structure persistence integration collectionbuilder/*
Pure tree/slug utils unit collectionbuilder-utils
Empty state / multi-spec sprawl demoted (ADR: one golden)
Field reordering / relation / templates not P0

5. /config/* — System Configuration (8 specs)

Route Feature Test File CI Project Status
/config/system-settings Cache repair system/settings.spec.ts system
/config/access-management Roles, permissions, save gating config/access-management.spec.ts config-routes
/config/access-management Permission matrix (system) system/permissions.spec.ts permissions
/config/access-management RBAC (admin/dev/editor) system/rbac.spec.ts rbac
/config/webhooks List, create form, delivery logs config/webhooks.spec.ts config-routes
/config/automations Workflow builder smoke config/automations.spec.ts config-routes
/config/migration Smart migration smoke config/data-management.spec.ts config-routes
Route Feature Test File CI Project Status
/config/sync Sync settings smoke config/data-management.spec.ts config-routes
/config/redirects Create / search / delete config/redirects.spec.ts config-routes
/config/monitor Stats + service health config/operations.spec.ts config-routes
/config/queue Stats + status filter URL config/operations.spec.ts config-routes
/config/webhooks Modal + logs shell config/webhooks.spec.ts config-routes
/config/extensions Tabs + plugins grid config/extensions.spec.ts config-routes
/config/automations List + new builder link config/automations.spec.ts config-routes
/config/sync Tabs + status shell config/data-ops.spec.ts config-routes
/config/trash Empty or restore table config/data-ops.spec.ts config-routes
/config/extensions Extension list config/operations.spec.ts config-routes
/config/design-system Sole Design System / Appearance workspace config/design-system.spec.ts, appearance.spec.ts, appearance-golden.spec.ts appearance
/config/workflows

6. /admin/tenants — Multi-Tenant Management (1 spec)

Feature Test File CI Project Status
Tenant list load admin/tenants.spec.ts admin
Quota headers/empty admin/tenants.spec.ts admin
Create tenant
Edit tenant
Cross-tenant isolation

7. /dashboard — Shell 100% (widget catalog install-specific)

Feature Test File CI Project Status
Page shell + toolbar dashboard/dashboard.spec.ts dashboard / chromium
Empty state or grid same ✅ never fixed widget list
Add / search / reset same
AI toggle present same
Dashboard APIs integration dashboard.test.ts
Pointer drag-reorder E2E dashboard/dashboard.spec.ts ✅ ≥2 widgets, header band drag
Keyboard Ctrl+Arrow reorder same
Per-widget visual content ⚪ out of shell scope (varies per install)

8. /mediagallery — Shell strong (see routes/mediagallery.mdx)

Feature Test File CI Project Status
Page shell + toolbar mediagallery/mediagallery.spec.ts media
Search + type filter mediagallery/mediagallery.spec.ts media
Grid/table view toggle mediagallery/mediagallery.spec.ts media
Local file upload mediagallery/mediagallery.spec.ts media
Delete media mediagallery/mediagallery.spec.ts media
Image editor (crop/focal) mediagallery/image-editor.spec.ts media
Image editor save mediagallery/image-editor.spec.ts media ✅ manipulate API
Create / open folder mediagallery/folders-bulk.spec.ts media
Bulk select / download / Del mediagallery/folders-bulk.spec.ts media
media:read gate unit mediagallery-page-server
Selection + breadcrumb move mediagallery/move-remote.spec.ts media
Remote URL upload page mediagallery/move-remote.spec.ts media

9. /language — Partially tested

Feature Status
Language switching dropdown ⚠️ data-testid missing
RTL layout (Arabic/Hebrew) ⚠️ Axe violation
Translation coverage

Summary

Route Features Tested Coverage
/setup 16 6 37%
/login 16 10 62%
/user 15 9 60%
/config/collectionbuilder 15 10 66%
/config/* (other) 15 11 73%
/admin/tenants 5 2 40%
/dashboard 8 7 88% shell (widget content N/A)
/mediagallery 8 4 50%
/language 3 1 33%
TOTAL 105 63 60%

Critical gaps: Setup presets, 2FA, SAML, rate limiting, image editor, dashboard drag-reorder.

See Next Steps for prioritized action items.

CI Playwright Projects (17 + wizard + auth-setup)

Project Spec(s) Depends on
wizard setup/setup-wizard.spec.ts
auth-setup auth.setup.ts, login/login.spec.ts
signup user/account-smoke.spec.ts auth-setup
content (merged into builder golden) auth-setup
system system/settings.spec.ts auth-setup
a11y login/accessibility.spec.ts auth-setup
branding login/branding.spec.ts auth-setup
Project Spec(s) Depends on
visual-regression admin-theme/visual-regression.spec.ts auth-setup
rbac system/rbac.spec.ts auth-setup
language system/language.spec.ts auth-setup
users user/profile.spec.ts, user/management.spec.ts auth-setup
builder collection-builder/builder.spec.ts only (shell + golden) auth-setup
permissions system/permissions.spec.ts auth-setup
Project Spec(s) Depends on
firstuser login/signup.spec.ts, login/oauth.spec.ts
config-routes config/access-management, webhooks, automations, data-management, operations auth-setup
admin admin/tenants.spec.ts auth-setup
dashboard dashboard/dashboard.spec.ts auth-setup
appearance config/appearance.spec.ts, config/design-system.spec.ts auth-setup
media mediagallery/mediagallery.spec.ts auth-setup

Next Steps

testinge2ecoverageplaywright
Was this page helpful?