Overview

SveltyCMS uses Playwright for End-to-End (E2E) testing. Our strategy is optimized for speed and reliability:

1. Wizard Coverage: The setup wizard is tested directly in the dedicated wizard project so fresh-install flows stay PR-gating.
2. Auth Bootstrap: The auth-setup project provisions authenticated state once and persists it for the maintained app smoke projects.
3. Maintained Smoke Scope: PR-gating browser coverage includes all 12 E2E projects running in parallel — from auth and accessibility to visual regression, branding, and multi-tenant management.
4. Black-Box Safety: State setup happens through /api/testing plus TEST_API_SECRET; browser flows still navigate the real app and assert real redirects.

Running Tests

bun run test:e2e is the local convenience entrypoint. It starts the dev server and runs Playwright against 127.0.0.1.

# Run the maintained local E2E flow
bun run test:e2e

# Run the setup wizard contract
bun x playwright test --project=wizard  # (the wizard project targets tests/e2e/routes/setup/setup-wizard.spec.ts)

# Generate auth state for downstream smoke suites
bun x playwright test --project=auth-setup

# Run maintained app smoke suites
bun x playwright test --project=signup
bun x playwright test --project=content
bun x playwright test --project=system
bun x playwright test --project=config-routes
bun x playwright test --project=admin
bun x playwright test --project=dashboard
bun x playwright test --project=appearance
bun x playwright test --project=media

🧠 Smart Testing

We also provide a smart runner for local iteration:

bun run scripts/test/test-smart.ts

• Fresh Environment: Biases toward setup coverage first.
• Configured Environment: Runs the suites that match the working tree.
• Local Helper Only: GitHub Actions does not depend on the smart runner for PR gating.

Writing Tests

1. Use the Auth Helper or Stored State

Avoid repeating the login UI in every spec. Use loginAsAdmin from helpers/auth.ts for targeted tests, or depend on the auth-setup project when a suite should consume persisted browser state.

import { test, expect } from "@playwright/test";
import { loginAsAdmin } from "./helpers/auth";

test("Admin can access dashboard", async ({ page }) => {
  // Logs in and waits for navigation to dashboard
  await loginAsAdmin(page);

  // Your test logic
  await expect(page.getByText("Dashboard")).toBeVisible();
});

2. Database State & Worker Isolation

Playwright uses database-per-worker isolation to eliminate SQLite locking issues and keep E2E runs reproducible.

• Unique Database per Worker: Each Playwright worker operates on its own dedicated SQLite file (e.g., cms_worker_1.db).
• Header-Based Routing: The system uses the x-test-worker-index header to automatically route requests to the correct isolated database.
• Secret Handshake: /api/testing calls must include the shared TEST_API_SECRET.
• Safety Rule: config/private.test.ts is the source of truth. Any config/private.ts present in CI is only a temporary mirror of the test config and must never point at live data.

3. Use data-testid Selectors

For robust tests that don’t break when CSS classes change, always use data-testid attributes for interactive elements.

In Svelte Component:

<button data-testid="signin-submit">Sign In</button>
<input data-testid="signin-email" />

In Playwright Test:

await page.getByTestId("signin-email").fill("admin@example.com");
await page.getByTestId("signin-submit").click();

Avoid using:

• CSS classes (e.g., .btn-primary)
• Text content (e.g., text="Sign In") unless checking localized text specifically.
• XPath selectors

4. CI/CD

GitHub Actions runs Playwright in staged projects:

• wizard → firstuser → auth-setup: validates fresh setup, signup, then provisions and uploads tests/e2e/.auth/.
• chromium: fans the remaining specs out across 8 disjoint --grep groups (Config & System, Builder & Content, Users & Profile, RBAC & Enforcement, User Journeys, Media & Dashboard, Auth & Branding, Admin & Catch-All).

The partition is total: every spec must match exactly one group. A spec matching none silently never runs; a spec matching two runs twice. Verify with node .github/workflows/e2e-matrix.ts when adding a spec, and keep route-based names (avoid a second group keyword in the filename or test.describe title).

Failure artifacts include Playwright HTML reports, traces, video, and preview server logs.

5. Zero-Error Contract

Journeys that represent a real editorial flow attach the shared trap from tests/e2e/helpers/zero-error-guard.ts:

import { assertNoZeroErrors, attachZeroErrorGuard } from "../../helpers/zero-error-guard";

test.beforeEach(async ({ page }) => {
  attachZeroErrorGuard(page);
});

test.afterEach(async ({ page }) => {
  assertNoZeroErrors(page);
});

It fails the test on an uncaught pageerror, a non-allowlisted console.error, or a failed same-origin request. The allowlist is deliberately narrow (favicon/icon-CDN noise, net::ERR_ABORTED, ResizeObserver loops, expected 401/403/404 resource loads). Do not widen it to hide a product bug — fix the error or document the exception next to the pattern.

6. Enterprise Config Route Specs

The following enterprise smoke tests cover config and admin routes:

• config/access-management.spec.ts — Role CRUD, permission matrix
• config/webhooks.spec.ts — Webhook CRUD, delivery logs
• config/automations.spec.ts — Workflow builder
• config/data-management.spec.ts — Importer, sync, trash, redirects
• config/operations.spec.ts — Monitor, queue, extensions, system-settings
• admin/tenants.spec.ts — Multi-tenant management

Troubleshooting

“Server did not start in time”

• Check if port 4173 is already in use.
• Verify that PLAYWRIGHT_TEST_BASE_URL points at http://127.0.0.1:4173.
• Run bun run test:integration if the preview build is failing before the browser starts.

“Login failed”

• Ensure the auth-setup project completed and wrote state into tests/e2e/.auth/.
• Check config/private.test.ts exists and contains only test credentials.
• Verify TEST_API_SECRET matches the value in tests/e2e/.auth/test-secret.txt.

Remote Playwright Testing

You can run Playwright tests from your local machine against a remote SveltyCMS instance (e.g. running on a Plesk server) using an SSH tunnel.

Method 1: SSH Port Forwarding (Recommended)

This method securely forwards your local port 5173 to the remote server’s port 5173, making the remote app appear local.

1. Open an SSH Tunnel in a separate terminal window:

   # Forwards local 5173 -> remote 5173
   ssh -L 5173:localhost:5173 user@your-remote-server.com

2. Run Playwright Locally: By default, playwright.config.ts is configured to reuse an existing server on port 5173.

   # Run tests against the tunnel
   PLAYWRIGHT_TEST_BASE_URL=http://localhost:5173 npx playwright test

Method 2: Direct URL

If your remote server is publicly accessible (e.g. https://dev.example.com or a specific IP), you can point Playwright directly to it.

PLAYWRIGHT_TEST_BASE_URL=https://dev.example.com npx playwright test

Note: Ensure the remote server is running in a mode that accepts connections (e.g. bun dev --host or a preview build) and that any firewalls allow traffic on the target port.

---

Related

• Test Documentation
• Security Testing
• Architecture Overview