Overview

SveltyCMS uses Playwright for End-to-End (E2E) testing. Our strategy is optimized for speed and reliability:

1. Wizard Coverage: The setup wizard is tested directly in the dedicated wizard project so fresh-install flows stay PR-gating.
2. Auth Bootstrap: The auth-setup project provisions authenticated state once and persists it for the maintained app smoke projects.
3. Maintained Smoke Scope: PR-gating browser coverage includes all 12 E2E projects running in parallel — from auth and accessibility to visual regression, branding, and multi-tenant management.
4. Black-Box Safety: State setup happens through /api/testing plus TEST_API_SECRET; browser flows still navigate the real app and assert real redirects.

Running Tests

bun run test:e2e is the local convenience entrypoint. It starts the dev server and runs Playwright against 127.0.0.1.

# Run the maintained local E2E flow
bun run test:e2e

# Run the setup wizard contract
bun x playwright test --project=wizard  # (the wizard project targets tests/e2e/routes/setup/setup-wizard.spec.ts)

# Generate auth state for downstream smoke suites
bun x playwright test --project=auth-setup

# Run maintained app smoke suites
bun x playwright test --project=signup
bun x playwright test --project=content
bun x playwright test --project=system
bun x playwright test --project=config-routes
bun x playwright test --project=admin
bun x playwright test --project=dashboard
bun x playwright test --project=appearance
bun x playwright test --project=media
```

### 🧠 Smart Testing

We also provide a smart runner for local iteration:

```
bun run scripts/test-smart.ts
```

- **Fresh Environment:** Biases toward setup coverage first.
- **Configured Environment:** Runs the suites that match the working tree.
- **Local Helper Only:** GitHub Actions does not depend on the smart runner for PR gating.

## Writing Tests

### 1. Use the Auth Helper or Stored State

Avoid repeating the login UI in every spec. Use `loginAsAdmin` from `helpers/auth.ts` for targeted tests, or depend on the `auth-setup` project when a suite should consume persisted browser state.

```
import { test, expect } from "@playwright/test";
import { loginAsAdmin } from "./helpers/auth";

test("Admin can access dashboard", async ({ page }) => {
  // Logs in and waits for navigation to dashboard
  await loginAsAdmin(page);

  // Your test logic
  await expect(page.getByText("Dashboard")).toBeVisible();
});
```

### 2. Database State & Worker Isolation

Playwright uses **database-per-worker isolation** to eliminate SQLite locking issues and keep E2E runs reproducible.

- **Unique Database per Worker**: Each Playwright worker operates on its own dedicated SQLite file (e.g., `cms_worker_1.db`).
- **Header-Based Routing**: The system uses the `x-test-worker-index` header to automatically route requests to the correct isolated database.
- **Secret Handshake**: `/api/testing` calls must include the shared `TEST_API_SECRET`.
- **Safety Rule**: `config/private.test.ts` is the source of truth. Any `config/private.ts` present in CI is only a temporary mirror of the test config and must never point at live data.

### 3. Use `data-testid` Selectors

For robust tests that don't break when CSS classes change, always use `data-testid` attributes for interactive elements.

**In Svelte Component:**

```
<button data-testid="signin-submit">Sign In</button>
<input data-testid="signin-email" />
```

**In Playwright Test:**

```
await page.getByTestId("signin-email").fill("admin@example.com");
await page.getByTestId("signin-submit").click();
```

Avoid using:

- CSS classes (e.g., `.btn-primary`)
- Text content (e.g., `text="Sign In"`) unless checking localized text specifically.
- XPath selectors

### 4. CI/CD

GitHub Actions runs Playwright in staged jobs:

- **`e2e-wizard`**: validates fresh setup and post-setup redirects against the production build.
- **`e2e-auth`**: provisions auth state and uploads `tests/e2e/.auth/` as a short-lived artifact.
- **`e2e-app`**: fans out all 17 projects (signup, content, system, a11y, rbac, language, branding, visual-regression, users, builder, permissions, firstuser, config-routes, admin, dashboard, appearance, media) in parallel.

Failure artifacts include Playwright HTML reports, traces, video, and preview server logs.

### 5. Enterprise Config Route Specs

The following enterprise smoke tests cover config and admin routes:

- `config/access-management.spec.ts` — Role CRUD, permission matrix
- `config/webhooks.spec.ts` — Webhook CRUD, delivery logs
- `config/automations.spec.ts` — Workflow builder
- `config/data-management.spec.ts` — Importer, sync, trash, redirects
- `config/operations.spec.ts` — Monitor, queue, extensions, system-settings
- `admin/tenants.spec.ts` — Multi-tenant management

## Troubleshooting

**"Server did not start in time"**

- Check if port 4173 is already in use.
- Verify that `PLAYWRIGHT_TEST_BASE_URL` points at `http://127.0.0.1:4173`.
- Run `bun run test:integration` if the preview build is failing before the browser starts.

**"Login failed"**

- Ensure the `auth-setup` project completed and wrote state into `tests/e2e/.auth/`.
- Check `config/private.test.ts` exists and contains only test credentials.
- Verify `TEST_API_SECRET` matches the value in `tests/e2e/.auth/test-secret.txt`.

## Remote Playwright Testing

You can run Playwright tests from your local machine against a remote SveltyCMS instance (e.g. running on a Plesk server) using an SSH tunnel.

### Method 1: SSH Port Forwarding (Recommended)

This method securely forwards your local port 5173 to the remote server's port 5173, making the remote app appear local.

1.  **Open an SSH Tunnel** in a separate terminal window:

    ```
    # Forwards local 5173 -> remote 5173
    ssh -L 5173:localhost:5173 user@your-remote-server.com
    ```

2.  **Run Playwright Locally**:
    By default, `playwright.config.ts` is configured to reuse an existing server on port 5173.
    ```
    # Run tests against the tunnel
    PLAYWRIGHT_TEST_BASE_URL=http://localhost:5173 npx playwright test
    ```

### Method 2: Direct URL

If your remote server is publicly accessible (e.g. `https://dev.example.com` or a specific IP), you can point Playwright directly to it.

```
PLAYWRIGHT_TEST_BASE_URL=https://dev.example.com npx playwright test
```

> **Note**: Ensure the remote server is running in a mode that accepts connections (e.g. `bun dev --host` or a preview build) and that any firewalls allow traffic on the target port.

---

Related

• Test Documentation
• Security Testing
• Architecture Overview