Skip to content

Documentation

API Testing & Coverage Report

Black-box API testing strategy, isolation rules, and CI execution model for SveltyCMS.

7/15/2026
5 min read Edit on GitHub

SveltyCMS validates its API layer primarily with black-box integration tests. The suite talks to a live server over HTTP, verifies the full middleware chain, and runs the same contracts across SQLite, MongoDB, MariaDB, and PostgreSQL.

Unit-level dispatcher checks use thin shared helpers (invokeApi in tests/unit/utils/mock-event.ts) β€” not a separate HTTP-style client. See API Unit Helpers.

Note

The source of truth for the latest pass/fail state is GitHub Actions. This guide explains the contract the black-box API suite is expected to enforce.

πŸ›‘οΈ Test Isolation & Safety

SveltyCMS is designed so test runs never touch live user data. The safety contract is:

  1. Config Isolation: Local integration/E2E/hooks use config/private.test.ts only. They must never read or write the developer’s config/private.ts (live CMS DB β€” e.g. sveltycms.db). See src/utils/private-config-policy.ts.
  2. Ephemeral Mirror Only: CI may create a short-lived config/private.ts on the runner as a mirror of the test config β€” never committed or pushed.
  3. Test Mode Gate: /api/testing is inactive unless TEST_MODE=true and a valid TEST_API_SECRET is provided.
  4. Loopback Lockdown: Local black-box jobs use 127.0.0.1 to avoid IPv4/IPv6 drift and keep the test API loopback-only.
  5. Database Isolation: SQLite jobs use isolated names (sveltycms_test, benchmark_shared, e2e_*). Service-backed jobs use disposable CI containers. Test DB_NAME must not equal live private.ts.

Coverage at a Glance

  • Execution Style: Real HTTP requests against a running SvelteKit server
  • Primary Concerns: Authentication, RBAC, setup gating, CRUD contracts, and DB parity
  • Database Matrix: SQLite, MongoDB, MariaDB, PostgreSQL
  • Security Posture: Fail-closed, loopback-only test API with shared secret
  • Runtime Parity: Production build preview for CI integration and most E2E jobs

πŸ—οΈ Testing Architecture

Our API tests operate at the integration level. Unlike unit tests, they boot a real server process and verify the complete request path:

Request -> Hooks -> Auth -> Route Handler -> Adapter -> Response

Key Components

  1. Fail-Closed Dispatcher: Unmapped or unauthorized routes must reject by default.
  2. Testing API (/api/testing): Handles reset, seed, and isolated fixture orchestration without direct adapter imports inside black-box tests.
  3. Integration harness: tests/integration/harness.ts provisions config/private.test.ts, starts the preview server, and seeds via /api/testing. Run with bun test --timeout 300000 tests/integration/.
  4. Real Authentication: Tests obtain cookies via /api/auth/login and then hit the same endpoints a browser or API consumer would use.

Verified API Areas

Authentication & Security

  • 2FA System: Setup, TOTP verification, backup codes, and session recovery.
  • Security Management: IP blocking, incident resolving, and CSP report processing.
  • Registration: Verification of invitation token consumption and role assignment.

User & Token Management

  • User Lifecycle: Creation, login/logout, multi-tenant isolation, and role-based access.
  • Token System: JWT creation, rotation, and revocation.
  • State Verification: Usage of get-user action to verify post-registration database state.

Content & Relational APIs

  • Collections: CRUD operations, complex relations, and schema validation.
  • Relational Benchmarks: Dedicated performance testing for Depth 2-3 populations and JOINs.

System & Dashboard APIs

  • Health and Settings: Critical admin endpoints are exercised through the same route dispatcher used in production.
  • Dynamic Config: Setup-state and system settings flows are validated alongside the browser smoke coverage.

CI Execution Model

The GitHub Actions matrix runs the API suite once per supported database:

  • SQLite: File-backed local parity and reset semantics
  • MongoDB: Adapter parity and auth coverage
  • MariaDB: SQL adapter parity and relational flows
  • PostgreSQL: SQL adapter parity and relational flows

Each job reuses the shared test secret, provisions an isolated database, boots the app in test mode, and runs the same HTTP contract suite.


πŸš€ Running the API Suite

Run All Integration Tests

bun run test:integration
```

### Targeted Testing (day-to-day loop)

Prefer **SQLite + skip rebuild** when the production bundle is already current:

```bash
# Fast local loop (SQLite once build/ exists)
bun test --timeout 300000 tests/integration/

# API suite only
bun test --timeout 300000 tests/integration/api/

# Full matrix parity (Docker profile up; slower β€” adapters/auth PRs)
DB_TYPE=postgresql bun test --timeout 300000 tests/integration/
DB_TYPE=mariadb bun test --timeout 300000 tests/integration/
DB_TYPE=mongodb bun test --timeout 300000 tests/integration/
```

> [!TIP]
> First run after a clean checkout needs `bun run build` (or `bun run test:integration`). Day-to-day, reuse `build/` and run `bun test --timeout 300000 tests/integration/`.

### Bulk user HTTP path

Bulk user actions are covered by black-box integration:

- `tests/integration/api/user-extended.test.ts` β†’ `POST /api/user/batch` (block/unblock/delete + negatives)

Keep bulk **HTTP** coverage there; use unit tests for pure batch helpers only.

### Session β†’ page load (P0, 2026-07-18)

`tests/integration/api/session-page-load.test.ts` proves a login cookie unlocks SvelteKit `__data.json` for admin pages (`/user`, `/dashboard`, `/config/collectionbuilder`, …) β€” not only `/api/user`.
Unit companions: `tests/unit/auth/session-cookies.test.ts`, `tests/unit/api/testing-login-cookie-contract.test.ts`.
See [Headless-First Test Inventory](/docs/tests/headless-test-inventory).

---

## Maintenance Notes

- Keep raw-response contracts aligned with endpoint tests. If `raw=true` returns a bare array or object for one route, tests should enforce that shape consistently across adapters.
- Continue migrating legacy integration files away from internal imports so black-box suites do not rely on privileged runtime access.
- Keep setup-state and auth helpers synchronized with the Playwright smoke suites.

---

_Last Updated: July 15, 2026_

Related

testingapicoveragefail-closed
Was this page helpful?