SveltyCMS Testing Documentation

Last Updated: November 21, 2025 Total Tests: ~615 Pass Rate: ~65% (403 passing, 172 failing, 40 skipped)

Why Multiple Test Documentation Files?

This testing documentation is organized into 12 specialized files for easy navigation and maintenance:

• By Test Type: Unit, Integration, E2E
• By Coverage Area: APIs, Hooks, Stores, Widgets, Utilities
• By Purpose: Status reports, workflow guides, coverage reports

This structure allows developers to quickly find relevant testing information without wading through a single massive document.

---

Test Frameworks Overview

SveltyCMS uses two complementary test frameworks to ensure comprehensive quality coverage:

🧪 Bun Test - Unit & Integration Tests

• Purpose: Fast, lightweight testing for business logic, services, and API endpoints
• Location: tests/bun/
• Test Count: ~575 tests
• Execution Time: ~2-15 seconds

What It Tests:

• ✅ Services (auth, security, caching)
• ✅ Utilities (validation, formatting, crypto)
• ✅ Stores (system state, UI state, loading)
• ✅ Widgets (validation, rendering, security)
• ✅ Middleware Hooks (all 11 hooks)
• ✅ API Endpoints (integration tests)
• ✅ Database Operations (MongoDB currently)

Why Bun?

• 100x faster than Jest/Mocha
• Native TypeScript support (no transpilation)
• Built-in mocking
• Perfect for headless CMS architecture

🎭 Playwright - End-to-End Tests

• Purpose: Browser automation for critical user journeys
• Location: tests/playwright/
• Test Count: ~40 tests
• Execution Time: ~30-60 seconds

What It Tests:

• ✅ Setup Wizard flow
• ✅ Login/Authentication
• ✅ Content CRUD operations
• ✅ Admin dashboard
• ✅ User management

Why Playwright?

• Cross-browser testing (Chromium, Firefox, WebKit)
• Reliable auto-waiting
• Screenshot/video recording on failure
• Parallel execution

---

GitHub Actions Automation

All tests run automatically in CI/CD via GitHub Actions:

Workflow: .github/workflows/playwright.yml

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - Run Bun unit tests (~350 tests)
      - Execution time: ~2 seconds

  integration-tests:
    runs-on: ubuntu-latest
    services:
      mongodb: # MongoDB service container
    steps:
      - Clean config directory
      - Start dev server
      - Seed test database
      - Run Bun integration tests (~225 tests)
      - Execution time: ~15 seconds

  e2e-tests:
    runs-on: ubuntu-latest
    services:
      mongodb:
    steps:
      - Clean config directory
      - Install Playwright browsers
      - Start dev server
      - Seed test database
      - Run Playwright E2E tests (~40 tests)
      - Execution time: ~60 seconds

[!IMPORTANT] Security Note: All GitHub workflows use pinned commit SHAs for actions and enforce frozen lockfiles (bun install --frozen-lockfile) to protect against supply chain attacks. Do not change these to version tags without verifying the commit SHA.

When Tests Run:

• On Pull Request: All tests must pass before merge
• On Push to main/next: Full test suite + release checks
• Manual Trigger: Via workflow_dispatch

Future: Multi-Database Matrix Testing

[!NOTE] Coming Soon: Parallel testing against multiple databases

strategy:
  matrix:
    database: [mongodb, postgresql, mariadb, mysql]

This will ensure SveltyCMS works correctly with all supported databases via Drizzle ORM.

---

Local Testing (Before Committing)

Quick Start

# Run ALL tests (unit + integration + E2E)
bun run test:all

# Run specific test types
bun run test:unit          # Fast (~2s) - No server needed
bun run test:integration   # Medium (~15s) - Requires server
bun run test:e2e           # Slow (~60s) - Requires server + browser

Unit Tests Only (Fastest)

# All unit tests
bun run test:unit

# Specific test files
bun test tests/bun/services/SecurityResponseService.test.ts
bun test tests/bun/stores/system.test.ts
bun test tests/bun/widgets/richText.test.ts

# Watch mode (re-run on file changes)
bun test --watch tests/bun/services/

Integration Tests (Requires Server)

# Terminal 1: Start dev server
bun run dev

# Terminal 2: Run integration tests
bun test tests/bun/api/
bun test tests/bun/hooks/
bun test tests/bun/databases/

Or use the automated script:

# Automatically starts server, waits for ready, runs tests
bun run test:integration

E2E Tests (Requires Server + Browser)

# Automated (recommended)
bun run test:e2e  # Starts server, seeds DB, runs Playwright

# Manual
# Terminal 1: Start server
bun run dev

# Terminal 2: Seed database
bun run scripts/seed-test-db.ts

# Terminal 3: Run Playwright
bun x playwright test tests/playwright/

Smart Testing (Local Development)

# Intelligent test runner
bun run scripts/test-smart.ts

This script:

1. Checks if server is running (starts if needed)
2. Detects if system is configured
3. Runs appropriate tests (setup tests OR full suite)
4. Cleans up after itself

---

Test Status & Coverage

Current Status (November 21, 2025)

Category	Passing	Failing	Skipped	Total
Unit Tests	350	0	0	350
Integration Tests	53	172	40	265
E2E Tests	TBD	TBD	TBD	~40
TOTAL	403	172	40	615

Recently Fixed ✅

• tests/bun/hooks/authorization.test.ts (22 tests) - Fixed mock database adapter
• tests/bun/hooks/theme.test.ts (40 tests) - Fixed mockResolve signature
• tests/bun/hooks/firewall.test.ts (27 tests) - Fixed logger import

Known Issues ⚠️

• 172 failing tests in API integration and Setup middleware
• Root cause: Server connection timeouts in test environment
• Fix in progress: Improving test environment setup

See Test Status Report for detailed breakdown.

---

Documentation Index

Core Testing Guides

• Test Status Report - Current pass/fail rates, known issues
• E2E Testing Guide - Playwright setup and best practices
• Git Workflow & CI/CD - GitHub Actions automation

Coverage Reports

• API Test Coverage - REST API endpoint testing (460+ tests)
• Hook Test Coverage - Middleware hooks (11 hooks, 485+ tests)
• Store Test Coverage - Svelte stores (system, UI, loading)
• Widget Test Coverage - Widget validation and security
• Utility Test Coverage - Helper functions and utilities

Specific Test Documentation

• API Test Summary - Quick API test overview
• User API Tests - User management endpoint tests
• Database Tests - Database adapter testing

---

Why Bun for Testing SveltyCMS?

SveltyCMS is an agnostic headless SvelteKit CMS - it works with multiple databases and focuses on API-first architecture. Bun is the perfect testing framework for this use case:

🚀 Speed & Performance

Lightning-Fast Execution

• Up to 100x faster than Jest/Mocha
• Our 76-test utility suite completes in ~1.5 seconds
• Full API test suite (460+ tests) completes in ~15-20 seconds
• Instant feedback during development

Why Speed Matters for a Headless CMS:

• Rapid iteration on API endpoints
• Quick validation of database operations
• Immediate feedback on widget validation and security
• Faster CI/CD pipelines = faster releases

🎯 Perfect Match for Headless CMS Architecture

Database Agnostic Testing

• SveltyCMS supports MongoDB, PostgreSQL, MySQL, MariaDB
• Bun’s speed allows testing against multiple database adapters without painful wait times
• Future: Matrix testing across all databases in parallel

Headless CMS Requirements

• REST API testing (460+ endpoint tests)
• GraphQL API validation
• Widget system testing (security, validation, rendering)
• Multi-tenant isolation testing
• Cache layer verification (Redis operations)
• Authentication & authorization flows

Native TypeScript Support

• Zero configuration - Bun runs .ts files directly
• No transpilation overhead
• Perfect for SvelteKit’s TypeScript-first approach
• Type-safe test code matches production code

⚡ Built-in Features

// Just import and go - no jest.config.js, no babel setup
import { describe, it, expect, mock } from 'bun:test';

// Native mocking
const mockFn = mock(() => 'mocked value');

// Async/await support
it('should handle async operations', async () => {
	const result = await fetchData();
	expect(result).toBeDefined();
});

---

Test Organization

Directory Structure

tests/
├── bun/                    # Bun unit & integration tests
│   ├── setup.ts           # Global test setup
│   ├── mocks/             # Mock implementations
│   ├── api/               # API integration tests
│   ├── databases/         # Database adapter tests
│   ├── hooks/             # Middleware hook tests
│   ├── services/          # Service layer tests
│   ├── stores/            # Svelte store tests
│   ├── utils/             # Utility function tests
│   └── widgets/           # Widget validation tests
│
├── playwright/            # Playwright E2E tests
│   ├── setup-wizard.spec.ts
│   ├── login.spec.ts
│   └── helpers/           # Test helpers
│
└── scripts/               # Test automation scripts
    ├── seed-test-db.ts    # Database seeding for tests
    └── test-smart.ts      # Intelligent test runner

Test File Naming Convention

• Unit Tests: *.test.ts (e.g., SecurityResponseService.test.ts)
• Integration Tests: *.test.ts in api/, hooks/, databases/
• E2E Tests: *.spec.ts (e.g., login.spec.ts)

---

Best Practices

For Developers

1. Run tests locally before committing

   bun run test:unit  # Quick sanity check

2. Write tests for new features
   • Unit tests for business logic
   • Integration tests for API endpoints
   • E2E tests for critical user flows (if needed)
3. Use watch mode during development

   bun test --watch tests/bun/services/MyNewService.test.ts

4. Check CI before merging
   • All tests must pass in GitHub Actions
   • Review test failures in PR checks

For Test Writers

1. Follow existing patterns
   • See Hook Test Coverage for examples
2. Use descriptive test names

   it('should return 401 for unauthenticated API requests', async () => {
   	// Test implementation
   });

3. Mock external dependencies

   import { mock } from 'bun:test';
   const mockDb = mock(() => ({ users: [] }));

4. Test edge cases
   • Empty inputs
   • Invalid data
   • Error conditions

---

Troubleshooting

Common Issues

“Server did not start in time”

• Check if port 5173 is already in use
• Run bun run dev manually to see errors
• Increase timeout in scripts/seed-test-db.ts

“Cannot find module ‘@src/…‘”

• Ensure bunfig.toml has correct path aliases
• Run bun install to refresh dependencies

“Test timeout”

• Integration tests need running server
• Use bun run test:integration (auto-starts server)
• Or start server manually in separate terminal

“Database connection failed”

• Ensure MongoDB is running (for integration/E2E tests)
• Check DB_HOST, DB_PORT, DB_NAME env vars
• Run bun run scripts/seed-test-db.ts manually to debug

---

Contributing

When adding new tests:

1. Choose the right test type:
   • Unit test: Pure functions, no external dependencies
   • Integration test: API endpoints, database operations
   • E2E test: Critical user journeys only
2. Update documentation:
   • Add test count to relevant coverage report
   • Update this index if adding new test category
3. Verify in CI:
   • Push to PR and check GitHub Actions
   • All tests must pass before merge

---

Related Documentation

• Architecture Overview
• API Documentation
• Contributing Guide
• Development Setup