The SveltyCMS benchmark matrix is a CI-integrated performance validation system that runs 62 registered benchmark modules against 8 database configurations (4 adapters × 2 Redis states), detects regressions via statistical analysis, generates per-adapter MDX reports with trend visualizations, and includes a pre-flight sanitizer that validates benchmark environment integrity before any measurement begins.

Overview

• Orchestration: scripts/benchmark-matrix/index.ts spawns isolated DB servers, runs all applicable benchmarks, and generates reports.
• Sanitizer: tests/benchmarks/modules/benchmark-sanitizer.ts validates collection cleanliness, CRUD sanity, and warmup isolation before benchmarks.
• Harness tests: tests/unit/benchmarks/benchmark-harness.test.ts (11 tests) validates assertSuccess(), warmup isolation, and error visibility in the benchmark framework itself.
• Statistical engine: Two-pass architecture — linear regression per metric, then cross-metric correlation for root cause classification.
• CI integration: ci.yml bench-core job runs core benchmarks per adapter; the full matrix can run on schedule or demand.
• History: All runs persist in tests/benchmarks/results/history.sqlite for trend analysis.
• Reports: 8 auto-generated MDX files in docs/project/benchmarks/ with executive summaries, full audit ledgers, and trend visualizations.

Mode Parity (read this before recording a baseline)

A benchmark number is only comparable to another number measured in the same execution mode. The ledger has exactly one production-parity mode and it must never be mixed with the dev/test mode:

The matrix runner boots every server from serverEnv (index.ts) with NODE_ENV=production, BENCHMARK=true and no TEST_MODE, and stamps SVELTY_BENCHMARK_SERVER_MODE=production into every test child. Every recorded row carries both stamps:

• history.jsonl → runMode + serverMode per row
• history.sqlite (runs) → run_mode + server_mode columns

Guards (fail loud, never silent):

1. A row labelled matrix whose server mode is not production is refused — it is never written to the JSONL or the trend store (isModeContaminated() in benchmark-history.ts). This is what kept a 0.538 ms TEST_MODE cache-invalidation sample from entering the ledger on 2026-09-18.
2. Trend queries are mode-filtered: the published per-adapter ledgers and the matrix regression detector read run_mode='matrix' + server_mode='production' samples only. Standalone runs compare against standalone samples of their own server mode.
3. Rows without a mode stamp (pre-2026-09-19 history) are unknown and are never used for comparison.

Why this exists

cache-hit-ratio → Cache Invalidation once reported a 1.089 ms regression against a 0.595 ms baseline. Neither number was wrong: the baseline came from a standalone run whose server had been spawned with NODE_ENV=development + TEST_MODE=true (security path bypassed — the default for any standalone benchmark, because bun test forces TEST_MODE), the matrix run measured production parity, and the same build under TEST_MODE=true measures 0.538 ms. Two modes, one table — the artifact was the ledger, not the cache. --no-build-style convenience flags are fine; convenience modes are not.

Reproducing a comparable standalone baseline

A standalone run measures a server that the test process spawns itself. bun test always sets TEST_MODE=true (bunfig preload → tests/unit/setup.ts), so without an explicit marker every standalone server is a dev-mode TEST_MODE server and its rows are stamped server_mode='test' — measurable, recorded, and never compared against matrix rows.

To record a production-parity standalone baseline, set the marker explicitly — it takes precedence over the ambient TEST_MODE flag in setupBenchmarkServer():

# Production parity (server boots NODE_ENV=production, TEST_MODE deleted)
# Linux/macOS/Git-Bash:
SVELTY_BENCHMARK_SERVER_MODE=production BENCHMARK_RECORD=1 bun test tests/benchmarks/cache-hit-ratio.test.ts

On PowerShell set the variable first ($env:SVELTY_BENCHMARK_SERVER_MODE="production"), then run BENCHMARK_RECORD=1 bun test …. Verify the result rather than assuming it: the recorded row must show "runMode": "standalone" and "serverMode": "production" in tests/benchmarks/results/history.jsonl, and the server log line must report NODE_ENV=production with no TEST_MODE. On Windows sh/Git Bash the inline prefix works; on PowerShell set the variable first ($env:SVELTY_BENCHMARK_SERVER_MODE="production"), then run BENCHMARK_RECORD=1 bun test …. The recorded row’s serverMode tells you which of the two you produced.

Architecture

Directory Layout

File counts updated following the benchmark matrix consolidation on 2026-10-12.

Orchestration Layer (scripts/benchmark-matrix/)

Shared Modules (tests/benchmarks/modules/)

Report Files (docs/project/benchmarks/)

Local Benchmark Safety (July 2026)

Local benchmarks use a sandboxed environment that protects live developer data in config/ and .compiledCollections/. Two profiles govern behavior:

Sandbox Isolation (src/utils/benchmark-sandbox.ts)

Every row is a path a benchmark was once caught writing to; assertLiveDataWriteAllowed() fails closed on the live column during the local profile. Since 2026-10-07 it also fires for non-E2E automated harnesses (vitest / bun test / integration), which must not materialize builder categories in the developer’s live config/collections — see the row below. E2E is exempt (PLAYWRIGHT_TEST), because ci.yml e2e-prep deliberately prepares a config/collections tree for the Playwright server.

Guard for the sweep row: tests/unit/scripts/integration-harness-cleanup.test.ts pins that harness shells are removed while live-named files (sveltycms.db, sveltycms.db.sqlite, scratch probe_plainA.sqlite) survive.

Why config/collections/ category folders are on this list (2026-10-07): syncContentState mirrors every builder category to a real directory under the collections root. An integration run wrote unit-reorder-* folders into the developer’s live config/collections; the scanner then adopted them as source: "filesystem" categories in content_nodes + .compilation-manifest.json, so every reconcile re-created them. The guard now fails closed for non-E2E harnesses, and the integration tests delete through syncContentState so their folders are removed too.

Why config/sync/ is on this list (2026-09-28): tests/benchmarks/config-promotion.test.ts exports once per iteration, so three benchmark runs had minted 148 config/sync/export_global_* folders in the developer’s project. resolveConfigSyncRoot() now resolves explicit env → sandbox → live, performExport calls assertLiveDataWriteAllowed(exportDir), and the benchmark banner prints the active root. The export folders are write-only artifacts (performImport reads the DB/source tree, never the export directory), so redirecting them changes nothing about what an import applies.

Runtime Modes

Exactly three runtime modes exist (see docs/tests/benchmark-isolation.mdx):

1. Production (default) — NODE_ENV=production, AUDIT_CHAIN_SYNC=false, DISABLE_AUDIT_LOGS=true.
2. Production + Enterprise Compliance — benchmark servers with BENCHMARK_AUDIT_MODE=compliance → AUDIT_CHAIN_SYNC=true, DISABLE_AUDIT_LOGS=false (synchronous SHA-256 audit chain on every mutation).
3. E2E / Integration testing — TEST_MODE=true, /api/testing + x-test-secret work (timing-safe, 403 in production).

Benchmark servers always run mode 1 or 2: NODE_ENV=production, TEST_MODE explicitly cleared, real session-cookie auth, real rate limits/WAF/CSRF. x-test-secret grants nothing.

External Services Guard (src/utils/benchmark-runtime.ts)

When BENCHMARK=true, outbound side-effects are disabled so load tests never fire real external calls:

• Email/SMTP — email.server.ts returns { benchmark_sandbox: true }
• AI translation — ai-translation.ts returns null
• Webhooks — webhook-service.ts and automation-service.ts skip triggers

Redis L2 cache is controlled by BENCHMARK_NO_REDIS=1 alone (set by default in getBenchmarkTestEnv). Set BENCHMARK_NO_REDIS=0 + USE_REDIS=true to benchmark the real Redis L2 path. Background jobs (job-queue-service) and the watchdog now run during benchmarks — production parity.

Preflight (scripts/verify-benchmark-local.ts)

Runs automatically before bun run benchmark and bun run benchmark:core. Checks:

• Build exists with full testing harness (bench mode)
• Live config/private.ts does not use test DB names (benchmark_shared, sveltycms_test)
• Prints isolation boundaries: database, compiled root, media sandbox, external services status

COMPILE_ALL_ADAPTERS=true bun run build
bun run benchmark --db=sqlite          # preflight runs first (full tier)
bun run benchmark:core --db=sqlite     # CI parity core suite

# Tiered execution:
bun run benchmark --db=sqlite --tier=competitive   # fastest: 1 replica test (10 workloads, ~15s)
bun run benchmark --db=sqlite --tier=core          # 8 essential CI tests (~60s)
bun run benchmark --db=sqlite --tier=full          # complete regression suite (default)

# Single benchmark (or a few) instead of the whole matrix — substring match on the
# file name, comma-separated; repeated flags OR together:
bun run scripts/benchmark-matrix/index.ts --db=sqlite --only=auth-session-amr
bun run scripts/benchmark-matrix/index.ts --db=postgresql --only=database-performance,truth-latency

git checkout -- docs/project/benchmarks/

Harness Contracts for Spawned Tooling (2026-09-27)

Two runner behaviours were fixed after a full local matrix run left the tree red. Both are contracts, not conveniences — a run must leave a tree that bun run check accepts, and a probe must measure the tool rather than the harness.

Both fixes were verified by running the matrix and then bun run check immediately afterwards, and by running the DX probe alone.

Key Features

Statistical Regression Detection

The system uses a two-pass architecture:

Pass 1 — Per-adapter trend analysis:

• Loads last 20 runs of each metric from history.sqlite
• Computes linear regression slope, R², and standard deviation
• Classifies trend direction via classifyTrend(slope, sd, budget, current, sampleSize)
• Requires minimum 3 runs (MIN_RUNS_FOR_TREND) before statistical analysis

Pass 2 — Root cause classification:

• Gathers all non-stable MetricTrends across the adapter
• Applies CORRELATION_RULES from config.ts to match degrading metrics
• Three correlation rules: middleware (hooks + collections + authAvg), adapter (dbRaw + collections + graphqlAvg + indexPressure), native (memGrowth + mediaAvg)

Enhancements (July 2026):

• Flapping detection: Metrics alternating pass/fail across runs are flagged “unstable” even on flat trend lines — catches non-deterministic tests.
• Improvement detection: Significant improvements (pct < −20%) emitted as positive notes.
• Cross-DB correlation: Same metric degrading across multiple adapters flagged as “cross-cutting regression” (code change, not DB-specific).
• Budget forecasting: Upward-trending metrics projected forward to estimate runs until budget breach.

Performance Budgets

Defined in scripts/benchmark-matrix/config.ts with per-adapter overrides:

export const PERFORMANCE_BUDGET = {
  coldStartMs: 5_000,
  collections: 5, // REST/Collections p95 (ms)
  graphqlAvg: 12, // GraphQL Avg (ms)
  dbRaw: 50, // DB Raw p95 (ms)
  hooks: 2.0, // Hooks/Middleware (ms)
  memGrowth: 60, // Memory Growth (MB)
  securityMs: 25,
  indexPressure: 250,
} as const;

export const ADAPTER_BUDGET_OVERRIDES = {
  sqlite: { indexPressure: 250, dbRaw: 80 },
  // Networked adapters pay a TCP round-trip per REST request; SQLite is in-process.
  postgresql: { indexPressure: 100, dbRaw: 30, collections: 8 },
  mariadb: { indexPressure: 150, dbRaw: 40, collections: 8 },
  mongodb: { indexPressure: 150, dbRaw: 40, collections: 8 },
};

The collections budget stays at 5 ms for in-process SQLite. Networked adapters get 8 ms: on 2026-10-24 the mongodb bench measured REST search p95 at 5.549 ms against a 2.79 ms recorded p95 while every other metric was flat — a shared-runner round-trip artifact, not a code regression. The smallest real signal seen so far is ≥1.5 ms, so 8 ms still gates a genuine slowdown.

History Persistence

Runs persist in tests/benchmarks/results/history.sqlite via benchmark-history.ts:

• Table runs with columns: run_id, run_mode, server_mode, test_id, db_type, redis, phase, metric, avg_ms, p95_ms, rps, error_count, status, timestamp
• Deduplication index on (run_id, run_mode, test_id, db_type, redis, phase, metric)
• Mode parity: matrix and standalone writers both stamp server_mode; trend reads filter on it (see Mode Parity)
• Auto-prunes to 50 runs per test to stay lean
• Also persisted as JSONL (results/history.jsonl) for portability — one row per metric with runMode + serverMode
• Per-adapter run snapshots live in a separate matrix_runs table (never runs — the trend store owns that name)
• A locked/unwritable store is reported (warning), never swallowed: the ledger is auxiliary telemetry

Warming-Up Detection

After a baseline reset or cold start, the system detects “warming-up” state:

• Checks reset_events table for recent resets
• Metrics >30% above recent average flag as warming
• Prevents false regression alarms during baseline rebuilding

Long-Running Soak Contract (longevity-soak.test.ts)

The soak is the only module that answers “does anything grow with uptime?” — so its contract is explicit, and a verdict is only issued when the window can carry one.

# Long row, production parity, 30 s buckets, one hour (≥ 15 min steady window required)
LONG_SOAK_HOURS=1 LONG_SOAK_SAMPLE_SEC=30 SVELTY_BENCHMARK_SERVER_MODE=production bun test tests/benchmarks/longevity-soak.test.ts

# Controls — same host, same DB, no new rows / no workers
LONG_SOAK_PROFILE=read-only LONG_SOAK_HOURS=0.75 SVELTY_BENCHMARK_SERVER_MODE=production bun test tests/benchmarks/longevity-soak.test.ts
LONG_SOAK_PROFILE=idle      LONG_SOAK_HOURS=0.75 SVELTY_BENCHMARK_SERVER_MODE=production bun test tests/benchmarks/longevity-soak.test.ts

Profile axis — growth is only attributable against a control: mixed is the production mix; read-only re-points the write slot at schema resolution (same request rate, no new rows); idle runs no workers (only the bucket health probe). An invalid profile value fails the run instead of silently falling back to mixed.

Per-bucket series (flushed to tests/benchmarks/results/<adapter>/soak-series-<profile>-<stamp>.json after every bucket, so an aborted run still leaves a curve): rss, heapUsed, heapTotal, external, arrayBuffers, the derived native/other residual (rss − heapTotal − external), host CPU busy %, and the bucket’s reqs/s, rolling avg + p95, error count.

The residual is what names the owner of an off-heap verdict: native allocators (SQLite, zlib) and JIT code space live there, and a falling residual means the working set is being trimmed, not that memory was released.

Probe contract — the bucket series is read from GET /api/system/health?verbose=true, whose memory object must expose rss, heapTotal, heapUsed, external and arrayBuffers (src/hooks.server.ts). A missing field reads as 0.00 for the whole run, so an omitted arrayBuffers silently disables the buffer-growth half of the off-heap branch. Recorded evidence and the profile matrix: achievements-2026.

Executive Summaries

Each report includes:

• Pass/fail status with sparklines for core metrics
• Dimension health rollup (latency, memory, throughput, cache)
• Quadrant chart (Mermaid) — delta% vs absolute ms, categorizing tests
• Historical pulse — sparklines for each test across last runs
• Issues table — actionable regressions with root cause and code paths
• Host environment — CPU, memory, OS details

Mermaid Visualizations

Reports include embedded Mermaid diagrams:

• Trend charts (line graphs) — per-metric history over last N runs
• Quadrant charts — scatter plot of delta% vs absolute latency, with quadrant labels
• These auto-render in the docs site and GitHub markdown

Two-pass regression detection flow:

CI benchmark pipeline:

Report zone structure:

CI Integration

bench-core job (per-adapter):

1. Runs 8 core benchmarks per adapter (cold-start-phased, truth-latency, database-performance, transaction-acid, cache-performance, hooks-performance, rest-api-performance, virtual-query-federation)
2. Runs scripts/benchmark-matrix/ci-regression-gate.ts after the tests:
   • absolute ceilings from PERFORMANCE_BUDGET / ADAPTER_BUDGET_OVERRIDES always fail
   • relative drift is compared against a rolling baseline (median of the last 7 samples): warn at ≥25 % + ≥0.25 ms, severe at ≥100 % + ≥1.5 ms
   • a severe drift must repeat on the next run before it fails the build; the first observation warns (pending confirmation (1/2)) and freezes the baseline so the noisy sample cannot re-baseline itself
   • policy: fail on main and merge_group, warn-only on next and PRs (BENCHMARK_GATE_POLICY overrides); absolute SLA budgets always fail
   • emits ::warning/::error annotations, a $GITHUB_STEP_SUMMARY table, and gate-report-<db>.json
3. Persists gate-baseline-<db>.json through actions/cache (per adapter + branch) so a fresh runner keeps its trend history
4. Reports pass/fail via generate-ci-markdown.ts

Benchmarks run on every push and PR — there are no skipped bench placeholders. The first two runs record baseline samples (building (1/2) → building (2/2)); the third compares against the median. Absolute budgets are enforced from run one.

bench-summary job (post-matrix):

1. Aggregates all per-adapter MDX reports
2. Aggregates the per-adapter gate reports into a single performance-gate table
3. Produces ci-summary.json and $GITHUB_STEP_SUMMARY

Quality gate (hooks + CI — no separate precheck-shared.ts):

• push tier: bench-core runs on every next/main push, then the performance gate fails the run on a budget breach or a confirmed severe drift (two consecutive runs)
• PR tier: bench-core runs on every PR; the gate warns instead of blocking review
• full tier: runs complete bench matrix

Commands Reference

Matrix Tiers & Full Matrix

# Fastest tier: runs only competitive-workload-replica (10 standard workloads, 1 file, ~15s)
bun run scripts/benchmark-matrix/index.ts --db=sqlite --tier=competitive
# (Aliases: --replica, --competitive)

# CI core tier: runs 8 essential CI benchmarks (~60s)
bun run scripts/benchmark-matrix/index.ts --db=sqlite --tier=core
# (Alias: --core)

# Full matrix tier (default): runs all discovered top-level benchmark modules
bun run scripts/benchmark-matrix/index.ts --db=sqlite
bun run scripts/benchmark-matrix/index.ts --db=sqlite --tier=full
bun run scripts/benchmark-matrix/index.ts --db=postgresql
bun run scripts/benchmark-matrix/index.ts --db=mongodb

# With Redis caching layer (env-controlled — there is no `--redis` flag)
USE_REDIS=true bun run scripts/benchmark-matrix/index.ts --db=sqlite

# Continue past individual test failures (still records + finalizes report)
bun run scripts/benchmark-matrix/index.ts --db=sqlite --continue-on-error

# Skip build (already built) — never rebuilds mid-run, still verifies build/index.js exists
bun run scripts/benchmark-matrix/index.ts --db=sqlite --no-build

Targeted Probe Execution

Isolated one-off diagnostic probes in tests/benchmarks/probes/ do not run in the standard matrix and can be run directly:

# Run specific isolated diagnostic probes
bun test tests/benchmarks/probes/probe-write-split.test.ts
bun test tests/benchmarks/probes/probe-point-read-split.test.ts
bun test tests/benchmarks/probes/probe-cache-tiers.test.ts

Mode contract: matrix runs are production parity by construction. If you need a standalone baseline for comparison, record it with SVELTY_BENCHMARK_SERVER_MODE=production and without TEST_MODE — see Mode Parity.

Real-Browser Vitality (Playwright)

tests/e2e/browser-vitality.spec.ts is the browser-side counterpart to the fetch-loop benchmarks. It runs in Chromium against the E2E harness and records Navigation Timing (ttfb, domInteractive, duration), input.fill() latency, and performance.memory heap growth across SPA route transitions. Budgets are intentionally loose (domInteractive

< 5 s, fill < 1 s, heap < 180 MB) so the suite catches regressions without flaking on shared CI runners; tighten them once a stable baseline is recorded.

bun run benchmark:browser

It is part of the Admin & Catch-All shard (browser-vitality in .github/workflows/e2e-matrix.ts) and therefore runs in CI, not only on demand.

Single Test with Recording

# Record mode — persists results to history for trend analysis
BENCHMARK_RECORD=1 bun test tests/benchmarks/auth-performance.test.ts

# Run a specific benchmark test with matrix environment
BENCHMARK_RECORD=1 BENCHMARK_MATRIX=1 bun test tests/benchmarks/truth-latency.test.ts

Report Generation

# Regenerate clean report shells (resets all LEDGER tags)
bun run scripts/benchmark-matrix/generate-benchmark-reports.ts --force-all

# Regenerate only SQLite shell
bun run scripts/benchmark-matrix/generate-benchmark-reports.ts --force-sqlite

CI-Core Benchmarks

# Run CI-core benchmarks for all adapters
bun run scripts/runs/run-core-benchmarks.ts

# Run for a single adapter
bun run scripts/runs/run-core-benchmarks.ts --db=sqlite
bun run scripts/runs/run-core-benchmarks.ts --db=postgresql

All Benchmarks (Standalone)

There is no single “run every benchmark” script. The orchestrator discovers all 86 files in tests/benchmarks/; the 24 that are missing from the BENCHMARK_SCRIPTS register still run, appended as a final Ungrouped group (coverage matches the directory).

# Every discovered module against one adapter (repeat per --db=, add USE_REDIS=true for the Redis variant)
bun run scripts/benchmark-matrix/index.ts --db=sqlite

# Standalone mode over the whole directory (each module spawns its own server)
bun test tests/benchmarks/

Report Structure

Each per-adapter MDX report (docs/project/benchmarks/benchmark_*.mdx) contains distinct zones controlled by marker comments:

Zone Markers

<!-- ── ZONE: executive ── -->
<!-- ── ZONE: summary ── -->
<!-- ── ZONE: ledger ── -->
<!-- ── ZONE: educational ── -->
<!-- ── ZONE: changelog ── -->
<!-- ── ZONE: metadata ── -->

Executive Zone Sub-Markers

<!-- ── fix_notes ── -->
<!-- ── partial_watermark ── -->
<!-- ── alerts ── -->

Fix notes: Actionable remediation suggestions for detected regressions. Partial watermark: Shown when not all tests completed (incomplete run). Alerts: Critical budget violations or cross-cutting regressions.

Summary Zone Sub-Markers

<!-- ── run_overlay ── -->
<!-- ── history_tables ── -->

Run overlay: Current run’s latency matrix (scenario, latency, trend, budget, result). History tables: Archive of past runs with sparklines.

Ledger Zone Sub-Markers

Each benchmark module gets a per-tag collapsed section:

<!-- ── TAG: rest-api-performance ── -->
<details>
<summary><strong>REST API Performance</strong></summary>
...
</details>
<!-- ── END TAG: rest-api-performance ── -->

Within each tag:

• Trend heading: 🟢/🔴/⚪ icon with delta% and run count
• Insight: Root cause analysis, code paths to check
• Truth table: Per-scenario latency data
• Run summary: Abbreviated results table

Adding New Benchmarks

Step 1: Create the Test Module

Create tests/benchmarks/<name>.test.ts following the existing pattern:

/**
 * @file tests/benchmarks/my-new-benchmark.test.ts
 * @description Benchmark: My New Feature Performance
 *
 * ### Scenarios:
 * - warm: Steady-state CRUD operations
 * - cold: First-request latency
 */
import { describe, it, expect } from "vitest";
import { setupBenchmarkServer, reportBenchmark } from "./modules/benchmark-utils";
import { resolve } from "node:path";

const TEST_FILE = resolve(__dirname, import.meta.filename ?? __filename);

describe("My New Feature", () => {
  it("warm steady-state performance", async () => {
    const { api } = await setupBenchmarkServer();

    const start = performance.now();
    const res = await api.get("/api/my-feature");
    const duration = performance.now() - start;

    expect(res.status).toBe(200);

    await reportBenchmark({
      testFile: TEST_FILE,
      avgMs: duration,
      p95Ms: duration, // or compute from multiple iterations
      rps: 1000 / duration,
      scenario: "warm",
    });

    await api.close();
  });
});

Step 2: Register in benchmark-scripts.ts (perf benchmarks only)

Add an entry to the BENCHMARK_SCRIPTS array in scripts/benchmark-matrix/benchmark-scripts.ts:

{
  path: "tests/benchmarks/my-new-benchmark.test.ts",
  label: "My New Feature",
  shortLabel: "My Feature",
  level: "standard",
  section: "Feature Performance",   // section header in the ledger
  intensity: "medium",
  estimatedMs: 15_000,
  desc: "Measures CRUD and query performance for my new feature",
  strategy: "all",                   // "all" | "sql" | "once"
  tags: ["feature", "crud"],
  metricCategory: "latency",
  codePaths: ["src/services/my-feature.ts", "src/databases/my-adapter.ts"],
  correlatedWith: ["rest-api-performance"],   // optional
},

Key fields:

• strategy: "all" = run on every adapter; "sql" = only SQL adapters (PostgreSQL, MariaDB, SQLite); "once" = run once total
• level: "core" runs in CI bench-core; "standard" runs in full matrix
• section: Groups related benchmarks under a common <details> in the ledger
• codePaths: Source files to check when this benchmark degrades — appears in root cause insights
• correlatedWith / antiCorrelatedWith: Cross-metric hints for root cause classification

Step 3: Regenerate Report Shells

bun run scripts/benchmark-matrix/generate-benchmark-reports.ts --force-all

This ensures all 8 per-adapter MDX files have a <!-- ── TAG: my-new-benchmark ── --> section.

Step 4: Verify

# Single test with recording
BENCHMARK_RECORD=1 bun test tests/benchmarks/my-new-benchmark.test.ts

# Full matrix for SQLite, restricted to the new module
bun run scripts/benchmark-matrix/index.ts --db=sqlite --only=my-new-benchmark

Troubleshooting

Related

• Testing Index
• CI Performance Metrics
• Benchmark & Test Isolation
• Performance Benchmarks Index
• Performance Architecture
• Testing Scripts Catalog