<!-- source: openspec/changes/add-e2e-smoke-suite/proposal.md -->

<!-- source: openspec/changes/add-e2e-smoke-suite/proposal.md -->

# add-e2e-smoke-suite — proposal

## Why

Two prior changes — `add-network-public-surface` (#546) and `add-network-studio-surface` (#544) — explicitly **carved out** their Playwright e2e to "a wider `apps/web` / `apps/studio` e2e push" (per `openspec/ROADMAP.md § 6. Drift / cleanup items`, item 9). That wider push has not happened. `apps/studio/e2e/openspec-linear-sync.spec.ts` is also referenced from `changes/linear-as-agent-control-plane-openspec/verification.md § Pattern references` but no harness exists to run it.

Today the marketing site (`rapoport.studio`) and the workspace (`app.rapoport.studio`) ship to production behind a magic-link auth flow with no synthetic monitoring. Three concrete silent-failure modes are unguarded:

1. **Secrets stripped on Cloudflare Workers deploy** — `current/workflow.md § Stage 11 Release § Failure modes (a)` calls this out as a known gotcha; the only recovery is "run `wrangler secret put` immediately after deploy", which requires someone noticing the deploy.
2. **Schema migrations applied via Supabase MCP** drift from `db/types.md` — RLS regressions can ship without anyone exercising the auth-walled routes.
3. **Magic-link flow itself** — `current/auth.md § Magic-link cross-device fix` lists OAuth as the remaining open question; the magic-link path is "stable" but un-exercised post-deploy except by Pavel signing in occasionally.

The studio runs at low volume, so user-driven discovery of regressions is slow. A 2×/day Playwright sweep against production catches all three failure modes inside hours rather than days.

## What changes

### 1. New package `packages/e2e/`

Private (`private: true`), name `@rapoport-studio/e2e`, version `0.0.0`. Mirrors the `private + workspace:*` shape used by `packages/auth`, `packages/state`, `packages/config`. Not a library — no consumers import it. It is a **test harness**, declared as a package because the natural workspace home is monorepo-scoped (covers both `apps/web` and `apps/studio` from one place).

Stack: Playwright (`@playwright/test ^1.50`), Chromium-only at v1, Node 20+ (matches `apps/studio` runtime).

### 2. Smoke tier — six tests

| # | Surface | Test |
|---|---|---|
| 1 | `web` | `rapoport.studio/` returns 200 with the expected hero H1 |
| 2 | `web` | `/services` lists at least one service card |
| 3 | `web` | `/contact` intake form POST succeeds with `metadata.e2e = true` payload |
| 4 | `studio` | `/login` renders the magic-link form (no auth required) |
| 5 | `studio` | Authenticated flow: e2e user signs in → lands on workspace home → renders without 500 |
| 6 | `studio` | Spec smoke: open a pre-seeded spec at `/spec/<seeded-id>` → Architect chat surface renders |

Runtime budget: full suite under 3 minutes wall-clock. Six tests, one Chromium project, two browser contexts (anonymous + e2e-user).

### 3. Auth strategy — admin `generateLink` + `storageState`

No real email delivery. The `globalSetup` step calls Supabase admin API `auth.admin.generateLink({ type: 'magiclink', email: 'e2e@rapoport.studio' })`, extracts `token_hash` from the returned `action_link`, navigates Chromium to `/auth/confirm?token_hash=<token>&type=magiclink&next=/`, and saves the resulting cookies to `storageState.json`. Auth-walled tests load this state instead of repeating the dance.

This tests the **real** `/auth/confirm` route + `verifyOtp` + `dispatchAuthenticatedUser` chain. The only piece skipped is the Supabase email transport, which Supabase itself is responsible for.

### 4. Dedicated e2e user

One-time operator setup, **not in code**:

- Email: `e2e@rapoport.studio`.
- `app_metadata.platform_owner: true` (via Supabase Studio SQL or admin API). Required so the auth proxy in `packages/auth/src/proxy.ts` short-circuits on `platform_owner === true` without needing a `spec_invitees` row.
- No password set — magic-link only — `generateLink` is the sole auth path.
- One pre-seeded `specs` row owned by this user (and one parent `projects` row, since `specs.project_id` is `NOT NULL`), id of the `specs` row captured in `E2E_SEEDED_SPEC_ID` GitHub Actions secret. Schema: `openspec/current/db/entities.md § Project → spec → idea hierarchy` (applied 2026-05-22 from change `rearchitect-spec-document-model`, RAP-1170).

### 5. GitHub Actions scheduled workflow

`.github/workflows/e2e-prod.yml`. `schedule: '0 6,18 * * *'` (06:00 + 18:00 UTC) + `workflow_dispatch` for manual runs. Mirrors the structure of `.github/workflows/publish-forge.yml` (Node setup, `pnpm install --filter`, run step). Playwright traces uploaded as workflow artefact on failure.

### 6. Telegram alert on failure

On suite failure the workflow POSTs a Telegram message to the operator chat using existing `TELEGRAM_BOT_TOKEN` + `TELEGRAM_OPERATOR_CHAT_ID` secrets (precedent: `openspec/archive/2026-05-25-add-marketing-funnel-instrumentation/ § Phase 5`). Payload format defined in `design.md § 7`. On suite pass — silence.

## Affected capability specs

Archive-time amendments target:

1. **`openspec/current/platform.md`** — note the e2e harness in the capability map and the 2×/day schedule under "Operating model".
2. **`openspec/current/auth.md`** — document the admin-`generateLink`-based test auth path as an officially supported flow (it tests the real `/auth/confirm` route, not a bypass).

`openspec/current/` is otherwise read-only during the change (per `AGENTS.md § What gets amended where`).

## Out of scope

1. **Preview-PR e2e tier.** Running the same suite against per-PR preview deployments is a separate change — `add-e2e-preview-gate` — once the production tier is stable for ≥2 weeks. Preview URLs also need `STUDIO_COOKIE_DOMAIN` reasoning that this change does not solve.
2. **Stripe webhook tests.** The Stripe test-mode + signature fixturing surface is non-trivial; carve into `add-e2e-stripe-tier` after `wire-stripe-checkout-on-discovery` ships and stabilises.
3. **Linear OAuth, Telegram bot, real email delivery.** Each has its own integration cost; not in v1.
4. **Cross-browser matrix.** Chromium-only. Firefox + WebKit deferred until a real bug surfaces that justifies the runtime cost.
5. **Visual regression / Lighthouse / a11y scans.** Playwright gives basic a11y for free via `@axe-core/playwright`, but a curated baseline is its own change.
6. **Test-data tenant isolation beyond `e2e: true` tagging.** Every POST in v1 tags rows with `metadata.e2e = true`; a weekly cleanup job is documented as operator runbook, not code (see `design.md § 6`). Per-run isolated tenant is deferred to a future change if noise materialises.
7. **Performance / load testing.** Synthetic monitoring only — single user, no concurrency.

## Conflicts surfaced

1. **`platform_owner: true` on a non-Pavel user.** The e2e user gains the highest privilege tier. Mitigation: no password set, magic-link only, `generateLink` requires service-role key (Infisical-scoped to CI), audit-log entries from this user are flagged in operator runbook. Risk accepted.
2. **Production writes from tests.** Intake-form POST creates a real `leads` row — the only write in v1. The spec smoke is read-only against the pre-seeded `specs` row (per `design.md § Test data shape`). Every test write carries `metadata.e2e = true`; cleanup is a weekly cron query, not transactional. If row volume grows past ~50/week, revisit.
3. **Cloudflare rate limits.** Two browser contexts × six tests × 2×/day = ~24 requests/day from the GitHub Actions IP range against `rapoport.studio` and `app.rapoport.studio`. Below any Cloudflare default threshold, but if a future firewall rule rate-limits by IP, the workflow will need a header-based allowlist exception.
4. **`generateLink` API rate limits.** Supabase rate-limits `auth.admin.generateLink` per project. At 2×/day this is far below any limit; documented for future scale.

## Cross-references

- `current/auth.md § Authentication flow` — the real magic-link path being exercised.
- `current/auth.md § Magic-link cross-device fix` — the `/auth/confirm` route + `verifyOtp` that this suite calls into.
- `current/platform.md § Repository` — package namespace + `apps/` vs `packages/` split.
- `current/workflow.md § Stage 11 Release § Failure modes` — the silent failure modes this suite guards against.
- `openspec/archive/2026-05-25-add-marketing-funnel-instrumentation/ § Phase 5` — Telegram POST + cron precedent.
- `.github/workflows/publish-forge.yml` — workflow shape precedent.
- `openspec/ROADMAP.md § 6. Drift / cleanup items` item 9 — the carve-out this change resolves.
