Summary
Wire npm run test:ui (Playwright E2E tests) into the existing pr-checks.yml CI workflow so security-critical browser tests run on every PR.
Background / Context
PR #292 hardens the GraphSandbox iframe with a restrictive CSP and sandbox="allow-scripts", and adds Playwright browser tests (frontend/tests/graph-sandbox-security.spec.ts) that verify:
- Malicious DSL cannot make outbound network requests from within the iframe
- Malicious DSL cannot access
window.parent.document
These tests currently run only locally. If they are not executed in CI, regressions to the CSP or sandbox configuration will not be caught automatically. This is especially important before #293 removes the frontend semantic DSL validator, at which point the iframe CSP/sandbox becomes the sole client-side security boundary.
Problem
The pr-checks.yml workflow runs only vitest (npm test -- --run) for frontend tests and Docker health curls for smoke tests. No CI job executes Playwright specs. A future PR could accidentally weaken the CSP or sandbox attribute, and all existing CI checks would still pass.
Goal / Expected Behavior
Every PR triggers the Playwright E2E test suite in CI, and the workflow fails if any Playwright test fails.
Scope
- Add a CI job that runs
cd frontend && npm run test:ui in pr-checks.yml
- Ensure the job has the same Node.js/npm setup as the existing frontend test job
- The job should fail the workflow on Playwright test failure
- Cache Playwright browser binaries between runs for reasonable performance
Out of Scope
- Adding new Playwright tests (existing tests in
frontend/tests/ are sufficient)
- Modifying Playwright configuration files
- Changing the existing vitest or Docker smoke jobs
- Adding
test:smoke:compose to CI (it requires Docker Compose, which is already covered separately)
Chosen Implementation Approach
Add a new job to .github/workflows/pr-checks.yml that:
- Checks out the repo
- Sets up Node.js (same version as existing frontend job)
- Installs frontend dependencies
- Installs Playwright browsers (
npx playwright install --with-deps)
- Runs
cd frontend && npm run test:ui
Implementation Plan
- Inspect
.github/workflows/pr-checks.yml for existing job patterns
- Add a
playwright-tests job modeled after the existing frontend-tests job
- Ensure the job uses
working-directory: frontend
- Verify the job installs browsers before running tests
Relevant Files / Areas
.github/workflows/pr-checks.yml
Tests Required
- The new CI job itself serves as the test; verify it passes on a PR after implementation
Manual Verification / Self-Check
- Open a test PR and confirm the new Playwright job appears and passes
- Confirm that breaking a CSP directive causes the Playwright security tests to fail and the CI job to fail
Reviewer Acceptance Checklist
Dependencies
Follow-Up Work
None.
Definition of Done
Summary
Wire
npm run test:ui(Playwright E2E tests) into the existingpr-checks.ymlCI workflow so security-critical browser tests run on every PR.Background / Context
PR #292 hardens the
GraphSandboxiframe with a restrictive CSP andsandbox="allow-scripts", and adds Playwright browser tests (frontend/tests/graph-sandbox-security.spec.ts) that verify:window.parent.documentThese tests currently run only locally. If they are not executed in CI, regressions to the CSP or sandbox configuration will not be caught automatically. This is especially important before #293 removes the frontend semantic DSL validator, at which point the iframe CSP/sandbox becomes the sole client-side security boundary.
Problem
The
pr-checks.ymlworkflow runs only vitest (npm test -- --run) for frontend tests and Docker health curls for smoke tests. No CI job executes Playwright specs. A future PR could accidentally weaken the CSP or sandbox attribute, and all existing CI checks would still pass.Goal / Expected Behavior
Every PR triggers the Playwright E2E test suite in CI, and the workflow fails if any Playwright test fails.
Scope
cd frontend && npm run test:uiinpr-checks.ymlOut of Scope
frontend/tests/are sufficient)test:smoke:composeto CI (it requires Docker Compose, which is already covered separately)Chosen Implementation Approach
Add a new job to
.github/workflows/pr-checks.ymlthat:npx playwright install --with-deps)cd frontend && npm run test:uiImplementation Plan
.github/workflows/pr-checks.ymlfor existing job patternsplaywright-testsjob modeled after the existingfrontend-testsjobworking-directory: frontendRelevant Files / Areas
.github/workflows/pr-checks.ymlTests Required
Manual Verification / Self-Check
Reviewer Acceptance Checklist
npm run test:uion every PRDependencies
Follow-Up Work
None.
Definition of Done
pr-checks.ymlincludes a job that runs Playwright tests