mirror of
https://github.com/Stirling-Tools/Stirling-PDF.git
synced 2026-09-03 05:10:16 +03:00
Reece Browne
8a5470dd01
## What Follow-up to #7073. Turns the story scan into an accessibility gate: stories run axe in a real browser, and CI flags a change that adds a **new** violation. The app has plenty of existing a11y problems (mostly theme-level colour contrast), so rather than block everything on those, they're recorded in `.storybook/a11y-baseline.json` and grandfathered. The gate cares about three things: - a story breaking a rule it wasn't already breaking - a story that fails to render at all - a scan that didn't cover everything it was asked to Starting point: 839 stories carry a known violation, 1058 story-rule pairs. ## Where it runs - **Pull requests** scan only the stories the branch touches — usually seconds. A full sweep is ~30 minutes, too slow to sit in front of every merge, and the `frontend` path filter is broad enough that unrelated changes would pay for it. - **Nightly** scans every story, so a violation introduced somewhere other than the story itself — a shared component, a theme token — still surfaces within a day. - Both upload their scan reports as artifacts; the reports carry the offending selector and help text, without which a red run can only be understood by reproducing it locally. - **Advisory to start with.** It is deliberately not in `all-checks-passed`, so it reports without blocking. Worth promoting once a few weeks of runs show the pass/fail is stable. ## Using it - **Fixed some violations?** `task frontend:storybook:a11y:record` re-records so the gate locks the improvement in. - **Locally:** `task frontend:storybook:a11y:changed` for your branch, `task frontend:storybook:a11y` for everything. - **New component?** Its story is picked up automatically. ## Testing - Every story — 526 files, ~1,450 stories — runs in a real browser with no render failures, and the gate reports no regressions against the baseline. - Running the gate over a single changed story takes seconds, which is the pull-request path. - The gate's own behaviour is covered against synthetic scan reports: a new rule fails, the same rule on more nodes does not, a crashed story fails, an incomplete scan refuses to report, and re-recording refuses while anything is crashing. - Typecheck (all build variants), ESLint and Prettier pass. ## Notes for reviewers Some of this PR is making the mechanism trustworthy rather than adding features, so it's worth knowing what changed and why: - Rule ids come from the axe docs URL in each violation, not a hand-maintained list of rule names — the old list silently ignored 39 of axe's 104 rules, including `object-alt`, `target-size` and the table rules. - The baseline records **which** rules a story breaks, not how many nodes break them. Node counts drift between runs because stories fetch asynchronously and axe samples whatever has rendered, which made unrelated changes look like regressions. For the same reason the baseline is the union of repeated scans, so a run can only be a subset of it. - A story that fails for a non-a11y reason used to yield no rule id and was recorded as clean, which hid crashes and could mask real violations. Those now fail, and re-recording refuses to run while any story is crashing. - The scan writes a manifest of every story file it intends to cover and the check fails unless all of them reported, so a dropped batch can't read as "no violations". - Vite was pre-bundling the JSX runtime mid-run and reloading the page, which crashed whichever stories were loading; those deps are now named up front and the per-story timeout is above the 5s default. Colour contrast dominates the baseline and is theme-level, tracked separately from this.
58 lines
2.2 KiB
YAML
58 lines
2.2 KiB
YAML
name: Frontend a11y regression gate
|
|
|
|
# Reusable workflow called from build.yml when frontend sources change.
|
|
#
|
|
# Scans the stories this branch touches in real Chromium and runs axe against
|
|
# each. Existing violations are grandfathered in .storybook/a11y-baseline.json;
|
|
# the check fails on a NEW violation — a story breaking a rule it wasn't already
|
|
# breaking — or on a story that fails to render at all.
|
|
#
|
|
# Only changed stories, because a full sweep is ~30 minutes: far too slow to sit
|
|
# in front of every merge. The whole suite is scanned nightly instead
|
|
# (nightly.yml), which catches anything a branch didn't touch.
|
|
#
|
|
# Advisory for now: this is not in build.yml's all-checks-passed list, so a
|
|
# failure reports without blocking. Promote it once a few weeks of runs show the
|
|
# pass/fail is stable.
|
|
on:
|
|
workflow_call:
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
frontend-a11y:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 25
|
|
steps:
|
|
- name: Harden Runner
|
|
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
|
with:
|
|
egress-policy: audit
|
|
- name: Checkout repository
|
|
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
# Need the base branch too, to diff against it.
|
|
fetch-depth: 0
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
|
|
with:
|
|
node-version: "22"
|
|
cache: "npm"
|
|
cache-dependency-path: frontend/package-lock.json
|
|
- name: Install Task
|
|
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
|
|
- name: a11y gate (changed stories)
|
|
run: task frontend:storybook:a11y:changed -- origin/${{ github.base_ref || 'main' }}
|
|
- name: Upload scan reports
|
|
# The reports carry the offending selector and help text for each
|
|
# violation; without them a red run can only be understood by
|
|
# reproducing the whole scan locally.
|
|
if: always()
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
|
with:
|
|
name: a11y-scan-${{ github.run_id }}
|
|
path: frontend/.a11y-scan/
|
|
retention-days: 7
|
|
if-no-files-found: ignore
|