mirror of
https://github.com/Stirling-Tools/Stirling-PDF.git
synced 2026-09-02 21:03:34 +03:00
a11y job: scan stories when their component changes; fix cold-start false failures (#7191)
## What Two fixes to the pull-request a11y job (#7086 follow-up), both found on its first day live. ### It now scans a component's stories when the component changes The job picked its scan set from changed **story files** alone. But a story renders the live component — editing `Button.tsx` changes what every Button story shows without touching a story file, and the job scanned nothing. That's the common way a11y regressions arrive, and it was exactly the case the job missed. The scan set now also includes stories whose **same-named sibling source file changed**: edit `Button.tsx` or `Button.css` and `Button.stories.tsx` is scanned. Changes that ripple further than a component's own stories (shared UI, theme tokens) remain the nightly sweep's job. ### It no longer fails on cold-start infrastructure noise The job's first real run (#7163) flagged a story as "failed to render". The story was fine — on a cold dependency cache (**every** CI run), Vite discovered the preview's own dependency graph mid-run and reloaded the page, killing whichever story happened to be loading with `Failed to fetch dynamically imported module`. Reproduced on a cold cache, passes on a warm one. - The preview's deps are named in `optimizeDeps.include`, which removes the mid-run reload (verified cold). - A batch whose report contains crash-class failures (failures carrying no axe rule) is retried once — a one-off infrastructure death passes the retry, a story that genuinely can't render fails both attempts and is still reported. Also: the scan-report artifacts were never actually uploading — they live in a dot-directory, which `upload-artifact` silently skips as hidden by default. `include-hidden-files: true` fixes that for the PR job and the nightly, so a red run finally has its evidence attached. ### The glue is Node now, so tasks work from any shell Raised in review: the pipeline leaned on `bash`, `sed`, `grep`, `sort` and `tr`. Task runs its commands in an embedded POSIX interpreter, but those are external binaries it has to find on PATH — and a Windows dev calling tasks from **PowerShell** has none of them (`sed`/`tr` missing outright, `sort` resolves to Windows' own, and `bash` resolves to *WSL's*). Confirmed broken by running the task from PowerShell before the change. The batch runner and affected-story detection are now small Node scripts (`a11y-scan.mjs`, `a11y-changed.mjs`) — the repo already requires Node, so one implementation serves PowerShell, git-bash and CI alike, instead of maintaining `.sh`/`.ps1` twins. ## Testing - Sibling detection: editing `Tabs.tsx` (component only) pulls `Tabs.stories.tsx` into the scan set; editing a `.css` sibling does the same; nothing unrelated leaks in. - **From PowerShell**: `task frontend:storybook:a11y:changed` early-exits cleanly with no changes, and with a component edit it detects the sibling, runs the browser scan and passes the gate — same result from git-bash. - Cold cache end-to-end: cleared both Vite caches, ran the scan — no re-optimize, no reload, stories fail only on their (baselined) axe results. - Crash classifier: 1 on a synthetic crash report, 0 on axe-only failures, 0 on a real report — so the retry can't be triggered by legitimate violations. - Full scan + gate run green end-to-end; taskfile parses, workflows are valid YAML, Prettier/ESLint pass. #7163's red check needs no action from that PR's author — it should go green on re-run once this lands.
This commit is contained in:
+14
-14
@@ -214,42 +214,42 @@ tasks:
|
||||
desc: "a11y regression gate over every story: fail only on NEW axe violations"
|
||||
deps: [install, storybook:browser]
|
||||
cmds:
|
||||
- bash .storybook/a11y-scan.sh
|
||||
- node .storybook/a11y-scan.mjs
|
||||
- node .storybook/a11y-check.mjs --in .a11y-scan --manifest .a11y-scan/manifest.txt
|
||||
|
||||
storybook:a11y:changed:
|
||||
desc: "a11y gate over stories changed vs a base ref (default origin/main)"
|
||||
desc: "a11y gate over the stories this branch affects (default base origin/main)"
|
||||
summary: |
|
||||
Scans only the stories this branch touches, which is what pull requests
|
||||
run — a full scan takes ~30 minutes, far too long to sit in front of every
|
||||
merge. The nightly job covers the rest of the suite.
|
||||
Scans the stories a branch affects, which is what pull requests run — a
|
||||
full scan takes ~30 minutes, far too long to sit in front of every merge.
|
||||
A story is affected if its file changed, or if a same-named sibling
|
||||
source file changed (editing Button.tsx or Button.css re-scans
|
||||
Button.stories.tsx — the story renders the live component, so a component
|
||||
edit changes what the story shows without touching the story file).
|
||||
Changes that ripple further than a component's own stories are covered by
|
||||
the nightly full sweep.
|
||||
|
||||
Pass a base ref through CLI_ARGS, e.g.
|
||||
task frontend:storybook:a11y:changed -- origin/release
|
||||
deps: [install, storybook:browser]
|
||||
vars:
|
||||
BASE: '{{.CLI_ARGS | default "origin/main"}}'
|
||||
# Stories touched by this branch, plus any not yet committed.
|
||||
CHANGED:
|
||||
sh: |
|
||||
{ git diff --name-only --diff-filter=d {{.CLI_ARGS | default "origin/main"}}...HEAD -- '*.stories.ts' '*.stories.tsx';
|
||||
git diff --name-only --diff-filter=d -- '*.stories.ts' '*.stories.tsx';
|
||||
git ls-files --others --exclude-standard -- '*.stories.ts' '*.stories.tsx'; } \
|
||||
| sed 's|^frontend/||' | sort -u | tr '\n' ' '
|
||||
sh: node .storybook/a11y-changed.mjs {{.CLI_ARGS | default "origin/main"}}
|
||||
cmds:
|
||||
- cmd: |
|
||||
if [ -z "{{.CHANGED}}" ]; then
|
||||
echo "a11y: no story files changed vs {{.BASE}} — nothing to check"
|
||||
echo "a11y: no story files affected vs {{.BASE}} — nothing to check"
|
||||
exit 0
|
||||
fi
|
||||
bash .storybook/a11y-scan.sh {{.CHANGED}}
|
||||
node .storybook/a11y-scan.mjs {{.CHANGED}}
|
||||
node .storybook/a11y-check.mjs --in .a11y-scan --manifest .a11y-scan/manifest.txt
|
||||
|
||||
storybook:a11y:record:
|
||||
desc: "Re-record the a11y baseline (run after intentionally fixing/adding violations)"
|
||||
deps: [install, storybook:browser]
|
||||
cmds:
|
||||
- bash .storybook/a11y-scan.sh
|
||||
- node .storybook/a11y-scan.mjs
|
||||
- node .storybook/a11y-check.mjs --in .a11y-scan --manifest .a11y-scan/manifest.txt --record
|
||||
|
||||
# ============================================================
|
||||
|
||||
Reference in New Issue
Block a user