a11y scan: generate required assets, and fail when a story file can't load (#7201)

## What

Two related bugs found while looking at why #7187's a11y check behaves
differently on CI than locally.

### 21 stories were never being scanned on CI

The scan tasks only depended on `install`, not `prepare`. On a fresh
checkout that means the generated icon set
(`editor/src/assets/material-symbols-icons.json`, gitignored) doesn't
exist, so every story that reaches `LocalIcon` fails to import:

```
Failed to resolve import "../../../assets/material-symbols-icons.json"
from "editor/src/core/components/shared/LocalIcon.tsx"
```

On CI that was four story files / 21 stories, every run. It works
locally only because our trees already have the file from a previous
build. The scan tasks now depend on `prepare`, like the `build:*` tasks
do.

### The gate reported those runs as clean

Worse than the missing stories: a file that fails to import produces a
**failed suite with no assertions**. Every check in `a11y-check.mjs`
reads assertions, so the file satisfied the manifest, contributed
nothing to compare, and the run printed `✓ no a11y regressions`.

An assertion-less failed suite now fails the gate and points at the scan
log for the underlying resolve error. `--record` refuses in the same
situation, so a baseline can't be written that quietly drops those
stories.

Also switched the affected-story emptiness test to single quotes, since
that list now carries its own per-path quoting (it was producing `[ -z
""a" "b"" ]`).

## Testing

- Deleted the generated asset to reproduce a fresh checkout: the gate
**fails** with the file named and the cause explained, where before it
printed `✓ no a11y regressions` and exited 0.
- With the `prepare` dependency the task regenerates the asset itself
and the previously-invisible files scan: 21 stories, 35 story-rule
pairs, all already baselined.
This commit is contained in:
Reece Browne
2026-07-30 08:13:41 +00:00
committed by GitHub
parent 4dc0927104
commit b35329c8f5
2 changed files with 40 additions and 10 deletions
+7 -7
View File
@@ -184,13 +184,13 @@ tasks:
storybook:
desc: "Start Storybook dev server"
deps: [install]
deps: [prepare]
cmds:
- npx storybook dev -p 6006 {{.CLI_ARGS}}
storybook:build:
desc: "Build static Storybook"
deps: [install]
deps: [prepare]
cmds:
- npx storybook build {{.CLI_ARGS}}
@@ -204,7 +204,7 @@ tasks:
storybook:test:
desc: "Scan every story in real Chromium: it must render and pass axe"
deps: [install, storybook:browser]
deps: [prepare, storybook:browser]
cmds:
# Runs each story as a browser test. Pass a filter through, e.g.
# task frontend:storybook:test -- Button
@@ -212,7 +212,7 @@ tasks:
storybook:a11y:
desc: "a11y regression gate over every story: fail only on NEW axe violations"
deps: [install, storybook:browser]
deps: [prepare, storybook:browser]
cmds:
- node .storybook/a11y-scan.mjs
- node .storybook/a11y-check.mjs --in .a11y-scan --manifest .a11y-scan/manifest.txt
@@ -231,14 +231,14 @@ tasks:
Pass a base ref through CLI_ARGS, e.g.
task frontend:storybook:a11y:changed -- origin/release
deps: [install, storybook:browser]
deps: [prepare, storybook:browser]
vars:
BASE: '{{.CLI_ARGS | default "origin/main"}}'
CHANGED:
sh: node .storybook/a11y-changed.mjs {{.CLI_ARGS | default "origin/main"}}
cmds:
- cmd: |
if [ -z "{{.CHANGED}}" ]; then
if [ -z '{{.CHANGED}}' ]; then
echo "a11y: no story files affected vs {{.BASE}} — nothing to check"
exit 0
fi
@@ -247,7 +247,7 @@ tasks:
storybook:a11y:record:
desc: "Re-record the a11y baseline (run after intentionally fixing/adding violations)"
deps: [install, storybook:browser]
deps: [prepare, storybook:browser]
cmds:
- node .storybook/a11y-scan.mjs
- node .storybook/a11y-check.mjs --in .a11y-scan --manifest .a11y-scan/manifest.txt --record