Files
OwnCord/docs/plans/infrastructure-roadmap.md
J3vbandClaude Opus 5 2a37f386f9 B1-3: repository hygiene gates (RL-19 / L-13, S-05) (#1414)
* chore(format): one Prettier config at the repository root

Every formatting rule in this repository lived under Client/ and covered
exactly two globs: Client/src/**/*.ts and Client/tests/**/*.ts. Root Markdown,
all of docs/, every YAML and JSON, all CSS, the root scripts and
tools/mcp-introspect were formatted by nothing. There was no .editorconfig.

The obvious fix -- a second Prettier config at the root for "everything else"
-- gives two configs and two ignore files that can silently disagree about the
same file. So the root takes ownership instead: config, ignore file and gate
move up, and Client/ folds in. Client's inline "prettier" block, its
.prettierignore, its format/format:check scripts and its now-unused prettier
devDependency are all deleted; knip would have failed client-check on that last
one.

The .prettierrc.json values are lifted byte-for-byte from Client/package.json,
which is what keeps the reformat commit free of client TypeScript churn: 87
tracked files need reformatting and not one of them is under Client/src or
Client/tests.

.prettierignore carries only what .gitignore does not. Prettier 3 reads the
root .gitignore by default, so node_modules/, dist/, coverage/,
Client/src/generated/ and docs/security-findings/ need no entry. It does NOT
read nested .gitignore files, which is why .remember/ is listed explicitly --
38 untracked per-machine scratch files were otherwise able to turn a shared
gate red. graphify-out/ is listed because its seven files are tracked and
.graphify_labels.json is signed byte-for-byte by its .sig, so formatting it
would silently invalidate the signature.

check:hygiene is registered in scripts/run.mjs and folded into check and
release:preflight. It deliberately contains no `gofmt -l` step: gofmt -l prints
offenders and still exits 0, so it cannot fail a build. Go formatting is
enforced separately.

shellcheck and actionlint take their file lists from `git ls-files`, never a
filesystem glob -- .claude/worktrees/ holds a gitignored pre-flatten copy of
the tree with three .sh files a glob would happily lint.

This commit leaves the tree non-conformant on purpose. The reformat is the next
commit, so the 87-file diff is reviewable separately from the rule that caused
it.

Not included: editorconfig-checker. .editorconfig is the editor baseline the
audit asked for; Prettier, gofmt and rustfmt already fail CI on the same
indentation and newline rules, so a fourth tool checking them again is a gate
with no failure mode of its own.

Verified: `npx prettier --check .` names 87 tracked files and zero untracked
ones; the same command listed 38 .remember/ scratch files before the ignore
entry and none after. `node scripts/run.mjs --list` resolves check:hygiene to 8
shell targets and 4 workflow targets. Both package.json files parse.

Refs RL-19 / L-13, S-05.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore(format): reformat the tree to the repository Prettier rules

Mechanical. This commit is `npx prettier --write .` and nothing else -- the
rule that caused it landed in the previous commit so this diff can be reviewed
as a transformation rather than as 84 files of hunks.

84 tracked files: 54 Markdown, 7 .mjs, 7 JSON, 6 YAML, 4 .js, 3 CSS, 2
TypeScript (the two Playwright configs at Client's root, which the old
Client/src + Client/tests globs never covered). No file under Client/src or
Client/tests moves, because .prettierrc.json carries Client's former inline
values byte-for-byte.

Prettier rewrote 87 files, not 84. The three in .github/ISSUE_TEMPLATE/ had
CRLF on disk and differ only in line endings, which .gitattributes
(`* text=auto eol=lf`) already normalises, so their committed blobs are
unchanged. Worth knowing before someone reconciles the two numbers.

The largest single diff is .superpowers/findings-ledger.json at 7976 lines
rewritten. That is safe to format: nothing writes the ledger programmatically
-- render-ledger.mjs reads it and writes only FINDINGS.md -- so no tool will
fight Prettier over its style on the next hunt. FINDINGS.md itself is ignored
as generated.

Verified: `npx prettier --check .` reports "All matched files use Prettier code
style", so the pass is both complete and idempotent. All 7 reformatted JSON
files were parsed before and after and compared as values: semantically
identical, zero content changes. `node .superpowers/render-ledger.mjs --check`
still reports 348 valid findings and leaves FINDINGS.md untouched.
`node scripts/check-doc-counts.mjs` still passes its selftest and still agrees
on 27 claims across 9 watched documents -- table realignment did not break the
patterns it matches on. `node scripts/run.mjs --list` still parses.

Refs RL-19 / L-13, S-05.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore(lint): enforce Go formatting in the Server linter

S-05: repository-wide Go formatting was not a required gate. The only gofmt
enforcement anywhere was .githooks/pre-commit, which is opt-in per clone
(`npm run hooks:install`), only sees staged files, and warns-and-skips when
gofmt is off PATH.

The obvious fix -- a `gofmt -l` step in CI -- does not work: `gofmt -l` prints
its offenders and still exits 0, so the step passes no matter what it finds.
scripts/run.mjs has the same problem, which is why check:hygiene has no Go step
either.

So gofmt goes where it can actually fail something: Server/.golangci.yml. The
file was already `version: "2"` but had no `formatters:` block at all, so the
19 enabled linters ran with zero formatters. In v2 gofmt/gofumpt/goimports
moved out of `linters.enable` into their own section with its own exclusions.
Adding it there means the gate reports through the Lint step of "Server Build &
Test", which is already pinned as required on dev -- no new job and no new pin.
Every tracked .go file is under Server/ (551 of them, one go.mod), so
Server-scoped is repository-wide here.

One file was genuinely misformatted: a one-space struct field alignment in
Server/admin/handlers_users_broadcast_test.go, fixed in the same commit because
a single line does not need its own reformat commit.

Trap worth recording: `gofmt -l .` on a Windows working tree lists every file
that has CRLF on disk, because gofmt normalises line endings. That reported 18
offenders here, 17 of them ghosts. The blobs are all LF -- .gitattributes
forces `eol=lf` -- so CI never saw them, and the honest test is to run gofmt
over `git show HEAD:<file>` rather than the working copy. Doing that across all
551 tracked Go files found exactly the one real offender above.

Verified both directions with golangci-lint v2 locally: `golangci-lint run
./...` reports 0 issues on the formatted tree; appending a misformatted
function to Server/auth/constants.go produces 2 gofmt findings; appending the
same misformatted function to Server/db/dbgen/admin.sql.go produces 0, so the
exclusion holds. Both files restored and verified clean afterwards.

Refs RL-19 / L-13, S-05.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(scripts): escape the NUL separator instead of embedding one

The `tracked()` helper added earlier in this branch splits `git ls-files -z`
output on NUL. The separator was written as a literal NUL byte rather than the
two-character JavaScript escape, so scripts/run.mjs became a binary file: `git
diff` refused to show it, `grep` reported "Binary file matches" instead of the
line, and `* text=auto` in .gitattributes stops normalising line endings for a
blob it detects as binary.

The code worked -- splitting on a raw NUL and splitting on "\0" are the same
operation -- which is exactly why this is worth fixing before it is inherited.
A source file that tooling classifies as binary is a file nobody can review.

Verified: zero NUL bytes remain, `grep -n "split("` now prints line 50 instead
of "Binary file matches", `node scripts/run.mjs --list` still resolves the same
8 shell and 4 workflow targets, and prettier still reports the file clean.

* chore(lint): enforce Rust formatting

Rust had no formatting gate of any kind: no rustfmt.toml, no `cargo fmt`
anywhere in CI, in scripts/run.mjs, in the Makefile or in the git hooks. Clippy
was the only Rust gate, and clippy does not check layout.

`cargo fmt --all -- --check` now runs in the rust-tests job, ahead of clippy: a
formatting failure is cheap to produce and cheap to fix, and there is no reason
to spend a clippy pass to surface one. The stable toolchain in that job
requested `components: clippy` only, so rustfmt is added there.

Only that job. ci.yml has a second, byte-identical `Install Rust` block in
tauri-build; it stays clippy-only, because a full desktop build is the wrong
place to discover a misplaced brace.

No rustfmt.toml. The default profile is the point of a baseline -- a config
file here would be a second opinion about style with nothing to say.
Client/src-tauri is a single `[package]`, not a workspace, so `--all` is a
safeguard against a future member rather than a fan-out today.

Verified: `node scripts/run.mjs --list` resolves check:rust to three steps with
`cargo fmt --all -- --check` first, `npm run format` now also runs `cargo fmt
--all`, and prettier reports ci.yml, run.mjs and the ci-check skill clean.
`cargo fmt --all -- --check` currently fails on 13 files -- that is the
reformat, and it is the next commit.

Refs RL-19 / L-13.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore(format): reformat the Rust crate to rustfmt defaults

Mechanical. This commit is `cargo fmt --all` and nothing else; the gate that
demands it landed in the previous commit so this diff is reviewable on its own.

13 of the 17 tracked .rs files, +343/-164. The crate had never been formatted,
so the changes are the usual first-run set: aligned trailing comments collapsed
to single spaces, single-element slice literals folded onto one line, long
method chains broken across lines, closure bodies expanded into blocks.

Verified: `cargo fmt --all -- --check` is clean, so the pass is complete and
idempotent. `cargo clippy --all-targets -- -D warnings` finishes with no
warnings, and `cargo test --lib` reports 115 passed / 0 failed -- identical to
before the reformat, which is what "mechanical" has to mean for a commit that
touches this much of the crate.

Refs RL-19 / L-13.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(scripts): make the root facade actually run on Windows

Adding the first gate that a contributor would run from the repository root
exposed two bugs in the facade, both of which made it silently wrong on the
platform this project is developed on.

1. Every npm and npx step failed. bin() appends `.cmd` on Windows, but Node
   refuses to spawn a .cmd or .bat with shell:false -- the CVE-2024-27980
   mitigation -- and fails with EINVAL and a *null* exit status. run.mjs only
   special-cased ENOENT, so the result was `FAILED: npx prettier --check .
   exited null` with nothing to explain it. check:client has three npm steps and
   has never been able to run here.

   Fixed by spawning only the npm shims through a shell. They are concatenated
   into a single command string rather than passed as an args array, because
   shell:true plus a separate array is deprecated (DEP0190) and prints a warning
   on every invocation; no argument in this file contains a space.

2. Every optional() step was skipped, always. onPath() shelled out to
   `where` on Windows, but where.exe lives in C:\WINDOWS\System32, which a Git
   Bash PATH does not necessarily contain -- on this machine PATH carries
   System32\Wbem, System32\WindowsPowerShell\v1.0 and System32\OpenSSH but not
   System32 itself. The probe could not start, `probe.status === 0` was false,
   and golangci-lint and sqlc reported as "not installed" while installed.

   Fixed by resolving against PATH and PATHEXT directly. No subprocess, and no
   dependency on which directories happen to be on PATH.

A spawn error other than ENOENT now reports its code instead of surfacing as a
null exit status.

Verified: before, `node scripts/run.mjs check:hygiene` died with "exited null"
and both optional steps printed SKIP with the tools present on PATH. After, the
same command runs prettier, shellcheck and actionlint and prints
"check:hygiene: passed", with no deprecation warning. `golangci-lint` is
detected by the new onPath where the old one missed it.

Refs RL-20 / L-14.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore(format): ignore build output that nested gitignores hide

Prettier honours the root .gitignore and no other. Every build and scratch
directory in this repository is ignored by a *nested* one -- Client/.gitignore,
.serena/.gitignore, .superpowers/sdd/.gitignore -- so none of them were
excluded from the new repository-wide gate.

The effect is not subtle. Running `cargo test` once drops roughly 850
formattable files into Client/src-tauri/target/, and the hygiene gate goes from
clean to "Code style issues found in 939 files". CI never sees it, because a
fresh checkout has no build output; every contributor sees it on their second
command.

Mirrors the three nested files rather than inventing a list: dist, coverage,
playwright-report, test-results, .vite, src-tauri/target and src-tauri/gen from
Client/.gitignore, plus .serena/ and .superpowers/sdd/. node_modules needs no
entry -- Prettier ignores it by default.

Verified: `npx prettier --check .` reports "All matched files use Prettier code
style" with a fully populated Client/src-tauri/target/ present on disk, and
still names README.md when a misformatted table is appended to it.

Refs RL-19 / L-13.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore(ci): shellcheck, actionlint, and a repository hygiene job

The last two gates RL-19 asks for. Neither existed: the shell scripts were
never linted, the workflows were never syntax-checked, and .githooks/pre-commit
carried hand-written `# shellcheck disable=` directives that nothing had ever
read.

New `hygiene` job, ubuntu-only and root-scoped, modelled on docs-consistency
for the same reason: every gate in it is platform-independent text analysis,
and .gitattributes pins eol=lf so a second OS would only re-prove line endings.
It runs `npm run check:hygiene` -- the same entry point a contributor runs, not
a parallel copy of the commands.

shellcheck ships in the runner image. actionlint does not, so it is pinned by
version and verified by sha256: an installer script piped from a branch would
be the one unverified download in a workflow file that pins every action by
commit SHA.

Prettier's step moves here from client-check, where it no longer belongs.

Both linters found real defects.

shellcheck, 3 findings in 8 scripts. Two are SC1125 errors in
.githooks/pre-commit: `# shellcheck disable=SC2086 - repo paths contain no
spaces` is not a valid directive. Trailing prose makes shellcheck discard the
rest of the line, so neither suppression was ever in effect -- and one of the
two was written earlier in this same branch, which is a fair demonstration of
why the gate is worth having. The prose moves to its own line above. The third
is SC2015 in start-server.sh, rewritten as an explicit if.

actionlint, 5 findings, all inside `run:` blocks it shellchecks once shellcheck
is on PATH. Three SC2015 in load-baseline.yml, rewritten as explicit ifs. Two
SC2035 in release.yml, where `sha256sum *` should not become `sha256sum ./*`:
the comment four lines above records that ParseChecksumFile exact-matches the
last field, so a "./" prefix would strand every deployed server exactly as a
"windows/" prefix would. `sha256sum -- *` satisfies the linter and leaves the
output bytes identical.

Verified all three gates in both directions with shellcheck 0.10.0 and
actionlint 1.7.7 on PATH. Passing: `node scripts/run.mjs check:hygiene` prints
"check:hygiene: passed" with all three steps run, not skipped. Failing:
appending `bait_fn() { cat $1; }` to Server/scripts/voice-test.sh fails on
SC2086; changing a runs-on to `ubunt-latest` fails on runner-label; appending a
misformatted table to README.md fails prettier. All three files restored and
confirmed clean afterwards. actionlint validates the new job in ci.yml itself.

Refs RL-19 / L-13, S-05.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(plans): record B1 progress through B1-3

The header still read "B1-0 is complete; B1-1 is the next step" three merged
phases later. A plan that misstates where it is costs a reader the same
confusion whether it is stale by one phase or three.

B1-0 (#1410), B1-1 (#1411), B1-2 (#1412) and B1-3 (this branch) are done; B1-4,
dependency automation, is next.

Verified: `node scripts/check-doc-counts.mjs` still agrees on 27 claims across
9 watched documents -- this file is one of them -- and prettier reports it
clean.

* chore(ci): pin Repository Hygiene as a required check on dev

The second half of S-05. Its acceptance criterion is "tree is formatted AND a
fast required gate fails future drift" -- a check that runs but is not pinned
lets a formatting regression merge, so the gate is not a gate until this lands.

The name was read off PR #1414 with `gh pr checks` after the job reported
`pass` in 26s, not copied out of ci.yml. That order matters: the B0 script
records that three pinned names exist in no workflow file at all, and that a
required check which never reports blocks every PR forever.

Extends the existing script rather than adding a second one, per the B1 plan.

Also records, in the "deliberately NOT pinned" list, that Docs & Ledger
Consistency reports and passes on a dev PR yet is unpinned. That reads as an
oversight from the 2026-08-25 pass rather than a decision, but it belongs to
G-04, so it is documented here and not changed.

NOT APPLIED YET. Running this script now would pin a check that PR #1413 cannot
report -- its branch predates the hygiene job, so the job does not exist in its
workflow file and the check would never arrive. Run it after #1414 merges;
#1413 needs a rebase onto dev regardless.

Verified: shellcheck clean, the embedded JSON parses, and `check:hygiene`
passes with prettier, shellcheck and actionlint all running.

Refs S-05, RL-14 / G-03.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 18:00:26 +00:00

183 lines
11 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Infrastructure roadmap — design
Date: 2026-08-15
Status: implemented 2026-08-15 (same PR), with two deliberate leftovers:
the TOTP/partial-auth persister seam (Track 2 §3 — lowest impact, cut to
bound the change) and published capacity numbers (Track 3 §7 — the
`load-baseline` workflow now exists to produce them; publish only measured
values).
## Problem
OwnCord is deliberately single-instance (D8, `docs/architecture/system-overview.md`),
and that decision holds. A multi-model review pass (inventory sweep, eight review
lenses, adversarial verification) concluded the runtime is further along than the
operations story: the biggest growth risks are operational gaps, not throughput
ceilings. This roadmap records the verified, non-sensitive recommendations in
three tracks. Findings with security-sensitive detail were reported to the
maintainer separately per `docs/security.md` and are intentionally not itemized
here.
Overall verdict: no re-architecting needed. The codebase already fixes its own
bottlenecks where it finds them (session-touch throttle, batched audit/event
writers, WAL reader/writer pool split, the auth lockout persister seam). The
work below lets a single instance absorb roughly a 10x user increase without
revisiting D8.
## Track 1 — Raise the single-instance ceiling
Ordered by leverage. All stay inside D8; no new subsystems.
1. **Hub dispatch-loop liveness.** The hub's panic breaker (3 panics/60s,
`Server/ws/hub.go`) stops broadcast delivery permanently with nothing
observing it — clients still connect and appear online. On trip: `os.Exit(1)`
so a supervisor restarts the process, and expose dispatch-loop liveness on
`/health`. Do not attempt in-process self-recovery.
2. **Connection capacity guardrail.** Add a configurable global ceiling on
concurrent WebSocket connections, checked before the upgrade and returning
503 when reached. Static value; no adaptive logic.
3. **Presence broadcast coalescing.** Connect/disconnect presence broadcasts go
through the sequenced path and fan out to all clients; a reconnect storm
(proxy blip, deploy) multiplies that. Coalesce into one frame per
~250500 ms window. Do **not** restructure `seqMu`'s fan-out itself — per-client
FIFO ordering depends on it (see "What not to do").
4. **Narrow permission-cache invalidation.** Role-scoped channel-override edits
call `InvalidateAll()` and then `RefreshChannelVisibility`, repopulating
~2×N entries synchronously inside the admin request. Narrow to the affected
role's users — the per-user endpoints already use `InvalidateUser` with
exactly this rationale (`Server/admin/handlers_channel_perms.go`).
5. **Read-state write short-circuit.** `channel_focus`/`mark_read` UPSERT the
read-state row on the single writer even when it is already correct
(`Server/service/channel.go`). Skip the write when `latestID` matches and
`mention_count` is 0; optionally debounce bursts. Same shape as the
session-touch throttle already in `Server/api/middleware.go`.
6. **Single-process file lock.** Take an exclusive flock on a `.lock` beside the
SQLite file and fail fast with a legible message (warn-and-continue if the
lock syscall errors — network filesystems). Process-local presence/replay
state assumes one process owns the DB; make that assumption enforced.
7. **Small DB items.** Make the replay ring (1000) and cold-replay cap (5000)
configurable; add `database.max_readers` with a sane bound; rewrite
`DeleteExpiredSessions` to be index-friendly (note: `expires_at` is stored in
RFC3339 `T` format — format the cutoff to match or migrate the data); gate
boot-time `ANALYZE` on schema change plus a cheap `PRAGMA optimize`; add a
table-driven test for the read/write SQL router (`isReadOnlySQL`).
## Track 2 — Cheap seams for a multi-instance future
Interfaces and documentation only. Nothing here builds distributed systems.
1. **Storage backend interface.** `*storage.Storage` is threaded concretely
through ~9 handler signatures. Carve a consumer-side interface in `api/`
(repo precedent: `service.Store`). Note `Open` must return
`io.ReadSeekCloser` + size/modtime because both serve paths use
`http.ServeContent` — that constraint is exactly what makes an S3 backend
nontrivial, and discovering it now is the point. Do not implement S3.
2. **Split the admin CIDR list by purpose.** `/admin`, `/api/v1/metrics`, the
Prometheus exporter, and the LiveKit webhook all share `admin_allowed_cidrs`.
The webhook is already cryptographically authenticated; metrics scraping and
human admin access are different trust domains. Separate config keys so
moving one off-box never widens another. Cheapest seam for voice as a
separate scaling unit.
3. **Persist TOTP/partial-auth stores via the existing persister seam.**
`RateLimiter` already has the optional-persister shape; give
`UsedTOTPCodeStore`/`PartialAuthStore` the same (store hashes, not raw
codes). Do **not** persist the rate-limiter sliding windows — hottest path,
benefit only exists post-multi-instance.
4. **Document the fifth D8 blocker.** Boot-time presence/voice reset
(`Server/main.go`) is a single-instance assumption missing from D8's blocker
list. One doc bullet: "process-local presence/voice state, wiped and rebuilt
per process."
## Track 3 — Ops hygiene
The highest-impact track. Ordered.
1. **Implement the backup scheduler and retention — or visibly disable the
controls.** `backup_schedule`/`backup_retention` exist in the settings table,
admin UI, and API docs, but nothing reads them; a fresh install shows
"Daily" selected and never backs up. Implement inside the existing 15-min
maintenance loop (retention is in days, per the UI), or grey the controls
out today. Do not build a general job scheduler.
2. **Enrich the default-build metrics surface.** `/api/v1/metrics` omits signals
already computed in memory: reconnect tier stats, event-persister stats,
writer-pool `WaitCount`/`WaitDuration` (the single most direct signal for the
single-writer bottleneck), aggregate per-client backpressure counters, and
permission-cache hit/miss. ~60 lines across ~4 files; do this before
touching the otel path. Also wire or delete the seven declared-but-never-
recorded OTel instruments in `Server/telemetry/metrics.go`.
3. **Make `/health` honest.** It returns a static "ok" — never checks the DB,
disk, or hub dispatch loop. Add a bounded `SELECT 1`, disk-free check, and
hub liveness; return 503 with a reason. Cache the result — the endpoint is
unauthenticated and rate-limit exempt.
4. **Boot-smoke release artifacts.** The release pipeline signs and publishes
server binaries and a Docker image it never executes. Boot each artifact
against a scratch dir, poll `/health`, kill it; gate signing/publishing on
that. This is the failure whose blast radius scales with adoption via
self-update.
5. **Bare-metal Linux posture.** Ship a systemd unit template (note:
`ProtectSystem=strict` breaks the self-updater unless the install dir is
writable; ACME needs `AmbientCapabilities=CAP_NET_BIND_SERVICE`;
`TimeoutStopSec=35` matches the 30s drain), a "Linux (systemd)" deployment
section, and a cron backup one-liner. Add a "Reverse Proxy Topology" section
with a working nginx snippet — and state correctly that LiveKit _signaling_
is already proxied at `/livekit/*`; only WebRTC media (UDP range / TCP
fallback) must be directly reachable.
6. **Backup robustness.** Make the backup directory configurable (mirror the
`SetDatabasePath` plumb), document an optional post-backup hook command for
off-host shipping (rsync/rclone left to the operator), remove the output
file on `VACUUM INTO` error, and run `PRAGMA integrity_check` before listing
a backup as restorable. Document that backups stall writes for their
duration and schedule them off-peak. No S3, no manifests.
7. **Fix the k6 load script, then publish one capacity number.**
`Server/scripts/k6/ws-load.js` predates the envelope protocol: auth fails on
the first frame, three of four message types are wrong, and the only
assertion checks HTTP 101 — a fully broken run reports green. Fix it, gate
VU-connected on `auth_ok`/`ready`, add a `workflow_dispatch`-only job, and
publish one reference sizing (connections vs p99 broadcast latency vs
CPU/RAM) naming the two real bottlenecks. Do not gate main CI on it.
8. **Config and admin-settings honesty.** Warn (never fail) on unknown config
keys — capture the koanf key set after the defaults layer as the allow-list
and diff a second instance loaded from the file. Remove or disable the five
admin-settings fields nothing reads (`server_icon`, `backup_schedule`,
`backup_retention`, `max_upload_bytes`, `voice_quality`) and document which
require config.yaml + restart. Add a boot-time disk-free warning and a
metric (needs a build-tagged Windows path). Return 507/503, not 400, when
upload storage fails at the OS level.
9. **CI/release polish.** Add a `concurrency` group to `release.yml`
(`cancel-in-progress: false`); move `client-check`/`client-tests` off
windows-latest or write down why they are there; record a graduation
criterion for the non-blocking admin-e2e job; record the macOS scope
decision near D8 rather than adding an unsigned build. Add the Tailscale
CGNAT range note to `docs/tailscale.md` (admin routes 403 by default from
`100.x.y.z` addresses).
## What not to do
- Do not decompose the hub's `seqMu` fan-out or make its queue sizes tunable —
per-client FIFO ordering depends on the current structure
(`Server/ws/hub_broadcast.go`, `Server/ws/CLAUDE.md`). Architecture fix or
nothing; never a dial.
- Do not derive broadcast audience from pubsub subscribers —
`hub_broadcast.go` documents why that was rejected.
- Do not build S3, a job scheduler, cloud backup shipping, disk-based admission
control, adaptive connection limits, or settings hot-reload.
- Do not persist rate-limiter sliding windows or carve a pub/sub-slash-replay
interface — speculative abstraction over the most delicate code in the repo.
- Do not publish capacity numbers before the k6 script is fixed.
- Do not attempt hub self-recovery after the panic breaker trips.
## Suggested sequencing
1. Track 3 #1 (backup scheduler) — the current UI state misleads operators.
2. Track 3 #2 + #3 (metrics + health) — everything in Track 1 is guesswork
without these signals.
3. Track 3 #4 (release boot-smoke) — blast radius scales with adoption.
4. Track 3 #5 + #6 (systemd + proxy docs + backup dir) — one coherent
bare-metal pass; also the prerequisite for the hub breaker `os.Exit(1)`.
5. Track 1 #1 + #2 (hub liveness + connection ceiling) — small, self-contained,
no locking changes.
Follow-up review passes suggested where this one was thin: release-path supply
chain, TLS/ACME renewal failure handling and secrets at rest, and voice/LiveKit
failure and scaling behavior.