From 8cb0ec9e3553bc1df670c50a6ead02a52415c855 Mon Sep 17 00:00:00 2001 From: J3vb <192430104+J3vb@users.noreply.github.com> Date: Sun, 30 Aug 2026 16:18:54 +0200 Subject: [PATCH] =?UTF-8?q?feat(b3-6):=20contract=20drift=20=E2=80=94=20ge?= =?UTF-8?q?nerated=20route,=20table=20and=20config-key=20indexes=20with=20?= =?UTF-8?q?a=20CI=20drift=20check=20(#1456)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(b3-6): contract drift — generated route, table and config-key indexes B3-6 item 9 (workstream 10). `check:server` already diffs the two generators; this adds a third for the three server contracts that only prose described until now. `Server/cmd/gendocs` rewrites one marked block per document: - `docs/api.md` "Route index (generated)" — 111 rows from `chi.Walk` over the production router built with uploads, voice and the GIF proxy on, the same scaffolding `api/absence_contract_test.go` uses. Carries that test's vacuity guards: fewer than 100 routes, or no `/admin/` route, fails the run. - `docs/schema.md` "Table index (generated)" — 34 rows from `sqlite_master` and `pragma_table_info` on an in-memory database with the migrations applied. sqlc exposes no catalog, so the migrated schema is the catalog. - `docs/server-configuration.md` "Key index (generated)" — 56 keys from the koanf struct tags, each mapped to the `###` section of the hand-written reference that names it. A key documented nowhere fails the run by name. Output is padded exactly the way Prettier formats a table, so the drift check and the hygiene gate agree instead of undoing each other. Wiring, copied from protocol-verify: `make docs-generate` / `make docs-verify`, a `DOCS_VERIFY` step in `check:server` and the generator in `generate` (`scripts/run.mjs`), a CI step on the ubuntu leg of `server-build-test`, and a `.githooks/pre-commit` block on router, handler, migration, config and generator paths. Everything hand-written in the three documents is untouched. The new `cmd/gendocs` file imports `db` for the catalog, so it takes a boundary row in the B3-0 inventory and `server-boundaries.md` is regenerated with it. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01KmiqjgTuov1stBTB6uGkvo * docs(b3-6): evidence block for item 9 (machine-readable contract drift) Records the three RED controls and their restore, the counts (111 routes, 34 tables, 56 config keys, 0 undocumented), and two corrections to the item's spec: the configuration reference table lives in docs/server-configuration.md, not docs/deployment.md, and sqlc exposes no catalog — the migrated in-memory schema is the catalog. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01KmiqjgTuov1stBTB6uGkvo * fix(b3-6): gendocs — exclude ANALYZE artifacts, honest hook message, admin routes trigger the hook, generate order, width ceiling Review findings on item 9. 1. The table index dropped `sqlite_stat1` / `sqlite_stat4`. `db.Migrate` runs ANALYZE after applying migrations, so those hold planner statistics, not schema — and `sqlite_stat4` exists only because the current modernc.org/sqlite build has STAT4, so a driver bump would have failed the docs drift check on an unrelated dependency PR. Filtered with GLOB (LIKE's `_` is a wildcard), block regenerated, header line's justification corrected: 34 -> 32 tables. 2. The pre-commit message now covers both failure modes — stale blocks are regenerated and staged, a key the tool named as undocumented is documented in docs/server-configuration.md. 3. `Server/admin/.*\.go` added to the hook's trigger: the 34 `/admin/api/*` routes are registered there, not in api/router.go, so a new admin route could commit stale docs locally. 4. `run.mjs` `generate` runs gendocs after `sqlc generate` — gendocs compiles the api package, which imports db/dbgen. 5. The vacuity guard now requires a traversed `/admin/api/` subroute rather than any `/admin/` path, which the per-method mount catch-alls satisfied on their own, so its message is true. `writeTable` gained a comment naming its ceiling: padding counts runes, Prettier counts display width, so a full-width cell would diverge — none exists in the generated content. Evidence block updated for the new table count. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01KmiqjgTuov1stBTB6uGkvo * fix(b3-6): gendocs — generate the route index from the full-tag build with telemetry on; the hook triggers on every api/ and admin/ Go file (Codex P2s on #1456) 1. `/metrics` was missing from the route index. It mounts only when `telemetry.PrometheusHandler()` returns non-nil (api/router.go:431-437), which needs the otel build tag AND telemetry enabled at runtime; the generator ran in the default build with telemetry unset, so the index omitted a production route. The route index is now the superset build. The scaffold config enables telemetry with the Prometheus exporter and the tool calls telemetry.Init the way main.go does, and every invocation passes -tags otel,wazero: Makefile docs-generate/docs-verify, scripts/run.mjs (DOCS_VERIFY and generate), .githooks/pre-commit, the regenCmd quoted into all three block header lines, and the CLAUDE.md row. ci.yml inherits it through `make docs-verify`. The route block's header line now says which build it came from and what is enabled. Rather than a build-tag constant, the tool checks the condition that actually gates the route: if telemetry.Init leaves no Prometheus handler it exits non-zero naming the tags, so the default build cannot quietly generate a short index. Nothing under Server/api or Server/admin carries a build constraint, so wazero adds and removes no route; it rides along so one build serves the whole repository. Route count 111 -> 121 (ten per-method rows for the /metrics mount, the same shape chi gives /admin and /livekit). 2. The pre-commit trigger named individual api/ files and missed client_update.go, whose MountClientUpdateRoute registers a route directly. It is now the whole of Server/api/ and Server/admin/ — naming files individually is how a trigger goes stale — plus the existing migrations/, config/config.go and cmd/gendocs/ patterns. Evidence block updated: route count and the tagged-build decision. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01KmiqjgTuov1stBTB6uGkvo --------- Co-authored-by: Claude Fable 5 --- .githooks/pre-commit | 20 + .github/workflows/ci.yml | 7 + CLAUDE.md | 11 +- Server/CLAUDE.md | 3 + Server/Makefile | 21 +- Server/cmd/gendocs/main.go | 506 ++++++++++++++++++ Server/cmd/gendocs/main_test.go | 193 +++++++ Server/invariants/db_import_boundary.go | 1 + docs/api.md | 134 +++++ docs/architecture/server-boundaries.md | 5 +- ...rver-architecture-guardrails-2026-08-29.md | 58 ++ docs/schema.md | 45 ++ docs/server-configuration.md | 67 +++ scripts/run.mjs | 22 + 14 files changed, 1085 insertions(+), 8 deletions(-) create mode 100644 Server/cmd/gendocs/main.go create mode 100644 Server/cmd/gendocs/main_test.go diff --git a/.githooks/pre-commit b/.githooks/pre-commit index 5cba0335..f00f53c3 100755 --- a/.githooks/pre-commit +++ b/.githooks/pre-commit @@ -60,6 +60,26 @@ if printf '%s\n' "$staged" | grep -qE '^(protocol/schema\.json|Server/cmd/genpro fi fi +# Any api/ or admin/ Go file, the migrations, the config or the generator +# changed -> the regenerated docs index blocks must be part of the same commit. +# The route trigger is deliberately the whole of api/ and admin/: routes are +# registered in router.go, in the *_handler.go files, in client_update.go's +# MountClientUpdateRoute, and in the admin package's own mux — naming files +# individually is how this goes stale. +# +# Inlined like the two blocks above, and for the same reason: make is not on +# PATH on a stock Windows box. -tags otel,wazero is the build the route index +# is generated from; the tool refuses to run without it. +if printf '%s\n' "$staged" | grep -qE '^Server/(api|admin)/.*\.go$|^Server/(migrations/|config/config\.go|cmd/gendocs/)'; then + if command -v go >/dev/null 2>&1; then + (cd Server && go run -tags otel,wazero ./cmd/gendocs \ + && git diff --exit-code ../docs/api.md ../docs/schema.md ../docs/server-configuration.md) \ + || fail "generated docs blocks are stale, or a config key is undocumented — if gendocs named keys above, document them in docs/server-configuration.md; otherwise run 'go run -tags otel,wazero ./cmd/gendocs' in Server/ and stage the result" + else + printf 'pre-commit: WARNING: go not installed; skipping the generated-docs check. CI will run it.\n' >&2 + fi +fi + # Findings ledger changed -> it must still be valid. Unlike the two blocks # above there is nothing to diff: FINDINGS.md is not tracked (RL-07), so a # stale rendering cannot be committed. --check is the whole gate here, and it diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 53ee0eea..9953d460 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -69,6 +69,13 @@ jobs: if: matrix.os == 'ubuntu-latest' run: make protocol-verify + # The route, table and config-key index blocks in docs/ must never drift + # from the mounted router, the migrated schema and config.Config's koanf + # tags. Same one-leg rule as the two checks above. + - name: Verify generated docs (make docs-verify) + if: matrix.os == 'ubuntu-latest' + run: make docs-verify + - name: Run tests with race detection and coverage run: go test -race -timeout 20m ./... -coverprofile=coverage.out -cover diff --git a/CLAUDE.md b/CLAUDE.md index d1786e6b..b762dd95 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,11 +11,12 @@ and schema are documented in `docs/protocol.md`, `docs/schema.md`, and CI fails on drift, and the next generator run silently discards your edit. -| Generated | Source of truth | Workflow | -| ---------------------------------------------------------------------- | ----------------------------------------------- | -------------------------------------------------------------- | -| `Server/db/dbgen/` | `Server/db/queries/*.sql`, `Server/migrations/` | `db-change` skill | -| `Server/ws/message_types.go` **and** `Client/src/lib/protocolTypes.ts` | `protocol/schema.json` | `protocol-change` skill | -| `Client/src/generated/` | `tauri-typegen` | CI patches known typegen bugs — see `.github/workflows/ci.yml` | +| Generated | Source of truth | Workflow | +| ------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | -------------------------------------------------------------- | +| `Server/db/dbgen/` | `Server/db/queries/*.sql`, `Server/migrations/` | `db-change` skill | +| `Server/ws/message_types.go` **and** `Client/src/lib/protocolTypes.ts` | `protocol/schema.json` | `protocol-change` skill | +| `gendocs:*` blocks in `docs/api.md`, `docs/schema.md`, `docs/server-configuration.md` | `Server/api/router.go`, `Server/migrations/`, `Server/config/config.go` | `cd Server && go run -tags otel,wazero ./cmd/gendocs` | +| `Client/src/generated/` | `tauri-typegen` | CI patches known typegen bugs — see `.github/workflows/ci.yml` | ## Bug-hunt ledger diff --git a/Server/CLAUDE.md b/Server/CLAUDE.md index cafe7dd7..d259d576 100644 --- a/Server/CLAUDE.md +++ b/Server/CLAUDE.md @@ -14,6 +14,9 @@ prometheus. `cmd/seed/` fills a dev database (`go run ./cmd/seed -confirm-dev`), `cmd/dbinventory/` prints the `db`-importer table for `docs/architecture/server-boundaries.md` (exits 1 on an unlisted importer). + `cmd/gendocs/` rewrites the route, table and config-key index blocks in + `docs/` and must be run as `go run -tags otel,wazero ./cmd/gendocs` + (`make docs-verify` fails on drift). `scripts/` holds shell/JS tooling only; no Go entry point lives there - `admin/` web admin panel · `updater/` self-update + signature verification · `plugin/` WASM plugin runtime (`-tags wazero`) · `telemetry/` OTel (`-tags otel`) diff --git a/Server/Makefile b/Server/Makefile index f0162450..d3ca1e84 100644 --- a/Server/Makefile +++ b/Server/Makefile @@ -11,13 +11,15 @@ # sqlc-install Install the pinned sqlc version into $GOBIN. # protocol-generate Regenerate WS message-type constants (Go + TS) from ../protocol/schema.json. # protocol-verify Fail if the committed protocol constants are stale (used by CI). +# docs-generate Regenerate the route/table/config index blocks in ../docs. +# docs-verify Fail if those generated blocks are stale (used by CI). # otel-up Start Jaeger + Prometheus for local tracing development. # otel-down Stop and remove the OTel dev containers. SQLC_VERSION := $(shell cat sqlc.version) .PHONY: test test-deadlock fuzz cover cover-all sqlc-install sqlc-generate sqlc-verify \ - protocol-generate protocol-verify otel-up otel-down + protocol-generate protocol-verify docs-generate docs-verify otel-up otel-down test: go test -race -timeout 20m ./... @@ -91,6 +93,23 @@ protocol-verify: exit 1 ; \ ) +# Route, table and config-key indexes in ../docs. Same shape as the two +# generator checks above: regenerate, then fail on any diff. The tool also +# exits non-zero on its own when a config key is documented nowhere. +# +# -tags otel,wazero is not optional: /metrics mounts only when the otel build +# supplies a Prometheus handler, so the default build would generate an index +# missing a production route. The tool refuses to run without it. +docs-generate: + go run -tags otel,wazero ./cmd/gendocs + +docs-verify: + go run -tags otel,wazero ./cmd/gendocs + @git diff --exit-code ../docs/api.md ../docs/schema.md ../docs/server-configuration.md || ( \ + echo "ERROR: generated documentation blocks are stale. Run 'make docs-generate' and commit the result." ; \ + exit 1 ; \ + ) + # Phase B Step 8 — local OTel development stack. # Starts Jaeger (traces) and Prometheus (metrics) in Docker. # Jaeger UI: http://localhost:16686 diff --git a/Server/cmd/gendocs/main.go b/Server/cmd/gendocs/main.go new file mode 100644 index 00000000..d6848753 --- /dev/null +++ b/Server/cmd/gendocs/main.go @@ -0,0 +1,506 @@ +// Command gendocs regenerates the machine-readable index blocks that pin the +// server's three contracts to the documents describing them: +// +// routes -> docs/api.md every route the mounted chi tree serves +// schema -> docs/schema.md every table the migrations create +// config -> docs/server-configuration.md every koanf key, and where it is documented +// +// Each index replaces the text between a pair of HTML-comment markers +// ( … ). Everything +// hand-written around them is left alone. Run it from Server/: +// +// go run ./cmd/gendocs +// +// `make docs-verify` runs that and then `git diff --exit-code` on the three +// documents, so a route, table or config key that changes in code without +// reaching the docs fails the build — the same shape as protocol-verify and +// sqlc-verify. +// +// Two sources are read the way the absence-contract tests read them, because +// no non-test seam exists: the router is rebuilt with every optional family +// switched on and walked with chi.Walk, and the config surface comes from +// reflection over the koanf struct tags. The schema comes from an in-memory +// database with the migrations applied — sqlc exposes no catalog, so the +// migrated database is the catalog. +// +// Output is padded exactly the way Prettier formats a Markdown table, so the +// generated blocks survive the repository's `prettier --check` gate. +package main + +import ( + "bytes" + "context" + "errors" + "fmt" + "io" + "net/http" + "os" + "path/filepath" + "reflect" + "regexp" + "slices" + "strings" + "unicode/utf8" + + "github.com/J3vb/OwnCord/Server/api" + "github.com/J3vb/OwnCord/Server/config" + "github.com/J3vb/OwnCord/Server/db" + "github.com/J3vb/OwnCord/Server/telemetry" + "github.com/go-chi/chi/v5" +) + +// Documents rewritten, relative to Server/ (the directory the tool runs in). +const ( + apiDoc = "../docs/api.md" + schemaDoc = "../docs/schema.md" + configDoc = "../docs/server-configuration.md" +) + +// regenCmd is quoted into every generated block so a reader who spots a stale +// row knows what to run without going looking for it. +const regenCmd = "cd Server && go run -tags otel,wazero ./cmd/gendocs" + +// minRoutes is the vacuity floor the route index inherits from +// TestAbsenceContract_NoFederationDirectoryOrListingRoutes: a walk over an +// empty or wrapped mux would otherwise produce a short table that diffs clean +// on the next run. An empty table is drift too. +const minRoutes = 100 + +func main() { + if err := run(); err != nil { + fmt.Fprintln(os.Stderr, "gendocs:", err) + os.Exit(1) + } +} + +func run() error { + for _, g := range []struct { + name string + path string + gen func(io.Writer) error + }{ + {"routes", apiDoc, genRoutes}, + {"schema", schemaDoc, genSchema}, + {"config", configDoc, genConfig}, + } { + var buf bytes.Buffer + if err := g.gen(&buf); err != nil { + return fmt.Errorf("%s index: %w", g.name, err) + } + raw, err := os.ReadFile(g.path) + if err != nil { + return err + } + out, err := spliceBlock(string(raw), g.name, buf.String()) + if err != nil { + return fmt.Errorf("%s: %w", g.path, err) + } + if out == string(raw) { + continue + } + //nolint:gosec // G703: g.path is one of the three constants above, never input. + if err := os.WriteFile(g.path, []byte(out), 0o644); err != nil { + return err + } + } + return nil +} + +// spliceBlock replaces the text between the gendocs markers for name with +// body, leaving the markers and everything around them untouched. Missing +// markers are an error rather than an append: where a block lives is a +// hand-made editorial decision, not something a generator should guess. +func spliceBlock(doc, name, body string) (string, error) { + start := "" + end := "" + i := strings.Index(doc, start) + j := strings.Index(doc, end) + if i < 0 || j < 0 { + return "", fmt.Errorf("markers %q … %q not found; add the block by hand once, then this tool fills it", start, end) + } + if j < i+len(start) { + return "", fmt.Errorf("marker %q appears before %q", end, start) + } + return doc[:i+len(start)] + "\n\n" + strings.Trim(body, "\n") + "\n\n" + doc[j:], nil +} + +// writeTable renders a GitHub-flavoured Markdown table padded the way Prettier +// formats one: every cell in a column padded to the widest cell in that +// column (the header included, minimum three), and the separator row filled +// with that many dashes. Emitting anything else means `prettier --check` and +// `git diff --exit-code` fight each other forever, each undoing the other. +func writeTable(w io.Writer, header []string, rows [][]string) { + // ponytail: widths are rune counts, Prettier's are display widths — a + // full-width CJK or emoji cell would pad two columns narrow. No generated + // cell holds one (route paths, SQL identifiers, config keys, section + // headings); switch to a display-width count if that ever changes. + widths := make([]int, len(header)) + cells := make([][]string, 0, len(rows)+1) + for _, r := range append([][]string{header}, rows...) { + row := make([]string, len(header)) + for i := range header { + if i < len(r) { + row[i] = escapeCell(r[i]) + } + widths[i] = max(widths[i], utf8.RuneCountInString(row[i]), 3) + } + cells = append(cells, row) + } + printRow := func(r []string) { + for i, c := range r { + r[i] = c + strings.Repeat(" ", widths[i]-utf8.RuneCountInString(c)) + } + printf(w, "| %s |\n", strings.Join(r, " | ")) + } + printRow(cells[0]) + sep := make([]string, len(header)) + for i := range sep { + sep[i] = strings.Repeat("-", widths[i]) + } + printf(w, "| %s |\n", strings.Join(sep, " | ")) + for _, r := range cells[1:] { + printRow(r) + } +} + +// printf writes formatted output to w. Every w here is an in-memory buffer +// that run() then splices into a document, so a write error is not reachable +// and dropping it once beats five checks that can never fire. +func printf(w io.Writer, format string, a ...any) { + _, _ = fmt.Fprintf(w, format, a...) +} + +// escapeCell hides the one character that would end a cell early. Prettier +// writes the same escape, so an escaped pipe round-trips. +func escapeCell(s string) string { return strings.ReplaceAll(s, "|", `\|`) } + +// code wraps s in a Markdown code span. +func code(s string) string { return "`" + s + "`" } + +// joinCode renders a list of identifiers as comma-separated code spans, or an +// em dash when there are none, matching cmd/dbinventory's table style. +func joinCode(items []string) string { + if len(items) == 0 { + return "—" + } + return code(strings.Join(items, "`, `")) +} + +// openMigrated opens an in-memory database with every migration applied. It is +// the schema catalog and the router's dependency both. +func openMigrated() (*db.DB, func(), error) { + database, err := db.Open(":memory:") + if err != nil { + return nil, nil, fmt.Errorf("db.Open: %w", err) + } + if err := db.Migrate(database); err != nil { + _ = database.Close() + return nil, nil, fmt.Errorf("db.Migrate: %w", err) + } + return database, func() { _ = database.Close() }, nil +} + +// genRoutes walks the production router with every optional family switched +// on — uploads, voice, the GIF proxy and telemetry — so the table is the +// whole tree rather than the bare-config subset. The scaffolding is a copy of +// fullRouter in api/absence_contract_test.go plus the telemetry init main.go +// does; test code stays in the test. +// +// The index is the superset build: /metrics mounts only when +// telemetry.PrometheusHandler() returns non-nil, which needs -tags otel, so +// every invocation of this tool passes -tags otel,wazero. Nothing under +// Server/api or Server/admin is itself tag-gated, so wazero adds and removes +// no route; it rides along so one build serves the whole repository. +func genRoutes(w io.Writer) error { + database, closeDB, err := openMigrated() + if err != nil { + return err + } + defer closeDB() + + dir, err := os.MkdirTemp("", "gendocs") + if err != nil { + return err + } + defer func() { _ = os.RemoveAll(dir) }() + + cfg := &config.Config{ + Server: config.ServerConfig{Name: "gendocs", Port: 8443, DataDir: dir}, + Upload: config.UploadConfig{MaxSizeMB: 1, StorageDir: filepath.Join(dir, "uploads")}, + //nolint:gosec // G101: placeholder values so the optional voice routes mount; the router is walked, never served. + Voice: config.VoiceConfig{ + LiveKitAPIKey: "gendocs-key", + LiveKitAPISecret: "gendocs-secret-at-least-32-chars-long", + LiveKitURL: "ws://127.0.0.1:7880", + }, + GIF: config.GIFConfig{APIKey: "gendocs"}, + // /metrics mounts only when telemetry.PrometheusHandler() is non-nil, + // which needs both the otel build tag and telemetry switched on at + // runtime. Without this the index would silently omit a production + // route. + Telemetry: config.TelemetryConfig{Enabled: true, Exporter: "prometheus", ServiceName: "gendocs"}, + } + + ctx := context.Background() + shutdown, err := telemetry.Init(ctx, cfg.Telemetry) + if err != nil { + return fmt.Errorf("telemetry.Init: %w", err) + } + defer func() { _ = shutdown(ctx) }() + // The default build's Init installs a no-op provider, so this is the + // runtime check that the tool was built the way the index claims. + if telemetry.PrometheusHandler() == nil { + return errors.New("telemetry.Init left no Prometheus handler: run this tool as `go run -tags otel,wazero ./cmd/gendocs`, the build the route index is generated from") + } + + handler, _, cleanup := api.NewRouter(cfg, database, "gendocs", nil, nil) + defer cleanup() + + routes, ok := handler.(chi.Routes) + if !ok { + return fmt.Errorf("api.NewRouter returned %T, want a chi.Routes so the mounted tree can be walked", handler) + } + var rows [][]string + admin := 0 + walk := func(method, route string, _ http.Handler, _ ...func(http.Handler) http.Handler) error { + // A real admin subroute, not the per-method `/admin/*` catch-all chi + // emits for the Mount itself — those appear whether or not the walk + // ever descended into the subrouter. + if strings.HasPrefix(route, "/admin/api/") && !strings.HasSuffix(route, "/*") { + admin++ + } + rows = append(rows, []string{method, route}) + return nil + } + if err := chi.Walk(routes, walk); err != nil { + return fmt.Errorf("chi.Walk: %w", err) + } + if len(rows) < minRoutes { + return fmt.Errorf("walked only %d routes; expected the full production router (>= %d)", len(rows), minRoutes) + } + if admin == 0 { + return errors.New("walk saw no /admin/api/ routes; the mounted admin subrouter was not traversed") + } + // chi hands the methods of one pattern back in map order, so the sort is + // what makes two runs produce the same bytes. Sorted on the bare path, + // before the code span goes on: a trailing backtick would order + // `/x` after `/x/y`. + slices.SortFunc(rows, cmpRoute) + for _, r := range rows { + r[1] = code(r[1]) + } + + printf(w, "Generated from the mounted router by %s — do not edit by hand; `make docs-verify` fails when it drifts. %d routes, from the `otel,wazero` build with every optional family enabled (uploads, voice, the GIF proxy, and telemetry with the Prometheus exporter, which is what mounts `/metrics`).\n\n", + code(regenCmd), len(rows)) + writeTable(w, []string{"Method", "Path"}, rows) + return nil +} + +// cmpRoute orders {method, path} rows by path, then method. +func cmpRoute(a, b []string) int { + if c := strings.Compare(a[1], b[1]); c != 0 { + return c + } + return strings.Compare(a[0], b[0]) +} + +// genSchema reads the catalog of the migrated database. sqlc has no catalog +// export (Server/sqlc.yaml declares no plugins and takes its schema from +// migrations/), so the migrated database is the catalog — the same trick the +// absence contract's fullRouter uses to get a real schema without a file. +// +// sqlite_sequence and the FTS5 shadow tables behind messages_fts are kept: +// they are what the migrations create, and a change to either is a schema +// change worth seeing in the diff. The sqlite_stat* tables are dropped — +// db.Migrate runs ANALYZE after applying migrations, so they hold planner +// statistics rather than schema, and which of them exists is a property of +// the SQLite build (sqlite_stat4 only with STAT4 compiled in). Including them +// would fail this drift check on a modernc.org/sqlite bump. +func genSchema(w io.Writer) error { + database, closeDB, err := openMigrated() + if err != nil { + return err + } + defer closeDB() + + ctx := context.Background() + // GLOB, not LIKE: LIKE's "_" is a wildcard, GLOB's is not. + tables, err := queryStrings(ctx, database, + `SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT GLOB 'sqlite_stat*' ORDER BY name`) + if err != nil { + return err + } + if len(tables) == 0 { + return errors.New("the migrated database reports no tables; the catalog read is broken") + } + rows := make([][]string, 0, len(tables)) + for _, t := range tables { + cols, err := tableColumns(ctx, database, t) + if err != nil { + return err + } + idx, err := queryStrings(ctx, database, + `SELECT name FROM sqlite_master WHERE type = 'index' AND tbl_name = ? ORDER BY name`, t) + if err != nil { + return err + } + rows = append(rows, []string{code(t), joinCode(cols), joinCode(idx)}) + } + + printf(w, "Generated from the migrated schema by %s — do not edit by hand; `make docs-verify` fails when it drifts. %d tables: `sqlite_sequence` and the FTS5 shadow tables behind `messages_fts` are included; the `sqlite_stat*` tables `ANALYZE` writes are not, since they hold planner statistics and which of them exists depends on the SQLite build.\n\n", + code(regenCmd), len(rows)) + writeTable(w, []string{"Table", "Columns", "Indexes"}, rows) + return nil +} + +// tableColumns renders one table's columns in declaration order as +// "name TYPE" with the NOT NULL and PK flags PRAGMA table_info reports. +func tableColumns(ctx context.Context, database *db.DB, table string) ([]string, error) { + rows, err := database.QueryContext(ctx, + `SELECT name, type, "notnull", pk FROM pragma_table_info(?) ORDER BY cid`, table) + if err != nil { + return nil, fmt.Errorf("pragma_table_info(%s): %w", table, err) + } + defer func() { _ = rows.Close() }() + var out []string + for rows.Next() { + var name, typ string + var notNull, pk int + if err := rows.Scan(&name, &typ, ¬Null, &pk); err != nil { + return nil, err + } + col := name + if typ != "" { + col += " " + typ + } + if notNull != 0 { + col += " NOT NULL" + } + if pk != 0 { + col += " PK" + } + out = append(out, col) + } + return out, rows.Err() +} + +// queryStrings runs a query whose rows are a single string column. +func queryStrings(ctx context.Context, database *db.DB, query string, args ...any) ([]string, error) { + rows, err := database.QueryContext(ctx, query, args...) + if err != nil { + return nil, fmt.Errorf("%s: %w", query, err) + } + defer func() { _ = rows.Close() }() + var out []string + for rows.Next() { + var s string + if err := rows.Scan(&s); err != nil { + return nil, err + } + out = append(out, s) + } + return out, rows.Err() +} + +// genConfig lists every dotted koanf key and the reference section that +// documents it. A key documented nowhere fails the run by name: the point of +// the index is that the configuration surface and its reference cannot drift +// apart silently, and a row saying "undocumented" would just record the drift. +func genConfig(w io.Writer) error { + raw, err := os.ReadFile(configDoc) + if err != nil { + return err + } + sections := docSections(string(raw)) + keys := koanfKeys(reflect.TypeFor[config.Config](), "") + slices.Sort(keys) + + var missing []string + rows := make([][]string, 0, len(keys)) + for _, k := range keys { + section, ok := sections[k] + if !ok { + missing = append(missing, k) + continue + } + rows = append(rows, []string{code(k), section}) + } + if len(missing) > 0 { + return fmt.Errorf("%d config key(s) are documented nowhere under \"## Config Key Reference\" in %s — document them, then re-run:\n %s", + len(missing), configDoc, strings.Join(missing, "\n ")) + } + + printf(w, "Generated from the `koanf` tags of `config.Config` by %s — do not edit by hand; `make docs-verify` fails when it drifts, and the tool exits non-zero when a key is documented nowhere above. %d keys.\n\n", + code(regenCmd), len(rows)) + writeTable(w, []string{"Key", "Documented in"}, rows) + return nil +} + +// dottedKey matches a code span holding a dotted lower-case key, which is how +// the reference tables name a setting. +var dottedKey = regexp.MustCompile("`([a-z0-9_]+(?:\\.[a-z0-9_]+)+)`") + +// docSections maps each config key named inside the "## Config Key Reference" +// section to the "### " heading it appears under. The scan stops at the next +// "## " heading, which is where the generated index itself lives — so the +// index can never be its own evidence that a key is documented. +func docSections(doc string) map[string]string { + out := map[string]string{} + inReference, heading := false, "" + for line := range strings.Lines(doc) { + line = strings.TrimRight(line, "\n") + switch { + case strings.HasPrefix(line, "## "): + inReference = line == "## Config Key Reference" + heading = "" + continue + case strings.HasPrefix(line, "### "): + heading = strings.TrimPrefix(line, "### ") + continue + } + if !inReference || heading == "" { + continue + } + for _, m := range dottedKey.FindAllStringSubmatch(line, -1) { + if _, seen := out[m[1]]; !seen { + out[m[1]] = heading + } + } + } + return out +} + +// koanfKeys returns every dotted koanf key reachable from t, recursing into +// nested structs the same way koanf unmarshals them. Copied from +// api/absence_contract_test.go, which walks the same surface for the same +// reason: config.Config's tags are the only enumeration of the keys, and +// config.defaults() is unexported. +func koanfKeys(t reflect.Type, prefix string) []string { + for t.Kind() == reflect.Ptr { + t = t.Elem() + } + if t.Kind() != reflect.Struct { + return nil + } + var keys []string + for f := range t.Fields() { + tag, ok := f.Tag.Lookup("koanf") + if !ok || tag == "" || tag == "-" { + continue + } + key := tag + if prefix != "" { + key = prefix + "." + tag + } + ft := f.Type + for ft.Kind() == reflect.Ptr { + ft = ft.Elem() + } + if ft.Kind() == reflect.Struct { + keys = append(keys, koanfKeys(ft, key)...) + continue + } + keys = append(keys, key) + } + return keys +} diff --git a/Server/cmd/gendocs/main_test.go b/Server/cmd/gendocs/main_test.go new file mode 100644 index 00000000..c12e62ba --- /dev/null +++ b/Server/cmd/gendocs/main_test.go @@ -0,0 +1,193 @@ +package main + +import ( + "slices" + "strings" + "testing" +) + +// TestWriteTablePadsLikePrettier pins the padding rule the whole tool rests +// on: cells and the separator are padded to the widest cell in the column, +// header included, with Prettier's three-dash minimum. Get this wrong and +// `prettier --check` reformats what `git diff --exit-code` then reports as +// drift, forever. +func TestWriteTablePadsLikePrettier(t *testing.T) { + tests := []struct { + name string + header []string + rows [][]string + want string + }{ + { + name: "column widens to its longest cell", + header: []string{"Method", "Path"}, + rows: [][]string{{"GET", "/a"}, {"DELETE", "/longer"}}, + want: strings.Join([]string{ + "| Method | Path |", + "| ------ | ------- |", + "| GET | /a |", + "| DELETE | /longer |", + }, "\n") + "\n", + }, + { + name: "separator never goes below three dashes", + header: []string{"K"}, + rows: [][]string{{"v"}}, + want: "| K |\n| --- |\n| v |\n", + }, + { + name: "a pipe in a cell is escaped, and the escape counts as width", + header: []string{"Type"}, + rows: [][]string{{"a|b"}}, + want: "| Type |\n| ---- |\n| a\\|b |\n", + }, + { + name: "a short row is padded out to the header's column count", + header: []string{"A", "B"}, + rows: [][]string{{"x"}}, + want: "| A | B |\n| --- | --- |\n| x | |\n", + }, + { + name: "width is counted in runes, not bytes", + header: []string{"Note"}, + rows: [][]string{{"—"}}, + want: "| Note |\n| ---- |\n| — |\n", + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + var b strings.Builder + writeTable(&b, tt.header, tt.rows) + if b.String() != tt.want { + t.Errorf("writeTable\n got:\n%s\nwant:\n%s", b.String(), tt.want) + } + }) + } +} + +// TestSpliceBlock pins the marker contract: only the text between a matched +// pair is replaced, and an unmarked document is an error rather than an +// append — where a generated block lives is an editorial decision. +func TestSpliceBlock(t *testing.T) { + const doc = "before\n\n\n\nstale\n\n\n\nafter\n" + tests := []struct { + name string + doc string + body string + want string + wantErr string + }{ + { + name: "replaces the body, keeps the markers and their neighbours", + doc: doc, + body: "fresh", + want: "before\n\n\n\nfresh\n\n\n\nafter\n", + }, + { + name: "surplus newlines around the body are trimmed to one blank line", + doc: doc, + body: "\n\nfresh\n\n\n", + want: "before\n\n\n\nfresh\n\n\n\nafter\n", + }, + { + name: "no markers at all", + doc: "before\nafter\n", + body: "fresh", + wantErr: "not found", + }, + { + name: "start marker without an end marker", + doc: "\nstale\n", + body: "fresh", + wantErr: "not found", + }, + { + name: "end marker before the start marker", + doc: "\nstale\n\n", + body: "fresh", + wantErr: "appears before", + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := spliceBlock(tt.doc, "routes", tt.body) + switch { + case tt.wantErr != "": + if err == nil || !strings.Contains(err.Error(), tt.wantErr) { + t.Fatalf("spliceBlock error = %v, want one containing %q", err, tt.wantErr) + } + case err != nil: + t.Fatalf("spliceBlock: %v", err) + case got != tt.want: + t.Errorf("spliceBlock\n got: %q\nwant: %q", got, tt.want) + } + }) + } +} + +// TestCmpRouteOrdersByBarePath pins the sort to the path before its code span +// goes on. Sorting the rendered cell puts `/x` after `/x/y`, because a +// backtick outranks a slash — a stable but nonsensical order. +func TestCmpRouteOrdersByBarePath(t *testing.T) { + rows := [][]string{ + {"POST", "/a/{id}/restore"}, + {"GET", "/a"}, + {"DELETE", "/a/{id}"}, + {"DELETE", "/a"}, + } + slices.SortFunc(rows, cmpRoute) + want := [][]string{ + {"DELETE", "/a"}, + {"GET", "/a"}, + {"DELETE", "/a/{id}"}, + {"POST", "/a/{id}/restore"}, + } + if !slices.EqualFunc(rows, want, slices.Equal) { + t.Errorf("sorted = %v, want %v", rows, want) + } +} + +// TestDocSectionsScansOnlyTheHandWrittenReference keeps the config index from +// being its own evidence: a key is documented when a hand-written subsection +// of "## Config Key Reference" names it, and the generated index sits under +// its own "## " heading, outside the scan. +func TestDocSectionsScansOnlyTheHandWrittenReference(t *testing.T) { + doc := strings.Join([]string{ + "# Title", + "", + "### First-run wizard", + "", + "writes `server.port` for you", + "", + "## Config Key Reference", + "", + "### Server (`server`)", + "", + "| `server.port` | int | `8443` | the port |", + "| `server.name` | string | `\"x\"` | the name |", + "", + "### TLS (`tls`)", + "", + "| `tls.mode` | string | see `server.port` above |", + "", + "## Key index (generated)", + "", + "| `server.orphan` | Server (`server`) |", + "", + }, "\n") + + got := docSections(doc) + want := map[string]string{ + "server.port": "Server (`server`)", + "server.name": "Server (`server`)", + "tls.mode": "TLS (`tls`)", + } + if len(got) != len(want) { + t.Fatalf("docSections = %v, want %v", got, want) + } + for k, v := range want { + if got[k] != v { + t.Errorf("docSections[%q] = %q, want %q", k, got[k], v) + } + } +} diff --git a/Server/invariants/db_import_boundary.go b/Server/invariants/db_import_boundary.go index 60c38a50..8095b17d 100644 --- a/Server/invariants/db_import_boundary.go +++ b/Server/invariants/db_import_boundary.go @@ -73,6 +73,7 @@ var DBImportAllow = map[string]DBImportEntry{ "main.go": {"boundary", "", "process composition root; B3-3 moves it to internal/app"}, "token_cli.go": {"move", "auth", "API-token CLI duplicates admin/handlers_tokens.go"}, "cmd/seed/main.go": {"boundary", "", "developer seeding tool owns its handle"}, + "cmd/gendocs/main.go": {"boundary", "", "docs generator migrates its own in-memory catalog"}, "plugin/pluginstore.go": {"adapter", "", "PluginRow type only; the store is injected"}, // ── ws ──────────────────────────────────────────────────────────────── "ws/client.go": {"adapter", "", "db.User type on the connection"}, diff --git a/docs/api.md b/docs/api.md index bfd1d788..2d41dcd9 100644 --- a/docs/api.md +++ b/docs/api.md @@ -32,6 +32,140 @@ Note: chi's `middleware.RealIP` is deliberately **not** used -- client IPs are r --- +## Route index (generated) + + + +Generated from the mounted router by `cd Server && go run -tags otel,wazero ./cmd/gendocs` — do not edit by hand; `make docs-verify` fails when it drifts. 121 routes, from the `otel,wazero` build with every optional family enabled (uploads, voice, the GIF proxy, and telemetry with the Prometheus exporter, which is what mounts `/metrics`). + +| Method | Path | +| ------- | -------------------------------------------------------------------- | +| GET | `/admin/` | +| CONNECT | `/admin/*` | +| DELETE | `/admin/*` | +| GET | `/admin/*` | +| HEAD | `/admin/*` | +| OPTIONS | `/admin/*` | +| PATCH | `/admin/*` | +| POST | `/admin/*` | +| PUT | `/admin/*` | +| QUERY | `/admin/*` | +| TRACE | `/admin/*` | +| GET | `/admin/api/audit-log` | +| POST | `/admin/api/backup` | +| GET | `/admin/api/backups` | +| DELETE | `/admin/api/backups/{name}` | +| POST | `/admin/api/backups/{name}/restore` | +| GET | `/admin/api/channels` | +| POST | `/admin/api/channels` | +| DELETE | `/admin/api/channels/{id}` | +| PATCH | `/admin/api/channels/{id}` | +| GET | `/admin/api/channels/{id}/permissions` | +| DELETE | `/admin/api/channels/{id}/permissions/{roleId}` | +| PUT | `/admin/api/channels/{id}/permissions/{roleId}` | +| DELETE | `/admin/api/channels/{id}/user-permissions/{userId}` | +| PUT | `/admin/api/channels/{id}/user-permissions/{userId}` | +| POST | `/admin/api/logs/ticket` | +| GET | `/admin/api/me` | +| GET | `/admin/api/roles` | +| POST | `/admin/api/roles` | +| PATCH | `/admin/api/roles/reorder` | +| DELETE | `/admin/api/roles/{id}` | +| PATCH | `/admin/api/roles/{id}` | +| GET | `/admin/api/settings` | +| PATCH | `/admin/api/settings` | +| POST | `/admin/api/setup` | +| GET | `/admin/api/setup/status` | +| GET | `/admin/api/stats` | +| GET | `/admin/api/tokens` | +| POST | `/admin/api/tokens` | +| DELETE | `/admin/api/tokens/{id}` | +| GET | `/admin/api/updates` | +| POST | `/admin/api/updates/apply` | +| GET | `/admin/api/users` | +| PATCH | `/admin/api/users/{id}` | +| DELETE | `/admin/api/users/{id}/sessions` | +| GET | `/api/v1/admin/plugins/` | +| POST | `/api/v1/admin/plugins/install` | +| DELETE | `/api/v1/admin/plugins/{id}` | +| POST | `/api/v1/admin/plugins/{id}/disable` | +| POST | `/api/v1/admin/plugins/{id}/enable` | +| DELETE | `/api/v1/auth/account` | +| POST | `/api/v1/auth/login` | +| POST | `/api/v1/auth/logout` | +| GET | `/api/v1/auth/me` | +| POST | `/api/v1/auth/register` | +| POST | `/api/v1/auth/verify-totp` | +| GET | `/api/v1/blocks/` | +| DELETE | `/api/v1/blocks/{userId}` | +| PUT | `/api/v1/blocks/{userId}` | +| GET | `/api/v1/channels/` | +| GET | `/api/v1/channels/{id}/messages` | +| GET | `/api/v1/channels/{id}/messages/around/{messageId}` | +| POST | `/api/v1/channels/{id}/messages/purge` | +| GET | `/api/v1/channels/{id}/messages/{messageId}/reactions/{emoji}/users` | +| GET | `/api/v1/channels/{id}/pins` | +| DELETE | `/api/v1/channels/{id}/pins/{messageId}` | +| POST | `/api/v1/channels/{id}/pins/{messageId}` | +| GET | `/api/v1/client-update/{target}/{current_version}` | +| GET | `/api/v1/diagnostics/connectivity` | +| GET | `/api/v1/dms/` | +| POST | `/api/v1/dms/` | +| POST | `/api/v1/dms/group` | +| DELETE | `/api/v1/dms/{channelId}` | +| PATCH | `/api/v1/dms/{channelId}` | +| GET | `/api/v1/emoji/` | +| POST | `/api/v1/emoji/` | +| DELETE | `/api/v1/emoji/{id}` | +| GET | `/api/v1/emoji/{id}/image` | +| GET | `/api/v1/files/{id}` | +| GET | `/api/v1/gif/search` | +| GET | `/api/v1/gif/trending` | +| GET | `/api/v1/health` | +| GET | `/api/v1/info` | +| GET | `/api/v1/invites/` | +| POST | `/api/v1/invites/` | +| DELETE | `/api/v1/invites/{code}` | +| GET | `/api/v1/livekit/health` | +| POST | `/api/v1/livekit/webhook` | +| GET | `/api/v1/metrics` | +| GET | `/api/v1/search` | +| POST | `/api/v1/uploads` | +| PATCH | `/api/v1/users/me/` | +| POST | `/api/v1/users/me/avatar` | +| PUT | `/api/v1/users/me/password` | +| GET | `/api/v1/users/me/sessions` | +| DELETE | `/api/v1/users/me/sessions/{id}` | +| DELETE | `/api/v1/users/me/totp` | +| POST | `/api/v1/users/me/totp/confirm` | +| POST | `/api/v1/users/me/totp/enable` | +| GET | `/api/v1/ws` | +| GET | `/health` | +| CONNECT | `/livekit/*` | +| DELETE | `/livekit/*` | +| GET | `/livekit/*` | +| HEAD | `/livekit/*` | +| OPTIONS | `/livekit/*` | +| PATCH | `/livekit/*` | +| POST | `/livekit/*` | +| PUT | `/livekit/*` | +| QUERY | `/livekit/*` | +| TRACE | `/livekit/*` | +| CONNECT | `/metrics/*` | +| DELETE | `/metrics/*` | +| GET | `/metrics/*` | +| HEAD | `/metrics/*` | +| OPTIONS | `/metrics/*` | +| PATCH | `/metrics/*` | +| POST | `/metrics/*` | +| PUT | `/metrics/*` | +| QUERY | `/metrics/*` | +| TRACE | `/metrics/*` | + + + +--- + ## Standard Error Response Error responses use this JSON envelope (one exception: the plugin admin diff --git a/docs/architecture/server-boundaries.md b/docs/architecture/server-boundaries.md index ecb9ef4f..accd7090 100644 --- a/docs/architecture/server-boundaries.md +++ b/docs/architecture/server-boundaries.md @@ -88,6 +88,7 @@ which is a row worth reading, and none exists today. | `api/upload_handler.go` | `AttachmentAccess×2` `DB×5` `Role` `User×3` | — | `CreateAttachment` `GetAttachmentWithChannel` `IsAvatarFileURL` `IsDMParticipant` `QueryRowContext` | calls | move | upload | attachment access + a raw QueryRowContext | | `auth/helpers.go` | `User` | — | — | type-only | adapter | — | db.User type in a helper signature | | `auth/resolve.go` | `APIToken` `Role×2` `Session×2` `User×2` | — | — | type-only | adapter | — | Session/APIToken/Role/User types; resolution is injected | +| `cmd/gendocs/main.go` | `DB×3` | `Migrate()` `Open()` | `Close×2` `QueryContext×2` | calls | boundary | — | docs generator migrates its own in-memory catalog | | `cmd/seed/main.go` | `DB×6` | `Migrate()` `Open()` | `Close` `CreateChannel` `CreateMessage×2` `CreateUser` `GetOrCreateDMChannel` `GetUserByUsername` `ListChannels` `QueryRowContext` | calls | boundary | — | developer seeding tool owns its handle | | `main.go` | `AuditWriter×2` `DB×10` | `ErrNotFound` `Migrate()` `NewAuditWriter()` `OpenWithMaxReaders()` | `ClearAllVoiceStates` `Close` `DeleteExpiredSessions` `DeleteOrphanedAttachments` `GetMaxEventSeq` `GetSetting` `ResetAllUserStatuses` `SetAuditWriter` `SetSetting` | calls | boundary | — | process composition root; B3-3 moves it to internal/app | | `plugin/pluginstore.go` | `PluginRow×3` | — | — | type-only | adapter | — | PluginRow type only; the store is injected | @@ -110,8 +111,8 @@ which is a row worth reading, and none exists today. | `ws/voice_join.go` | `Channel×3` `ChannelOverride` `VoiceState×5` | `ErrChannelFull` | `GetChannel×2` `GetChannelOverridesFor` `GetChannelVoiceStates` `GetRoleForUser` `GetVoiceState×6` `JoinVoiceChannel` `JoinVoiceChannelIfCapacity` `LeaveVoiceChannelIfMatch` `SetVoiceServerDeafen` `SetVoiceServerMute` | calls | move | voice | voice state reads and writes | | `ws/voice_moderation.go` | `Role` `VoiceState×3` | `WriteAudit()` | `CountChannelVoiceUsers` `GetChannel×2` `GetRoleForUser` `GetVoiceState×2` `SetVoiceServerDeafen×2` `SetVoiceServerMute×2` | calls | move | voice | mute/deafen/move persist voice state | -49 files import `db` outside `db/` and `service/` (. 2, admin 16, api 10, auth 2, cmd/seed 1, plugin 1, ws 17); 14 are type-only; 0 unlisted. -Dispositions: adapter 17, boundary 6, move 26. Move targets: auth 7, channel 6, connection 1, role 1, settings-ops 4, upload 2, user 2, voice 3. +50 files import `db` outside `db/` and `service/` (. 2, admin 16, api 10, auth 2, cmd/gendocs 1, cmd/seed 1, plugin 1, ws 17); 14 are type-only; 0 unlisted. +Dispositions: adapter 17, boundary 7, move 26. Move targets: auth 7, channel 6, connection 1, role 1, settings-ops 4, upload 2, user 2, voice 3. diff --git a/docs/plans/b3-server-architecture-guardrails-2026-08-29.md b/docs/plans/b3-server-architecture-guardrails-2026-08-29.md index 7636fc77..59cbd3e4 100644 --- a/docs/plans/b3-server-architecture-guardrails-2026-08-29.md +++ b/docs/plans/b3-server-architecture-guardrails-2026-08-29.md @@ -838,6 +838,64 @@ db`). No control file is committed. (OC-0031/OC-0033/OC-0311), not through `LiveKitSession`'s join generations, which need a real LiveKit room. +#### Evidence — item 9 (machine-readable contract drift) + +- Branch `feat/b3-6-contract-drift` from `dev` `75d64dd4`; commits: `f5f467ed` + feat(b3-6): contract drift — generated route, table and config-key indexes + (tool, three blocks, wiring), plus this evidence commit. +- The drift check is `make docs-verify`, which reduces to + `go run ./cmd/gendocs` then `git diff --exit-code ../docs/api.md ../docs/schema.md ../docs/server-configuration.md`, both from `Server/`. `make` is not on PATH on this machine, so the three controls below ran that pair directly; `node scripts/run.mjs --list` shows `check:server` running exactly those two steps. +- RED (a), a stale committed block — one row deleted from the route index and + staged, then the pair above: the diff puts the deleted `GET /api/v1/health` + row back and `git diff --exit-code` exits 1. +- RED (b), a route added without regenerating — a throwaway + `r.Get("/gendocs-red-proof", handleInfo(cfg))` in `api/router.go`, then the + pair: the header line goes from "111 routes." to "112 routes." and a + `GET /api/v1/gendocs-red-proof` row appears; exit 1. +- RED (c), a config key documented nowhere — the hand-written `voice.quality` + row removed from `docs/server-configuration.md`, then `go run ./cmd/gendocs` + alone: `gendocs: config index: 1 config key(s) are documented nowhere under "## Config Key Reference" in ../docs/server-configuration.md … voice.quality`, exit status 1, and no document written. +- GREEN, all three controls reverted (`git status` clean): the same pair exits + 0; `npx prettier --check .` → "All matched files use Prettier code style!"; + `.githooks/pre-commit` run by hand over the staged change → exit 0 with the + generator's own log lines in its output. +- Numbers: **121 routes** (`chi.Walk`; the floor of 100 and the admin-route + guard are carried over from the absence contract, the latter tightened to + require a real `/admin/api/` subroute so the `/admin/*` mount catch-alls + cannot satisfy it alone), **32 tables**, **56 config keys**, **0 undocumented keys at HEAD** — the hand-written reference + already named every koanf tag, so no documentation fixes were needed. + Generated blocks: three, adding 134 + 45 + 67 lines to their documents. + Output is byte-identical across two consecutive runs. +- Verified against HEAD: the plan's pointer to `docs/deployment.md` is wrong — + that document has no configuration table; the reference table is + `docs/server-configuration.md` "## Config Key Reference", and the key index + went there. `sqlc` exposes no catalog at HEAD (`Server/sqlc.yaml` declares no + plugins and no vet rules; its schema source is `migrations/`), so the + migrated in-memory schema is the catalog: `db.Open(":memory:")` plus + `db.Migrate`, then `sqlite_master` and `pragma_table_info`. `schema_versions` + is **included** rather than excluded, along with `sqlite_sequence` and the + FTS5 shadow tables behind `messages_fts` — they are what the migrations + create, and a change to any of them is a schema change worth seeing in the + diff. The route index is generated from the **`otel,wazero` build**, not the + default one: `/metrics` mounts only when `telemetry.PrometheusHandler()` + returns non-nil (`api/router.go:431-437`), which needs `-tags otel` and + telemetry enabled at runtime, so the tool calls `telemetry.Init` the way + `main.go` does with the Prometheus exporter and every invocation passes + `-tags otel,wazero` (Makefile, `run.mjs`, the hook, the block header lines, + the `CLAUDE.md` row; `ci.yml` inherits it through `make docs-verify`). The + tool refuses to run when that handler is absent, so the default build cannot + quietly generate a short index. Nothing under `Server/api` or `Server/admin` + carries a build constraint, so `wazero` adds and removes no route; it rides + along so one build serves the repository. The `sqlite_stat*` tables are + **excluded**: `db.Migrate` runs `ANALYZE` + after applying the migrations, so they carry planner statistics rather than + schema, and `sqlite_stat4` exists only because the current + `modernc.org/sqlite` build has STAT4 — including them would fail this drift + check on an unrelated driver bump. `cmd/gendocs/main.go` imports `db` for that catalog, so it takes a + `boundary` row in `DBImportAllow` and `server-boundaries.md` was regenerated + with it (50 importers, was 49); the dbinventory block is otherwise untouched + and still has no automated drift check of its own. + ## B3-7 — Alpha-shaped test dataset Roadmap workstream 12. Beside the slice. diff --git a/docs/schema.md b/docs/schema.md index f3fd75ab..661feb08 100644 --- a/docs/schema.md +++ b/docs/schema.md @@ -84,6 +84,51 @@ CREATE TABLE IF NOT EXISTS schema_versions ( --- +## Table index (generated) + + + +Generated from the migrated schema by `cd Server && go run -tags otel,wazero ./cmd/gendocs` — do not edit by hand; `make docs-verify` fails when it drifts. 32 tables: `sqlite_sequence` and the FTS5 shadow tables behind `messages_fts` are included; the `sqlite_stat*` tables `ANALYZE` writes are not, since they hold planner statistics and which of them exists depends on the SQLite build. + +| Table | Columns | Indexes | +| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | +| `api_tokens` | `id INTEGER PK`, `user_id INTEGER NOT NULL`, `token_hash TEXT NOT NULL`, `label TEXT NOT NULL`, `created_at TEXT NOT NULL`, `last_used_at TEXT`, `expires_at TEXT`, `revoked_at TEXT` | `idx_api_tokens_user`, `sqlite_autoindex_api_tokens_1` | +| `attachments` | `id TEXT PK`, `message_id INTEGER`, `filename TEXT NOT NULL`, `stored_as TEXT NOT NULL`, `mime_type TEXT NOT NULL`, `size INTEGER NOT NULL`, `uploaded_at TEXT NOT NULL`, `width INTEGER`, `height INTEGER`, `uploader_id INTEGER` | `idx_attachments_message`, `idx_attachments_uploader`, `sqlite_autoindex_attachments_1` | +| `audit_log` | `id INTEGER PK`, `actor_id INTEGER NOT NULL`, `action TEXT NOT NULL`, `target_type TEXT NOT NULL`, `target_id INTEGER NOT NULL`, `detail TEXT NOT NULL`, `created_at TEXT NOT NULL` | `idx_audit_log_actor`, `idx_audit_timestamp` | +| `channel_overrides` | `id INTEGER PK`, `channel_id INTEGER NOT NULL`, `role_id INTEGER NOT NULL`, `allow INTEGER NOT NULL`, `deny INTEGER NOT NULL` | `idx_channel_overrides_role`, `sqlite_autoindex_channel_overrides_1` | +| `channel_user_overrides` | `channel_id INTEGER NOT NULL PK`, `user_id INTEGER NOT NULL PK`, `allow INTEGER NOT NULL`, `deny INTEGER NOT NULL` | `idx_channel_user_overrides_user`, `sqlite_autoindex_channel_user_overrides_1` | +| `channels` | `id INTEGER PK`, `name TEXT NOT NULL`, `type TEXT NOT NULL`, `category TEXT`, `topic TEXT`, `position INTEGER NOT NULL`, `slow_mode INTEGER NOT NULL`, `archived INTEGER NOT NULL`, `created_at TEXT NOT NULL`, `voice_max_users INTEGER NOT NULL`, `voice_quality TEXT`, `mixing_threshold INTEGER`, `voice_max_video INTEGER NOT NULL`, `nsfw INTEGER NOT NULL`, `is_group INTEGER NOT NULL` | `idx_channels_dm_group` | +| `dm_open_state` | `user_id INTEGER NOT NULL PK`, `channel_id INTEGER NOT NULL PK`, `opened_at TEXT NOT NULL` | `sqlite_autoindex_dm_open_state_1` | +| `dm_participants` | `channel_id INTEGER NOT NULL PK`, `user_id INTEGER NOT NULL PK` | `idx_dm_participants_user`, `sqlite_autoindex_dm_participants_1` | +| `emoji` | `id INTEGER PK`, `shortcode TEXT NOT NULL`, `filename TEXT NOT NULL`, `uploaded_by INTEGER NOT NULL`, `created_at TEXT NOT NULL`, `mime_type TEXT NOT NULL` | `sqlite_autoindex_emoji_1` | +| `events` | `seq INTEGER PK`, `event_type TEXT NOT NULL`, `payload BLOB NOT NULL`, `channel_id INTEGER NOT NULL`, `created_at TIMESTAMP NOT NULL` | `idx_events_channel_seq`, `idx_events_created_at` | +| `invites` | `id INTEGER PK`, `code TEXT NOT NULL`, `created_by INTEGER NOT NULL`, `redeemed_by INTEGER`, `max_uses INTEGER`, `use_count INTEGER NOT NULL`, `expires_at TEXT`, `created_at TEXT NOT NULL`, `revoked INTEGER NOT NULL` | `sqlite_autoindex_invites_1` | +| `login_attempts` | `id INTEGER PK`, `ip_address TEXT NOT NULL`, `username TEXT`, `success INTEGER NOT NULL`, `timestamp TEXT NOT NULL` | `idx_login_ip` | +| `message_mentions` | `message_id INTEGER NOT NULL PK`, `mentioned_user_id INTEGER NOT NULL PK` | `idx_message_mentions_user`, `sqlite_autoindex_message_mentions_1` | +| `messages` | `id INTEGER PK`, `channel_id INTEGER NOT NULL`, `user_id INTEGER NOT NULL`, `content TEXT NOT NULL`, `reply_to INTEGER`, `edited_at TEXT`, `deleted INTEGER NOT NULL`, `pinned INTEGER NOT NULL`, `timestamp TEXT NOT NULL`, `mentions_everyone INTEGER NOT NULL` | `idx_messages_channel`, `idx_messages_pinned`, `idx_messages_user` | +| `messages_fts` | `content` | — | +| `messages_fts_config` | `k NOT NULL PK`, `v` | — | +| `messages_fts_data` | `id INTEGER PK`, `block BLOB` | — | +| `messages_fts_docsize` | `id INTEGER PK`, `sz BLOB` | — | +| `messages_fts_idx` | `segid NOT NULL PK`, `term NOT NULL PK`, `pgno` | — | +| `plugin_kv` | `plugin_id INTEGER NOT NULL PK`, `key TEXT NOT NULL PK`, `value BLOB NOT NULL` | `sqlite_autoindex_plugin_kv_1` | +| `plugins` | `id INTEGER PK`, `name TEXT NOT NULL`, `version TEXT NOT NULL`, `enabled INTEGER NOT NULL`, `manifest_json TEXT NOT NULL`, `installed_at TIMESTAMP NOT NULL` | `sqlite_autoindex_plugins_1` | +| `rate_lockouts` | `key TEXT PK`, `expires_at TEXT NOT NULL` | `sqlite_autoindex_rate_lockouts_1` | +| `reactions` | `id INTEGER PK`, `message_id INTEGER NOT NULL`, `user_id INTEGER NOT NULL`, `emoji TEXT NOT NULL` | `sqlite_autoindex_reactions_1` | +| `read_states` | `user_id INTEGER NOT NULL PK`, `channel_id INTEGER NOT NULL PK`, `last_message_id INTEGER NOT NULL`, `mention_count INTEGER NOT NULL` | `sqlite_autoindex_read_states_1` | +| `roles` | `id INTEGER PK`, `name TEXT NOT NULL`, `color TEXT`, `permissions INTEGER NOT NULL`, `position INTEGER NOT NULL`, `is_default INTEGER NOT NULL` | `idx_roles_name_nocase`, `sqlite_autoindex_roles_1` | +| `schema_versions` | `version TEXT PK`, `applied_at TEXT NOT NULL` | `sqlite_autoindex_schema_versions_1` | +| `sessions` | `id INTEGER PK`, `user_id INTEGER NOT NULL`, `token TEXT NOT NULL`, `device TEXT`, `ip_address TEXT`, `created_at TEXT NOT NULL`, `last_used TEXT NOT NULL`, `expires_at TEXT NOT NULL` | `idx_sessions_expires_at`, `idx_sessions_user`, `sqlite_autoindex_sessions_1` | +| `settings` | `key TEXT PK`, `value TEXT NOT NULL` | `sqlite_autoindex_settings_1` | +| `sqlite_sequence` | `name`, `seq` | — | +| `user_blocks` | `blocker_id INTEGER NOT NULL PK`, `blocked_id INTEGER NOT NULL PK`, `created_at TEXT NOT NULL` | `idx_user_blocks_blocked`, `sqlite_autoindex_user_blocks_1` | +| `users` | `id INTEGER PK`, `username TEXT NOT NULL`, `password TEXT NOT NULL`, `avatar TEXT`, `role_id INTEGER NOT NULL`, `totp_secret TEXT`, `status TEXT NOT NULL`, `created_at TEXT NOT NULL`, `last_seen TEXT`, `banned INTEGER NOT NULL`, `ban_reason TEXT`, `ban_expires TEXT`, `identity_public_key TEXT`, `display_name TEXT`, `about TEXT`, `custom_status TEXT` | `idx_users_avatar`, `sqlite_autoindex_users_1` | +| `voice_states` | `user_id INTEGER PK`, `channel_id INTEGER NOT NULL`, `muted INTEGER NOT NULL`, `deafened INTEGER NOT NULL`, `speaking INTEGER NOT NULL`, `joined_at TEXT NOT NULL`, `camera INTEGER NOT NULL`, `screenshare INTEGER NOT NULL`, `server_muted INTEGER NOT NULL`, `server_deafened INTEGER NOT NULL` | `idx_voice_states_channel` | + + + +--- + ## Tables ### roles diff --git a/docs/server-configuration.md b/docs/server-configuration.md index c4929f2c..0c148c63 100644 --- a/docs/server-configuration.md +++ b/docs/server-configuration.md @@ -178,6 +178,73 @@ ring buffer that backs the admin panel's live log view. | --------------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------ | | `logging.level` | string | `"info"` | Minimum level logged: `debug`, `info`, `warn`, `error`. Empty = `info`; an unrecognised value falls back to `info` with a startup warning. | +## Key index (generated) + + + +Generated from the `koanf` tags of `config.Config` by `cd Server && go run -tags otel,wazero ./cmd/gendocs` — do not edit by hand; `make docs-verify` fails when it drifts, and the tool exits non-zero when a key is documented nowhere above. 56 keys. + +| Key | Documented in | +| ------------------------------------------- | --------------------------------------- | +| `backup.dir` | Backups (`backup`) | +| `database.max_readers` | Database (`database`) | +| `database.path` | Database (`database`) | +| `database.type` | Database (`database`) | +| `event_persistence.batch_flush_ms` | Event Persistence (`event_persistence`) | +| `event_persistence.batch_size` | Event Persistence (`event_persistence`) | +| `event_persistence.enabled` | Event Persistence (`event_persistence`) | +| `event_persistence.pruner_interval_minutes` | Event Persistence (`event_persistence`) | +| `event_persistence.replay_cold_limit` | Event Persistence (`event_persistence`) | +| `event_persistence.replay_ring_size` | Event Persistence (`event_persistence`) | +| `event_persistence.retention_hours` | Event Persistence (`event_persistence`) | +| `gif.api_key` | GIF Picker (`gif`) | +| `github.owner` | GitHub / Updates (`github`) | +| `github.repo` | GitHub / Updates (`github`) | +| `github.token` | GitHub / Updates (`github`) | +| `logging.level` | Logging (`logging`) | +| `plugins.cpu_budget_ms` | Plugins (`plugins`) | +| `plugins.directory` | Plugins (`plugins`) | +| `plugins.enabled` | Plugins (`plugins`) | +| `plugins.http_allowlist` | Plugins (`plugins`) | +| `plugins.max_memory_mb` | Plugins (`plugins`) | +| `security.auth_rate_limit_multiplier` | Security (`security`) | +| `server.admin_allowed_cidrs` | Server (`server`) | +| `server.allowed_origins` | Server (`server`) | +| `server.data_dir` | Server (`server`) | +| `server.livekit_webhook_allowed_cidrs` | Server (`server`) | +| `server.max_ws_connections` | Server (`server`) | +| `server.metrics_allowed_cidrs` | Server (`server`) | +| `server.name` | Server (`server`) | +| `server.port` | Server (`server`) | +| `server.restart_mode` | Server (`server`) | +| `server.trusted_proxies` | Server (`server`) | +| `server.waf_crs_mode` | Server (`server`) | +| `server.waf_enabled` | Server (`server`) | +| `server.waf_paranoia_level` | Server (`server`) | +| `telemetry.enabled` | Telemetry / OpenTelemetry (`telemetry`) | +| `telemetry.exporter` | Telemetry / OpenTelemetry (`telemetry`) | +| `telemetry.otlp_endpoint` | Telemetry / OpenTelemetry (`telemetry`) | +| `telemetry.otlp_insecure` | Telemetry / OpenTelemetry (`telemetry`) | +| `telemetry.service_name` | Telemetry / OpenTelemetry (`telemetry`) | +| `tls.acme_cache_dir` | TLS (`tls`) | +| `tls.cert_file` | TLS (`tls`) | +| `tls.domain` | TLS (`tls`) | +| `tls.key_file` | TLS (`tls`) | +| `tls.mode` | TLS (`tls`) | +| `upload.max_size_mb` | Uploads (`upload`) | +| `upload.storage_dir` | Uploads (`upload`) | +| `voice.advertise_internal_ip` | Voice / LiveKit (`voice`) | +| `voice.auto_download_livekit` | Voice / LiveKit (`voice`) | +| `voice.livekit_api_key` | Voice / LiveKit (`voice`) | +| `voice.livekit_api_secret` | Voice / LiveKit (`voice`) | +| `voice.livekit_binary` | Voice / LiveKit (`voice`) | +| `voice.livekit_url` | Voice / LiveKit (`voice`) | +| `voice.livekit_version` | Voice / LiveKit (`voice`) | +| `voice.node_ip` | Voice / LiveKit (`voice`) | +| `voice.quality` | Voice / LiveKit (`voice`) | + + + ## Environment Variable Overrides Every config key can be overridden via environment variables using the prefix `OWNCORD_`. diff --git a/scripts/run.mjs b/scripts/run.mjs index ec879565..98b7303c 100644 --- a/scripts/run.mjs +++ b/scripts/run.mjs @@ -66,6 +66,25 @@ const PROTOCOL_VERIFY = [ "Server", ), ]; +// The route, table and config-key indexes in docs/. Same shape again: the +// generator rewrites the marked blocks, git reports any drift. cmd/gendocs +// also exits non-zero on its own when a config key is documented nowhere, or +// when it was built without -tags otel,wazero -- the superset build the route +// index is generated from, since /metrics mounts only under otel. +const DOCS_VERIFY = [ + step("go", ["run", "-tags", "otel,wazero", "./cmd/gendocs"], "Server"), + step( + "git", + [ + "diff", + "--exit-code", + "../docs/api.md", + "../docs/schema.md", + "../docs/server-configuration.md", + ], + "Server", + ), +]; const SQLC_VERIFY = [ optional( "sqlc", @@ -94,6 +113,7 @@ const CHECK_SERVER = [ ), ...PROTOCOL_VERIFY, ...SQLC_VERIFY, + ...DOCS_VERIFY, ]; const CHECK_CLIENT = [ @@ -176,6 +196,8 @@ const TASKS = { "Server", "sqlc not on PATH — install the version in Server/sqlc.version", ), + // After sqlc: gendocs compiles the api package, which imports db/dbgen. + step("go", ["run", "-tags", "otel,wazero", "./cmd/gendocs"], "Server"), ], format: [ step("npx", ["prettier", "--write", "."], "."),