Files
Stirling-PDF/scripts/lint/comment-lint-hook.mjs
ConnorYoh 4ab2505a6c Comment-quality standard, and the gate that enforces it (#7663)
## The problem

AI PRs write comments that restate the line below them, mark sections
with box drawing, and narrate the diff. Nothing in the repo said not to,
and nothing checked. `AGENTS.md` had one line about comments and it was
buried in the Python section.

Banners and `Step N:` narration have zero occurrences in the 15 months
before Aug 2025, so this is new.

## The fix

A written standard, plus a linter that enforces the mechanical part of
it on added lines only.

-
[devGuide/CODE_COMMENTS.md](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/devGuide/CODE_COMMENTS.md)
holds the reasoning and worked examples; a section in
[AGENTS.md](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/AGENTS.md)
holds the operative rules, kept short so they stay in an agent's
context. The two are split by kind rather than duplicated, because the
same prose in two places drifts.
- Rules in
[comment-rules.mjs](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs),
shared by both engines.
- Two engines. `.ts` / `.tsx` / `.mjs` go to an [oxlint JS
plugin](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-lint-oxlint-plugin.mjs)
so comments come from the parser rather than a line scan; `.java` /
`.py` go to a [line
scanner](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-lint.mjs).
Neither reads the other's files, so they cannot disagree about one file.
- Between them they read every comment form the repo writes: `//` and
`/* */`, Javadoc and JSDoc, JSX comments, `#`, and Python docstrings.
- Runs in `task pre-commit`, so the git hook and the `pre_commit.yml` CI
job both get it, and as a Claude Code [`Stop`
hook](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-lint-hook.mjs)
so an agent fixes the comment inside the turn that wrote it.

## The rules

The part worth arguing about. **Every rule blocks.** A rule that only
warns is a rule nobody acts on, so a finding you believe is wrong is a
bug in the rule: narrow it, or mark the line and say why.

| | Fires on |
| --- | --- |
|
[CMT001](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L71)
| Every word in the comment already appears in the code below it. Max 6
words, skipped for prose punctuation and for a bare Arrange/Act/Assert
marker |
|
[CMT002](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L92)
| 4+ rule or box-drawing characters, or a bare section label from [a
fixed
list](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L84)
(`Types`, `Helpers`, `State`, `Handlers`, ...) |
|
[CMT003](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L110)
| `Step N:` with a separator, or `Then,` / `Next,` / `Finally,`.
Suppressed in test files |
|
[CMT004](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L129)
| A comment about the code's own past: `this used to`, `renamed from`,
`was previously called`. Suppressed in test files |
|
[CMT005](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L154)
| 3+ consecutive comment lines where 2/3 [parse as
code](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L143)
|
|
[CMT006](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L31)
| A run of implementation comment over 12 lines, outside the first 5
lines of a file. Doc blocks are exempt, because the standard asks for
thorough contracts |
|
[CMT007](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L180)
| A parameter or return description that adds no word its name lacks.
Reads Javadoc/JSDoc `@param`, Sphinx `:param name:` and Google `name:
description` |
|
[CMT008](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L239)
| An allow directive naming a rule that does not exist, or one that
silenced nothing |
|
[CMT009](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L219)
| A `TODO` / `FIXME` / `HACK` naming no issue or link. An owner is not
accepted: a username goes stale, an issue outlives it |

Each rule carries the readings it deliberately excludes, next to the
rule. Those exclusions came from running the rules over this repo, not
from taste: `CMT004` does not match a bare "no longer needed" because
that is as often about runtime lifecycle as about history, and `CMT003`
needs a separator after the number so a wrapped line beginning "step 2
unmounts + remounts the panel" reads as the prose it is.

A comment sharing a line with code is judged by the rules that do not
depend on the code below it, so a trailing `// TODO fix this` or `/*
this used to run before the flush */` still reports, while `50L * 1024 *
1024 // 50 MB` does not. `CMT001` would have been wrong about six in
seven trailing comments here, so it stays out of them.

If a finding is wrong, `// comment-lint-allow: CMT002` on the line
above. Rule-specific, [no blanket
disable](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L229).
A directive naming a rule that does not exist, or silencing nothing, is
itself a `CMT008` failure, so a typo cannot quietly disable a rule and a
stale one gets deleted rather than accumulating.

No native linter covers `CMT007`. `eslint-plugin-jsdoc`'s
`require-param-description`, Checkstyle's `NonEmptyAtclauseDescription`
and ruff's D-rules all check that a description exists, not whether it
says anything.

## Scoping

Added comment **text** only, not lines git calls new. Reindenting a file
or moving a block makes git mark untouched comments as added; findings
are matched against the comment text at the base, so only genuinely new
content reports.

The whole file is read and every comment in it evaluated. Only the
*reporting* is filtered, so a rule still sees the code a comment
introduces, the full run it belongs to, and the base version of the
file.

Existing tree is untouched. `task pre-commit:comment-lint:all` reports
it and always exits 0:

| | java | ts/js | py |
| --- | --- | --- | --- |
| findings | 1,218 | 741 | 204 |

2,163 across 542 files, mostly `CMT002` banners (1,482) and `CMT001`
restatements (456). Clearing it is separate work, by directory.

Not in this PR: an advisory LLM review layer for the things no pattern
can judge.

## Verification

Run against
[#7494](https://github.com/Stirling-Tools/Stirling-PDF/pull/7494) as CI
would, in a throwaway worktree: **two findings on a 78 file, +4,512 line
change, both genuine banners, in 952ms**. A whole-file scan of those
same files gives 11; the other 9 were withheld because that PR's author
did not write them, and they are the `@param teamId the team ID` shape
this standard exists to stop.

Both scanners blank string and character literals before looking for
comment markers, because a partial lex desynchronises everything after
it: one apostrophe in a Java comment, or one Python template whose
closing quotes start a line, is enough to read dozens of lines of code
as a single comment. Two fixtures carry canaries that stop being
reported if either engine ever desynchronises again.

The [fixture
corpus](https://github.com/Stirling-Tools/Stirling-PDF/tree/claude/ai-pr-comment-quality-dd970e/scripts/lint/fixtures)
pins all 9 rules against both engines, and `--selftest` fails if the two
disagree about the same file.

## Two things reviewers should know

**The oxlint JS plugin API is alpha.** oxlint itself is stable and
already this repo's frontend linter; the plugin API is the new
dependency. Its documented failure mode
([oxc#25203](https://github.com/oxc-project/oxc/issues/25203)) is being
skipped silently while oxlint still reports success. That affects the
standalone release binary rather than the npm package this invokes, but
the class of failure reads exactly like clean code, so the run asserts
`number_of_rules >= 1` from oxlint's own report and a broken engine
exits 2 rather than passing. If the API ever breaks, the fallback is
folding these rules into the line scanner, which already implements all
nine for Java and Python.

**`.claude/settings.json` is now committed**, carrying the hook and
nothing else: 19 lines, no `permissions`, nothing machine-specific. That
partly reverts `c35546a212` ("Ignore claude dir"), which existed because
this file had twice been committed by accident with a personal
`permissions` allowlist, once with absolute machine paths. Personal
config still belongs in `.claude/settings.local.json`, which the new
pattern keeps ignored, and hook entries merge across the two so nobody's
own hooks are lost.

If you already hand-wrote a `.claude/settings.json`, copy it somewhere
first: that path used to be git-ignored, and git overwrites an ignored
file without warning when a commit starts tracking it. Across 19 local
checkouts here, 13 have `settings.local.json` and none has a
hand-written `settings.json`.

To turn the hook off, `{ "env": { "COMMENT_LINT_HOOK": "0" } }` in local
settings. Claude Code can only disable all hooks at once, hence the
switch. The commit-time gate still applies.

## How to test

```bash
task pre-commit:comment-lint:ci
```

The fixture corpus, then the diff. The corpus checks the rules
themselves rather than the code under review, so it runs on CI and
before a rule change, not on every local commit.

```bash
task comment-lint:branch
```

`clean (34 files in scope)`. `task comment-lint` is the same thing
scoped to uncommitted work, which is what the git hook and CI run.

To watch it bite, add `// Is banner` above `export function isBanner` in
`scripts/lint/comment-rules.mjs` and run `task comment-lint`: one
`CMT001`, exit 1. The gate covers its own source, which is why these
scripts have no section dividers.

```bash
task pre-commit:comment-lint:all
```

The standing backlog, report-only.

Verified on the pinned oxlint 1.77.0, not only the 1.79 the plugin was
prototyped against.
2026-08-28 10:56:50 +00:00

120 lines
4.9 KiB
JavaScript

#!/usr/bin/env node
// Claude Code Stop hook: check the comments this turn wrote, before it ends.
//
// Wired up by .claude/settings.json. Exit 2 stops Claude from finishing and shows
// stderr to it, so the comment is fixed inside the same turn and never reaches a
// diff, a CI run, or a reviewer.
//
// Stop rather than PostToolUse, measured over 605 real turns: a run costs the same
// whether it looks at one file or twenty-five, because node startup and one git
// diff dominate and the TS engine is spawned once for the batch. Per write it was
// 40 minutes of hook latency across those turns, and up to 35 seconds inside a
// single heavy one; per turn it is under a second, flat. Half of all writes were
// to a file already written that turn, so most of that work was repeated.
//
// To turn it off, set COMMENT_LINT_HOOK=0. Claude Code has no way to disable one
// hook (only disableAllHooks, which turns off everyone's), so the opt-out lives
// here instead. Per developer, in .claude/settings.local.json:
//
// { "env": { "COMMENT_LINT_HOOK": "0" } }
//
// The commit-time gate still applies either way, so opting out costs you the
// early warning, not the check.
import { execFileSync } from "node:child_process";
import { existsSync, readFileSync } from "node:fs";
import { dirname, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const HERE = dirname(fileURLToPath(import.meta.url));
const REPO = resolve(HERE, "..", "..");
// Invoked through Task so the taskfile stays the one place that defines how the
// linter is called.
const TASK_NAME = "pre-commit:comment-lint:hook";
const OFF = new Set(["0", "off", "false", "no"]);
// The linter's own exit codes: 1 when it found something, 2 when its engine could
// not run. The hook's codes mean different things, so they are mapped explicitly.
const FOUND = 1;
const ENGINE_BROKEN = 2;
if (OFF.has((process.env.COMMENT_LINT_HOOK ?? "").toLowerCase())) process.exit(0);
const payload = readStdin();
// Blocking the stop puts Claude back to work, which ends in another stop and
// another chance to block. Claude Code sets this flag once a Stop hook has
// already blocked in this turn, so a rule the agent cannot satisfy costs one
// extra attempt rather than looping. The commit gate still catches whatever
// survives.
if (payload?.stop_hook_active) process.exit(0);
const result = run();
if (result.status === 0) process.exit(0);
if (result.status === ENGINE_BROKEN) {
process.stderr.write("comment-lint could not run, so comments in this turn were not checked.\n");
process.exit(1);
}
// Anything else is Task itself failing, which means the check did not happen.
if (result.status !== FOUND) {
process.stderr.write(`comment-lint did not run (task exit ${result.status}), so comments in this turn were not checked.\n`);
process.exit(1);
}
// The linter's own report already names the file, line, rule and the standard, so
// it is passed through rather than rewritten.
process.stderr.write(`${result.output.trim()}\n\nFix these before finishing.\n`);
process.exit(2);
function readStdin() {
try {
return JSON.parse(readFileSync(0, "utf8"));
} catch {
return null;
}
}
// `task` on PATH is a shell wrapper that boots Node to launch the Go binary the
// npm package already ships, which costs about 400ms. Prefer the binary; fall
// back to the wrapper when the layout is not one of the ones probed, or when Task
// came from somewhere else entirely.
function taskCommand() {
const nodeDir = dirname(process.execPath);
const exe = process.platform === "win32" ? "task.exe" : "task";
const candidates = [
resolve(nodeDir, "node_modules/@go-task/cli/bin", exe),
resolve(nodeDir, "../lib/node_modules/@go-task/cli/bin", exe),
resolve(REPO, "node_modules/@go-task/cli/bin", exe),
];
for (const candidate of candidates) {
if (existsSync(candidate)) return { command: candidate, shell: false };
}
return { command: process.platform === "win32" ? "task.cmd" : "task", shell: process.platform === "win32" };
}
function run() {
const { command, shell } = taskCommand();
try {
// --output=interleaved because the root Taskfile sets `output: prefixed`,
// which would put the task name in front of every reported finding.
// --exit-code because Task otherwise reports its own 201 for any failed task,
// which hides whether the linter found something or could not run.
// Task's stderr is captured rather than inherited: it announces its own
// "Failed to run task" for any non-zero command, which would reach Claude
// alongside the findings and read as a tooling error.
const output = execFileSync(command, [TASK_NAME, "--silent", "--output=interleaved", "--exit-code"], {
cwd: REPO,
encoding: "utf8",
maxBuffer: 1 << 26,
shell,
stdio: ["ignore", "pipe", "pipe"],
});
return { status: 0, output };
} catch (error) {
return { status: error.status ?? -1, output: error.stdout ?? "" };
}
}