Files
ConnorYoh 4ab2505a6c Comment-quality standard, and the gate that enforces it (#7663)
## The problem

AI PRs write comments that restate the line below them, mark sections
with box drawing, and narrate the diff. Nothing in the repo said not to,
and nothing checked. `AGENTS.md` had one line about comments and it was
buried in the Python section.

Banners and `Step N:` narration have zero occurrences in the 15 months
before Aug 2025, so this is new.

## The fix

A written standard, plus a linter that enforces the mechanical part of
it on added lines only.

-
[devGuide/CODE_COMMENTS.md](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/devGuide/CODE_COMMENTS.md)
holds the reasoning and worked examples; a section in
[AGENTS.md](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/AGENTS.md)
holds the operative rules, kept short so they stay in an agent's
context. The two are split by kind rather than duplicated, because the
same prose in two places drifts.
- Rules in
[comment-rules.mjs](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs),
shared by both engines.
- Two engines. `.ts` / `.tsx` / `.mjs` go to an [oxlint JS
plugin](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-lint-oxlint-plugin.mjs)
so comments come from the parser rather than a line scan; `.java` /
`.py` go to a [line
scanner](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-lint.mjs).
Neither reads the other's files, so they cannot disagree about one file.
- Between them they read every comment form the repo writes: `//` and
`/* */`, Javadoc and JSDoc, JSX comments, `#`, and Python docstrings.
- Runs in `task pre-commit`, so the git hook and the `pre_commit.yml` CI
job both get it, and as a Claude Code [`Stop`
hook](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-lint-hook.mjs)
so an agent fixes the comment inside the turn that wrote it.

## The rules

The part worth arguing about. **Every rule blocks.** A rule that only
warns is a rule nobody acts on, so a finding you believe is wrong is a
bug in the rule: narrow it, or mark the line and say why.

| | Fires on |
| --- | --- |
|
[CMT001](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L71)
| Every word in the comment already appears in the code below it. Max 6
words, skipped for prose punctuation and for a bare Arrange/Act/Assert
marker |
|
[CMT002](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L92)
| 4+ rule or box-drawing characters, or a bare section label from [a
fixed
list](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L84)
(`Types`, `Helpers`, `State`, `Handlers`, ...) |
|
[CMT003](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L110)
| `Step N:` with a separator, or `Then,` / `Next,` / `Finally,`.
Suppressed in test files |
|
[CMT004](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L129)
| A comment about the code's own past: `this used to`, `renamed from`,
`was previously called`. Suppressed in test files |
|
[CMT005](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L154)
| 3+ consecutive comment lines where 2/3 [parse as
code](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L143)
|
|
[CMT006](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L31)
| A run of implementation comment over 12 lines, outside the first 5
lines of a file. Doc blocks are exempt, because the standard asks for
thorough contracts |
|
[CMT007](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L180)
| A parameter or return description that adds no word its name lacks.
Reads Javadoc/JSDoc `@param`, Sphinx `:param name:` and Google `name:
description` |
|
[CMT008](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L239)
| An allow directive naming a rule that does not exist, or one that
silenced nothing |
|
[CMT009](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L219)
| A `TODO` / `FIXME` / `HACK` naming no issue or link. An owner is not
accepted: a username goes stale, an issue outlives it |

Each rule carries the readings it deliberately excludes, next to the
rule. Those exclusions came from running the rules over this repo, not
from taste: `CMT004` does not match a bare "no longer needed" because
that is as often about runtime lifecycle as about history, and `CMT003`
needs a separator after the number so a wrapped line beginning "step 2
unmounts + remounts the panel" reads as the prose it is.

A comment sharing a line with code is judged by the rules that do not
depend on the code below it, so a trailing `// TODO fix this` or `/*
this used to run before the flush */` still reports, while `50L * 1024 *
1024 // 50 MB` does not. `CMT001` would have been wrong about six in
seven trailing comments here, so it stays out of them.

If a finding is wrong, `// comment-lint-allow: CMT002` on the line
above. Rule-specific, [no blanket
disable](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L229).
A directive naming a rule that does not exist, or silencing nothing, is
itself a `CMT008` failure, so a typo cannot quietly disable a rule and a
stale one gets deleted rather than accumulating.

No native linter covers `CMT007`. `eslint-plugin-jsdoc`'s
`require-param-description`, Checkstyle's `NonEmptyAtclauseDescription`
and ruff's D-rules all check that a description exists, not whether it
says anything.

## Scoping

Added comment **text** only, not lines git calls new. Reindenting a file
or moving a block makes git mark untouched comments as added; findings
are matched against the comment text at the base, so only genuinely new
content reports.

The whole file is read and every comment in it evaluated. Only the
*reporting* is filtered, so a rule still sees the code a comment
introduces, the full run it belongs to, and the base version of the
file.

Existing tree is untouched. `task pre-commit:comment-lint:all` reports
it and always exits 0:

| | java | ts/js | py |
| --- | --- | --- | --- |
| findings | 1,218 | 741 | 204 |

2,163 across 542 files, mostly `CMT002` banners (1,482) and `CMT001`
restatements (456). Clearing it is separate work, by directory.

Not in this PR: an advisory LLM review layer for the things no pattern
can judge.

## Verification

Run against
[#7494](https://github.com/Stirling-Tools/Stirling-PDF/pull/7494) as CI
would, in a throwaway worktree: **two findings on a 78 file, +4,512 line
change, both genuine banners, in 952ms**. A whole-file scan of those
same files gives 11; the other 9 were withheld because that PR's author
did not write them, and they are the `@param teamId the team ID` shape
this standard exists to stop.

Both scanners blank string and character literals before looking for
comment markers, because a partial lex desynchronises everything after
it: one apostrophe in a Java comment, or one Python template whose
closing quotes start a line, is enough to read dozens of lines of code
as a single comment. Two fixtures carry canaries that stop being
reported if either engine ever desynchronises again.

The [fixture
corpus](https://github.com/Stirling-Tools/Stirling-PDF/tree/claude/ai-pr-comment-quality-dd970e/scripts/lint/fixtures)
pins all 9 rules against both engines, and `--selftest` fails if the two
disagree about the same file.

## Two things reviewers should know

**The oxlint JS plugin API is alpha.** oxlint itself is stable and
already this repo's frontend linter; the plugin API is the new
dependency. Its documented failure mode
([oxc#25203](https://github.com/oxc-project/oxc/issues/25203)) is being
skipped silently while oxlint still reports success. That affects the
standalone release binary rather than the npm package this invokes, but
the class of failure reads exactly like clean code, so the run asserts
`number_of_rules >= 1` from oxlint's own report and a broken engine
exits 2 rather than passing. If the API ever breaks, the fallback is
folding these rules into the line scanner, which already implements all
nine for Java and Python.

**`.claude/settings.json` is now committed**, carrying the hook and
nothing else: 19 lines, no `permissions`, nothing machine-specific. That
partly reverts `c35546a212` ("Ignore claude dir"), which existed because
this file had twice been committed by accident with a personal
`permissions` allowlist, once with absolute machine paths. Personal
config still belongs in `.claude/settings.local.json`, which the new
pattern keeps ignored, and hook entries merge across the two so nobody's
own hooks are lost.

If you already hand-wrote a `.claude/settings.json`, copy it somewhere
first: that path used to be git-ignored, and git overwrites an ignored
file without warning when a commit starts tracking it. Across 19 local
checkouts here, 13 have `settings.local.json` and none has a
hand-written `settings.json`.

To turn the hook off, `{ "env": { "COMMENT_LINT_HOOK": "0" } }` in local
settings. Claude Code can only disable all hooks at once, hence the
switch. The commit-time gate still applies.

## How to test

```bash
task pre-commit:comment-lint:ci
```

The fixture corpus, then the diff. The corpus checks the rules
themselves rather than the code under review, so it runs on CI and
before a rule change, not on every local commit.

```bash
task comment-lint:branch
```

`clean (34 files in scope)`. `task comment-lint` is the same thing
scoped to uncommitted work, which is what the git hook and CI run.

To watch it bite, add `// Is banner` above `export function isBanner` in
`scripts/lint/comment-rules.mjs` and run `task comment-lint`: one
`CMT001`, exit 1. The gate covers its own source, which is why these
scripts have no section dividers.

```bash
task pre-commit:comment-lint:all
```

The standing backlog, report-only.

Verified on the pinned oxlint 1.77.0, not only the 1.79 the plugin was
prototyped against.
2026-08-28 10:56:50 +00:00

9.7 KiB

Code comments

A comment must carry information the code cannot. If a reader could derive it from the code in front of them, delete it: a redundant comment still has to be maintained, will eventually contradict the code, and dilutes the comments that matter.

The operative rules are in AGENTS.md, kept short so they stay in an agent's context. This document is the reasoning and the worked examples behind them, plus how to run the linter.

Comment the current state

Describe the code as it is. Not what it used to be, not what changed, not why it changed. A comment that narrates history is stale the moment the next change lands, and git already holds that record.

When you know the history and it explains the shape of the code, the useful half is the reason, not the sequence. State the reason:

// Don't:
// This used to reimplement the modal internals, which is how the procurement
// dialogs drifted from the billing ones.

// Do:
// Thin wrapper over the shared Modal: duplicating its portal and focus trap is
// how dialogs drift apart.

Future state is the exception, and it belongs in a TODO with an issue.

The four jobs

Contract. What a caller must know that the signature cannot say: preconditions, invariants, units, ownership and lifetime, thread-safety, error semantics, side effects.

The bound is the surface, not the volume: document the contract of everything a caller outside the file can reach, and nothing else. Inside that surface say whatever a caller needs; outside it a comment earns its place on the same terms as any other.

/**
 * Authority on which filesystem locations a policy may read or write. Fail-closed
 * in order: denied entirely under the saas profile; Stirling's own config dir is
 * always rejected; the path must resolve within policies.allowedFolderRoots.
 *
 * <p>Compared after normalisation so {@code ..} cannot escape a root. Symlink
 * escape is not defended: an operator who roots an allowlist on a symlink to a
 * sensitive location is trusted.
 */

Why. The constraint the code satisfies, the bug it avoids, the alternative rejected and the reason.

// whenComplete runs on the worker thread after the run finishes, so the
// terminal event never races the step events.
handle.completion()

A reference is supplementary, never load-bearing: the comment must survive deleting it. // See #1234 is a dead end.

// flatten() reads the annotation list that save() clears, so saving first loses
// every annotation (#6865).
document.flatten(annotations);

Prefer a spec (RFC 3161, ISO 4217) or a CVE where one applies. Both are immutable; a ticket can be closed, moved or made private.

Hazard. "Must stay in sync with X." "Order matters because Y." "Do not remove, it prevents Z."

Map. A short orientation at the head of a genuinely complex file: what it owns, and what it deliberately does not.

The test that decides it

A comment earns its place when it sits at a different level of detail than the line below it: lower, stating a precise fact the code implies but does not say, or higher, giving intent a reader would otherwise assemble from ten lines. Same-altitude is the definition of redundant.

  • Delete it. Is any information lost? If not, it stays deleted.
  • Could a name carry it instead? A better identifier, an extracted function or a named constant beats a comment. Prefer the code change.

What not to write

Don't Instead
// Handle drag start above handleDragStart Nothing. The name already says it.
// ─── Types ───, // Helpers, // ==== If a file needs internal signposting, split the file.
// Step 1: narrating a function body Extract functions. If the steps need labels they need names.
// No longer needed, // Previously this used X State why the code is as it is now, or nothing.
Commented-out code Delete it. Git remembers.
@param blob - The blob to download Omit the tag rather than pad it.
Docs on a self-explanatory member Nothing, unless there is a real constraint to state.

Step numbering is fine where it labels a genuinely numbered thing, such as a wizard step or a step in a written test procedure. It is narration when it numbers the lines of one function.

Comments at the end of a line

A trailing comment usually does a different job from one above the code: it decodes the line it sits on. Those are worth keeping, and the linter leaves them alone.

byte[] pdfBytes = {0x25, 0x50, 0x44, 0x46};   // "%PDF"
long maxAttachmentSize = 50L * 1024 * 1024;   // 50 MB
double buffer = 0.10;                         // 10% headroom
default -> toBytes(value, 2);                 // MB

Each overlaps in words with the code and each adds the interpretation the code leaves implicit, which is the lower-altitude case the test above asks for. So CMT001 does not judge trailing comments; on this codebase it would have been wrong about roughly six in seven of them.

What still applies is anything that does not depend on the code below: a trailing // TODO fix this is as unowned as one on its own line, and a trailing // this used to run before the flush narrates history wherever it sits.

A comment block over about 12 lines, outside a file or type header, is usually a sign the code needs restructuring. If it is genuinely product documentation, it belongs in the docs repo.

TODOs

A TODO needs an issue, because an issue is the only part that will close it:

// TODO(#1234): re-enable the checkout gate once account syncing lands

An owner is not a substitute: a username goes stale when someone changes team and means nothing to an outside contributor. If the work is not worth an issue it is not worth a TODO, and the options are to do it now or leave the code alone. A question is not a TODO.

Per language

Java. Google Java Style, which this repo already formats to. Its §7.3.1 exception applies: omit Javadoc on a self-explanatory member where there is genuinely nothing to add, but do not cite it to skip something a reader needs. Summary fragments are noun or verb phrases, not sentences starting "This method returns".

TypeScript. JSDoc on the @app/* seams, exported hooks, and anything crossing a layer boundary. No @param/@returns that restates a typed signature. JSX comments follow the same rules as any other.

Python. Docstrings on modules, public functions and Pydantic models where the contract is not obvious from the type.

The linter

task comment-lint          # what the working tree adds over HEAD
task comment-lint:branch   # what the branch adds over origin/main (BASE=<ref> to change)
task pre-commit:comment-lint:ci    # the fixture corpus, then the diff

comment-lint is the pre-commit question, so it reports nothing once you have committed; on a CI pull request it compares against the target branch via GITHUB_BASE_REF. comment-lint:branch is the review question. The corpus checks the rules themselves rather than the code under review, so it runs on CI and before a rule change, not on every local commit.

task comment-lint also runs inside task pre-commit, and as a Claude Code Stop hook, so an agent is told before it finishes a turn and fixes the comment inside that turn. Stop rather than per file write: a run costs the same for one file as for twenty-five, and half of all writes in a turn go to a file already written in it.

Findings are scoped to comment text that is new, not to lines git calls new, so reindenting or moving code does not resurface comments you did not write.

The rules are the RULES object in scripts/lint/comment-rules.mjs; the exact condition for each is the predicate of the same name in that file, with the readings it deliberately excludes beside it.

Every rule blocks. A rule that only warns is a rule nobody acts on. So a finding you believe is wrong is a bug in the rule, not something to live with: narrow the rule, or mark the line and say why.

Every comment form the repo writes is covered: // and /* */, Javadoc and JSDoc, JSX comments, #, and Python docstrings. CMT007 reads all three parameter conventions in use here, Javadoc/JSDoc @param, Sphinx :param name: and Google name: description under Args:.

Two engines, one rule set. .ts/.tsx/.mjs go to an oxlint JS plugin, so comments come from the parser: a // inside a string is not a comment, and JSX {/* … */} is. .java/.py go to a line scanner. Neither reads the other's files, so they cannot disagree about one file. scripts/lint/fixtures/ is the corpus that keeps them meaning the same thing.

When a finding is wrong

Name the rule on the line above:

// comment-lint-allow: CMT002
// ─── kept deliberately, because <reason> ───

There is no form that disables every rule, and the directive has to earn its place. CMT008 reports one that names something which is not a rule, and one that silences nothing, so a typo does not read as a suppression and a stale suppression does not sit there blinding the line. The whole comment must be the directive; prose that mentions the syntax is just prose.

If you reach for this more than occasionally the rule is wrong: fix it in comment-rules.mjs and update the fixture corpus in the same commit, so the diff shows what moved.

The existing backlog

task pre-commit:comment-lint:all reports the whole tree and never fails. There is a standing backlog being cleared by directory; diff scoping is what keeps it off whoever touches a file first.

To turn the editor hook off, put { "env": { "COMMENT_LINT_HOOK": "0" } } in .claude/settings.local.json. The commit-time gate still applies, so you lose the early warning rather than the check.