mirror of
https://github.com/Stirling-Tools/Stirling-PDF.git
synced 2026-09-02 21:03:34 +03:00
ConnorYoh
4ab2505a6c
## The problem AI PRs write comments that restate the line below them, mark sections with box drawing, and narrate the diff. Nothing in the repo said not to, and nothing checked. `AGENTS.md` had one line about comments and it was buried in the Python section. Banners and `Step N:` narration have zero occurrences in the 15 months before Aug 2025, so this is new. ## The fix A written standard, plus a linter that enforces the mechanical part of it on added lines only. - [devGuide/CODE_COMMENTS.md](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/devGuide/CODE_COMMENTS.md) holds the reasoning and worked examples; a section in [AGENTS.md](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/AGENTS.md) holds the operative rules, kept short so they stay in an agent's context. The two are split by kind rather than duplicated, because the same prose in two places drifts. - Rules in [comment-rules.mjs](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs), shared by both engines. - Two engines. `.ts` / `.tsx` / `.mjs` go to an [oxlint JS plugin](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-lint-oxlint-plugin.mjs) so comments come from the parser rather than a line scan; `.java` / `.py` go to a [line scanner](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-lint.mjs). Neither reads the other's files, so they cannot disagree about one file. - Between them they read every comment form the repo writes: `//` and `/* */`, Javadoc and JSDoc, JSX comments, `#`, and Python docstrings. - Runs in `task pre-commit`, so the git hook and the `pre_commit.yml` CI job both get it, and as a Claude Code [`Stop` hook](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-lint-hook.mjs) so an agent fixes the comment inside the turn that wrote it. ## The rules The part worth arguing about. **Every rule blocks.** A rule that only warns is a rule nobody acts on, so a finding you believe is wrong is a bug in the rule: narrow it, or mark the line and say why. | | Fires on | | --- | --- | | [CMT001](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L71) | Every word in the comment already appears in the code below it. Max 6 words, skipped for prose punctuation and for a bare Arrange/Act/Assert marker | | [CMT002](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L92) | 4+ rule or box-drawing characters, or a bare section label from [a fixed list](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L84) (`Types`, `Helpers`, `State`, `Handlers`, ...) | | [CMT003](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L110) | `Step N:` with a separator, or `Then,` / `Next,` / `Finally,`. Suppressed in test files | | [CMT004](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L129) | A comment about the code's own past: `this used to`, `renamed from`, `was previously called`. Suppressed in test files | | [CMT005](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L154) | 3+ consecutive comment lines where 2/3 [parse as code](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L143) | | [CMT006](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L31) | A run of implementation comment over 12 lines, outside the first 5 lines of a file. Doc blocks are exempt, because the standard asks for thorough contracts | | [CMT007](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L180) | A parameter or return description that adds no word its name lacks. Reads Javadoc/JSDoc `@param`, Sphinx `:param name:` and Google `name: description` | | [CMT008](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L239) | An allow directive naming a rule that does not exist, or one that silenced nothing | | [CMT009](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L219) | A `TODO` / `FIXME` / `HACK` naming no issue or link. An owner is not accepted: a username goes stale, an issue outlives it | Each rule carries the readings it deliberately excludes, next to the rule. Those exclusions came from running the rules over this repo, not from taste: `CMT004` does not match a bare "no longer needed" because that is as often about runtime lifecycle as about history, and `CMT003` needs a separator after the number so a wrapped line beginning "step 2 unmounts + remounts the panel" reads as the prose it is. A comment sharing a line with code is judged by the rules that do not depend on the code below it, so a trailing `// TODO fix this` or `/* this used to run before the flush */` still reports, while `50L * 1024 * 1024 // 50 MB` does not. `CMT001` would have been wrong about six in seven trailing comments here, so it stays out of them. If a finding is wrong, `// comment-lint-allow: CMT002` on the line above. Rule-specific, [no blanket disable](https://github.com/Stirling-Tools/Stirling-PDF/blob/claude/ai-pr-comment-quality-dd970e/scripts/lint/comment-rules.mjs#L229). A directive naming a rule that does not exist, or silencing nothing, is itself a `CMT008` failure, so a typo cannot quietly disable a rule and a stale one gets deleted rather than accumulating. No native linter covers `CMT007`. `eslint-plugin-jsdoc`'s `require-param-description`, Checkstyle's `NonEmptyAtclauseDescription` and ruff's D-rules all check that a description exists, not whether it says anything. ## Scoping Added comment **text** only, not lines git calls new. Reindenting a file or moving a block makes git mark untouched comments as added; findings are matched against the comment text at the base, so only genuinely new content reports. The whole file is read and every comment in it evaluated. Only the *reporting* is filtered, so a rule still sees the code a comment introduces, the full run it belongs to, and the base version of the file. Existing tree is untouched. `task pre-commit:comment-lint:all` reports it and always exits 0: | | java | ts/js | py | | --- | --- | --- | --- | | findings | 1,218 | 741 | 204 | 2,163 across 542 files, mostly `CMT002` banners (1,482) and `CMT001` restatements (456). Clearing it is separate work, by directory. Not in this PR: an advisory LLM review layer for the things no pattern can judge. ## Verification Run against [#7494](https://github.com/Stirling-Tools/Stirling-PDF/pull/7494) as CI would, in a throwaway worktree: **two findings on a 78 file, +4,512 line change, both genuine banners, in 952ms**. A whole-file scan of those same files gives 11; the other 9 were withheld because that PR's author did not write them, and they are the `@param teamId the team ID` shape this standard exists to stop. Both scanners blank string and character literals before looking for comment markers, because a partial lex desynchronises everything after it: one apostrophe in a Java comment, or one Python template whose closing quotes start a line, is enough to read dozens of lines of code as a single comment. Two fixtures carry canaries that stop being reported if either engine ever desynchronises again. The [fixture corpus](https://github.com/Stirling-Tools/Stirling-PDF/tree/claude/ai-pr-comment-quality-dd970e/scripts/lint/fixtures) pins all 9 rules against both engines, and `--selftest` fails if the two disagree about the same file. ## Two things reviewers should know **The oxlint JS plugin API is alpha.** oxlint itself is stable and already this repo's frontend linter; the plugin API is the new dependency. Its documented failure mode ([oxc#25203](https://github.com/oxc-project/oxc/issues/25203)) is being skipped silently while oxlint still reports success. That affects the standalone release binary rather than the npm package this invokes, but the class of failure reads exactly like clean code, so the run asserts `number_of_rules >= 1` from oxlint's own report and a broken engine exits 2 rather than passing. If the API ever breaks, the fallback is folding these rules into the line scanner, which already implements all nine for Java and Python. **`.claude/settings.json` is now committed**, carrying the hook and nothing else: 19 lines, no `permissions`, nothing machine-specific. That partly reverts `c35546a212` ("Ignore claude dir"), which existed because this file had twice been committed by accident with a personal `permissions` allowlist, once with absolute machine paths. Personal config still belongs in `.claude/settings.local.json`, which the new pattern keeps ignored, and hook entries merge across the two so nobody's own hooks are lost. If you already hand-wrote a `.claude/settings.json`, copy it somewhere first: that path used to be git-ignored, and git overwrites an ignored file without warning when a commit starts tracking it. Across 19 local checkouts here, 13 have `settings.local.json` and none has a hand-written `settings.json`. To turn the hook off, `{ "env": { "COMMENT_LINT_HOOK": "0" } }` in local settings. Claude Code can only disable all hooks at once, hence the switch. The commit-time gate still applies. ## How to test ```bash task pre-commit:comment-lint:ci ``` The fixture corpus, then the diff. The corpus checks the rules themselves rather than the code under review, so it runs on CI and before a rule change, not on every local commit. ```bash task comment-lint:branch ``` `clean (34 files in scope)`. `task comment-lint` is the same thing scoped to uncommitted work, which is what the git hook and CI run. To watch it bite, add `// Is banner` above `export function isBanner` in `scripts/lint/comment-rules.mjs` and run `task comment-lint`: one `CMT001`, exit 1. The gate covers its own source, which is why these scripts have no section dividers. ```bash task pre-commit:comment-lint:all ``` The standing backlog, report-only. Verified on the pinned oxlint 1.77.0, not only the 1.79 the plugin was prototyped against.