feat(bughunt): coverage-driven convergence (#1399)

* feat(bughunt): coverage-driven stop rule and directory-coherent sweep

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(bughunt): return uncredited explore draws to the pool

An explore lens denied coverage credit (dead finder or unverified
candidates) now un-consumes its draw so later rounds re-offer the files;
consumed-but-uncovered files could otherwise pin uncoveredCount above
zero and block convergence. Directory grouping reuses clusterOf().

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(bughunt): stalled-coverage guard, risky-file class sweep, exhausted-dry convergence

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(bughunt): stall guard never stops a still-confirming hunt

A round with newConfirmed > 0 resets the coverage-stall counter instead
of counting toward it; hotspot yield does not shrink the uncovered pool,
and a stuck sweep must not cut off a hunt that is still finding bugs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(bughunt): coverage telemetry in report and operator docs

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(bughunt): scoped-hunt coverage trap and current cost estimate

Final-review fixes: warn that args.lenses plus an examined-armed inventory
still sweeps the whole pool (pass a filtered inventory or legacy rows to
truly scope), and align the budget note with the coverage-run estimate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
J3vb
2026-08-20 21:24:41 +02:00
committed by GitHub
co-authored by Claude Fable 5
parent 5202e3fe1e
commit c22bc14946
3 changed files with 352 additions and 38 deletions
+35 -5
View File
@@ -16,7 +16,7 @@ reach a commit, an issue, or a PR body.
## 1. Hunt
**Launch the hunt from a turn that carries a token-budget directive** (recommended:
`+25M`, comfortably above a full 8-round run). The workflow's cost ceiling is gated
`+25M`, comfortably above a full coverage run's ~8-12M). The workflow's cost ceiling is gated
on `budget.total`, which is null without a directive — a directive-less run has **no
ceiling at all**. The workflow's first log line echoes the state: `budget=25M` means
armed; `budget=NONE - cost ceiling disarmed` means stop the run and relaunch with a
@@ -26,9 +26,19 @@ Before launching, in order:
1. **Rebuild the graph** (stale coordinates aim the explore lens at moved code):
`graphify update . --no-cluster` — local tree-sitter, zero LLM cost, ~10.7k nodes.
2. **Rank explore targets**: `node .superpowers/rank-explore.mjs` — writes
`.superpowers/explore-ranking.json`, deprioritizing files recorded clean in
`.superpowers/explored-clean.json` and dropping files that no longer exist.
2. **Build the inventory**: `node .superpowers/rank-explore.mjs` — writes
`.superpowers/explore-ranking.json`: EVERY non-test source file (~419 rows), each with
`examined` (already carries a ledger finding or a LIVE explored-clean record → the hunt
pre-seeds its covered set), `risky` (top coupling past-bug clusters top churn,
capped at 40 → they get an extra pass through all 5 bug-class lenses), and `churn`.
Explored-clean records carry content hashes: editing a file expires its clean record,
so re-runs automatically re-hunt what changed. The hunt cannot stop while any inventory
file is uncovered, so a full run now takes ~10-20 rounds and ~8-12M tokens — the `+25M`
directive still covers it. Regenerate the inventory and read `known` from the ledger in
the SAME session step: both derive from `findings-ledger.json`, and every `known` file
must be `examined` in the inventory — a `known` file the inventory does not mark
examined can never be drawn (the seen-filter blocks it) nor covered, which would
strand `uncoveredCount()` above zero and block convergence.
3. Read the ledger and pass every record in as `known`, so the hunt does not
re-derive anything already found, fixed, declined, or refuted.
@@ -39,7 +49,7 @@ Workflow({
known: <every record from findings-ledger.json, as {file, line, title, status}>,
graph: <the rows of .superpowers/explore-ranking.json>,
lenses: [ {key, prompt}, ... ], // optional: scope the hunt to one subsystem
maxRounds: 8,
maxRounds: 30, // safety backstop only - coverage + dry is the real stop
dryThreshold: 2,
},
})
@@ -49,8 +59,28 @@ If `graph` is omitted or empty the hunt logs
`explore: args.graph absent/empty - falling back to churn-based fresh eyes` and
still runs — degraded targeting, never a smaller lens family.
`converged: true` now means: every inventory file was covered by a completed
explicit-file lens (or carries a verdict), the risky class sweep ran, and then
`dryThreshold` consecutive eligible rounds confirmed nothing (a round where the
lens family comes up empty with the pool drained counts as dry — family
`exhausted`). Rows without the `examined` field fall back to the old
quietness-only stop. Two new run outcomes: `stalledCoverage: true` means adaptive
rounds stopped shrinking the uncovered pool (usually mass finder failures —
investigate before re-running); a budget stop now reports
`coverage.uncoveredAtStop` so the next run knows exactly what remains (re-run
with the ledger as `known`; live explored-clean records pre-cover what was
finished, so the sweep naturally continues where it stopped).
Omit `lenses` for a general hunt across the rotating families.
**Scoping a hunt while coverage mode is armed is a budget trap:** `lenses` only
replaces round 1, and inventory rows with `examined` force the coverage stop
rule — from round 2 the run sweeps the ENTIRE uncovered pool and the risky
sweep before it may converge, at general-hunt cost. For a true scoped hunt,
pass a subsystem-filtered inventory as `graph` (only the rows you want swept),
or rows without the `examined` field to fall back to the legacy quietness-only
stop.
Each lens object is `{key, prompt}`. `key` must match `^[a-z0-9-]+$`
lowercase letters, digits, and hyphens only. Keys get interpolated into agent
labels of the form `r<N>:hunt:<key>:<model>`, and a key containing uppercase,
+192
View File
@@ -73,6 +73,11 @@ export const finding = (n, over = {}) => ({
})
export const graphRows = (n) =>
Array.from({ length: n }, (_, i) => ({ file: `Server/gen/g${i}.go`, score: 1 - i / (n + 1), degree: 10, cited: 5 }))
export const inventoryRows = (n, over = () => ({})) =>
Array.from({ length: n }, (_, i) => ({
file: `Server/gen/g${i}.go`, degree: 10, cited: 5, score: 1 - i / (n + 1),
examined: false, risky: false, ...over(i),
}))
export const confirmAll = (cands) => ({
verdicts: cands.map((c) => ({
title: c.title, file: c.file, line: c.line,
@@ -845,6 +850,193 @@ scenarios.s_explore_rewind_on_thrown_stage = async () => {
assert.equal(result.rounds[3].dryEligible, false, 'a nulled lens result still makes the round ineligible')
}
// COV1 (spec §3): coverage mode may NOT stop on quietness while inventory files are uncovered.
// 60 rows, 10 pre-examined -> 50 to sweep. Nothing is ever found, so dry passes the threshold
// at round 2 - the old stop rule would have converged there. The new rule keeps going until
// round 4's explore lenses (quota 4 + backfill 2 slots; 5 draw files, the 6th comes up empty)
// cover all 50, then exits. Also locks the enriched explore prompt (class checklist).
scenarios.s_coverage_blocks_stop = async () => {
const inv = inventoryRows(60, (i) => (i >= 50 ? { examined: true } : {}))
const { result, calls } = await run({
args: { graph: inv },
agentStub: makeStub({ hunt: () => none, verify: (r, k, c) => confirmAll(c) }),
})
assert.equal(result.rounds.length, 4, 'must run past the dry threshold (hit at r2) to sweep in r4')
assert.equal(result.converged, true)
assert.deepEqual(result.rounds.map((r) => r.dryAfter), [1, 2, 3, 4])
assert.deepEqual(result.runStats.coverage, { inventory: 60, preCovered: 10, covered: 60, uncoveredAtStop: 0 })
assert.match(result.report, /CONVERGED/)
assert.match(result.report, /Coverage: 60\/60 files \(10 pre-covered/)
const ep = (calls.find((c) => (c.opts.label || '') === 'r4:hunt:explore-1:opus') || {}).prompt || ''
assert.match(ep, /error-path data loss/, 'explore lenses carry the distilled class checklist')
}
// COV2 (spec §2+§4): a dead explore finder's files stay uncovered and get re-offered; the
// run only converges after a LIVE lens covers them.
scenarios.s_coverage_dead_finder = async () => {
const inv = inventoryRows(20)
const { result, calls } = await run({
args: { graph: inv },
agentStub: makeStub({
hunt: (round, key) => (round === 4 && key === 'explore-1' ? null : none),
verify: (r, k, c) => confirmAll(c),
}),
})
const promptOf = (rnd, key) => (calls.find((c) => (c.opts.label || '') === `r${rnd}:hunt:${key}:opus`) || {}).prompt || ''
assert.match(promptOf(4, 'explore-1'), /Server\/gen\/g0\.go/, 'r4 explore-1 drew the head of the pool')
assert.match(promptOf(5, 'explore-1'), /Server\/gen\/g0\.go/, 'dead lens files are re-offered next round')
assert.equal(result.rounds[3].dryEligible, false, 'dead finder keeps the round ineligible')
assert.equal(result.converged, true)
assert.deepEqual(result.runStats.coverage, { inventory: 20, preCovered: 0, covered: 20, uncoveredAtStop: 0 })
}
// COV7 (amendment 4): explore draws are directory-coherent - one lens reads one module,
// not ten strangers. Cross-file classes (state desync, acquire/release pairs) need siblings
// in one agent's context.
scenarios.s_directory_coherent_draws = async () => {
const inv = inventoryRows(20, (i) => ({ file: i % 2 === 0 ? `Server/alpha/a${i}.go` : `Server/beta/b${i}.go` }))
const { calls } = await run({
args: { graph: inv },
agentStub: makeStub({ hunt: () => none, verify: (r, k, c) => confirmAll(c) }),
})
const p1 = (calls.find((c) => (c.opts.label || '') === 'r4:hunt:explore-1:opus') || {}).prompt || ''
assert.match(p1, /Server\/alpha\/a0\.go/)
assert.match(p1, /Server\/alpha\/a18\.go/, 'all ten alpha files ride in the first lens')
assert.doesNotMatch(p1, /Server\/beta\//, 'no stranger directories in a coherent draw')
}
// COV8 (Task 5 review finding): an explore lens whose candidates never get a verdict is
// denied coverage credit - its draw must return to the pool and be re-offered, or the
// consumed-but-uncovered files strand uncoveredCount() above zero and the run can never
// converge (it would grind to the round backstop instead).
scenarios.s_uncredited_draw_returns_to_pool = async () => {
const inv = inventoryRows(20)
const { result, calls } = await run({
args: { graph: inv },
agentStub: makeStub({
hunt: (round, key) =>
round === 4 && key === 'explore-1'
? { findings: [finding(1, { file: 'Server/gen/g0.go', title: 'orphaned candidate one' })] }
: none,
verify: (round, key, cands) =>
round === 4 && key === 'explore-1' ? { verdicts: [] } : confirmAll(cands),
}),
})
const promptOf = (rnd, key) => (calls.find((c) => (c.opts.label || '') === `r${rnd}:hunt:${key}:opus`) || {}).prompt || ''
assert.match(promptOf(4, 'explore-1'), /Server\/gen\/g0\.go/)
assert.match(promptOf(5, 'explore-1'), /Server\/gen\/g0\.go/, 'uncredited draw is re-offered next round')
assert.equal(result.rounds[3].dryEligible, false, 'unverified candidates keep the round ineligible')
assert.equal(result.converged, true, 'the run must still converge once a later lens covers the files')
assert.deepEqual(result.runStats.coverage, { inventory: 20, preCovered: 0, covered: 20, uncoveredAtStop: 0 })
assert.equal(result.unverified.length, 1, 'the orphaned candidate stays reported unverified')
}
// COV3 (spec §4): finders dying every adaptive round -> uncovered never shrinks -> stop with
// stalledCoverage after 2 stalled adaptive rounds instead of burning 26 more rounds.
scenarios.s_coverage_stall = async () => {
const inv = inventoryRows(20)
const { result, logs } = await run({
args: { graph: inv },
agentStub: makeStub({
hunt: (round, key) => (/^explore-/.test(key) ? null : none),
verify: (r, k, c) => confirmAll(c),
}),
})
assert.equal(result.stalledCoverage, true)
assert.equal(result.converged, false)
assert.equal(result.rounds.length, 5, 'r1-3 families, then exactly 2 stalled adaptive rounds')
assert.equal(result.runStats.coverage.uncoveredAtStop, 20)
assert.ok(logs.some((l) => /Coverage stalled/.test(l)))
assert.match(result.report, /NOT converged - coverage stalled/)
}
// COV4 (spec §2 smart depth): once the sweep completes, the 5 bug-class lenses run once more
// scoped to the risky files - then the run may converge.
scenarios.s_risky_sweep = async () => {
const inv = inventoryRows(10, (i) => (i < 2 ? { risky: true } : {}))
const { result, calls } = await run({
args: { graph: inv },
agentStub: makeStub({ hunt: () => none, verify: (r, k, c) => confirmAll(c) }),
})
const r5keys = [...new Set(calls.filter((c) => /^r5:hunt:/.test(c.opts.label || '')).map((c) => c.opts.label.split(':')[2]))]
assert.deepEqual(r5keys.sort(), ['risky-concurrency', 'risky-error-paths', 'risky-lifecycle', 'risky-ordering-boundary', 'risky-state-desync'])
const p = calls.find((c) => (c.opts.label || '') === 'r5:hunt:risky-concurrency:opus').prompt
assert.match(p, /Server\/gen\/g0\.go/)
assert.match(p, /Server\/gen\/g1\.go/)
assert.doesNotMatch(p, /Server\/gen\/g5\.go/, 'the sweep is scoped to risky files only')
assert.equal(result.rounds[4].family, 'risky-sweep')
assert.equal(result.converged, true)
assert.equal(result.rounds.length, 5, 'dry was already past threshold - the run ends right after the risky sweep')
assert.ok(!calls.some((c) => /^r6:hunt:risky-/.test(c.opts.label || '')), 'the risky sweep runs exactly once')
}
// COV5: legacy rows (no `examined` key) leave every new mechanism inert - old stop rule,
// no coverage stats, no risky sweep.
scenarios.s_legacy_rows_inert = async () => {
const { result, calls } = await run({
args: { graph: graphRows(30) },
agentStub: makeStub({ hunt: () => none, verify: (r, k, c) => confirmAll(c) }),
})
assert.equal(result.rounds.length, 2, 'legacy mode still converges on the plain dry threshold')
assert.equal(result.converged, true)
assert.equal(result.runStats.coverage, null)
assert.equal(result.stalledCoverage, false)
assert.ok(!calls.some((c) => /risky-/.test(c.opts.label || '')))
}
// COV6 (amendment 1): a confirm on the sweep's last round resets dry to 0; hotspot rounds then
// go clean, cooldown + an empty pool empty the family - in coverage mode that emptiness IS
// quietness and must count dry rounds instead of stranding a fully-covered run at converged:false.
scenarios.s_exhausted_counts_dry = async () => {
const inv = inventoryRows(10)
const { result } = await run({
args: { graph: inv },
agentStub: makeStub({
hunt: (round, key) =>
round === 4 && key === 'explore-1'
? { findings: [finding(1, { file: 'Server/gen/g0.go', title: 'late sweep bug one' })] }
: none,
verify: (r, k, c) => confirmAll(c),
}),
})
assert.equal(result.converged, true, 'a fully-covered, fully-quiet run must converge')
assert.equal(result.rounds.length, 6)
assert.equal(result.rounds[5].family, 'exhausted')
assert.equal(result.rounds[5].dryAfter, 2)
assert.equal(result.confirmed.length, 1)
}
// COV9 (Task 6 review finding): a stuck pool must NOT stop a hunt that is still confirming.
// Dead explore lenses pin uncovered at 20 while hotspot lenses confirm fresh bugs in rounds
// 4 and 6 - each productive round resets the stall counter, so the run survives to round 8
// and only then stops with stalledCoverage (without the productivity term it would have
// stopped at round 5, mid-yield).
scenarios.s_stall_deferred_while_productive = async () => {
const inv = inventoryRows(20)
const { result, logs } = await run({
args: { graph: inv },
agentStub: makeStub({
hunt: (round, key) => {
if (round === 1 && key === 'ws-hub')
return { findings: [finding(1, { file: 'Server/ws/hub.go', title: 'seed bug alpha one' })] }
if (round === 4 && key === 'hotspot-server-ws')
return { findings: [finding(2, { file: 'Server/ws/emit.go', title: 'adjacent bug beta two' })] }
if (round === 6 && key === 'hotspot-server-ws')
return { findings: [finding(3, { file: 'Server/ws/pubsub.go', title: 'adjacent bug gamma three' })] }
if (/^explore-/.test(key)) return null // dead explore finders: the pool never shrinks
return none
},
verify: (r, k, c) => confirmAll(c),
}),
})
assert.equal(result.rounds.length, 8, 'productive rounds 4 and 6 must defer the stall to round 8')
assert.equal(result.stalledCoverage, true)
assert.equal(result.converged, false)
assert.equal(result.confirmed.length, 3)
assert.equal(result.runStats.coverage.uncoveredAtStop, 20)
assert.ok(logs.some((l) => /Coverage stalled/.test(l)))
}
// ---------- runner ----------
const only = process.argv[2]
for (const [name, fn] of Object.entries(scenarios)) {
+125 -33
View File
@@ -15,7 +15,7 @@ const ARGS = (() => {
}
return args || {}
})()
const MAX_ROUNDS = ARGS.maxRounds || 8
const MAX_ROUNDS = ARGS.maxRounds || 30
const DRY_THRESHOLD = ARGS.dryThreshold || 2
// A scoped hunt (args.lenses) replaces the round-1 family outright; later rounds still go
// adaptive, so hotspot and explore coverage - and therefore convergence - still work.
@@ -281,17 +281,31 @@ const FLOW_LENSES = [
},
]
let riskySweepDone = false
function riskySweepLenses() {
if (riskySweepDone || !HAS_INVENTORY || !RISKY_FILES.length || uncoveredCount() > 0) return null
riskySweepDone = true // consumed even if this round's finders die: same at-most-once semantics as cooldown
const list = RISKY_FILES.map((f) => ` - ${f}`).join('\n')
return BUGCLASS_LENSES.map((l) => ({
key: `risky-${l.key}`,
prompt: `${l.prompt}\n\nScope this sweep to ONLY these highest-risk files (read each one in full):\n${list}`,
}))
}
let currentFamilyName = 'surfaces'
function lensesForRound(round) {
if (CUSTOM_LENSES) return round === 1 ? CUSTOM_LENSES : buildAdaptiveLenses(round)
if (round === 1) return SURFACE_LENSES
if (round === 2) return BUGCLASS_LENSES
if (round === 3) return FLOW_LENSES
return buildAdaptiveLenses(round)
}
function familyName(round) {
if (CUSTOM_LENSES) return round === 1 ? 'custom' : 'adaptive'
return ['surfaces', 'bug-classes', 'flows'][round - 1] || 'adaptive'
const pick = (name, lenses) => { currentFamilyName = name; return lenses }
if (CUSTOM_LENSES) {
if (round === 1) return pick('custom', CUSTOM_LENSES)
} else {
if (round === 1) return pick('surfaces', SURFACE_LENSES)
if (round === 2) return pick('bug-classes', BUGCLASS_LENSES)
if (round === 3) return pick('flows', FLOW_LENSES)
}
const risky = riskySweepLenses()
if (risky) return pick('risky-sweep', risky)
return pick('adaptive', buildAdaptiveLenses(round))
}
function familyName() { return currentFamilyName }
// Directory granularity: the old two/three-segment cluster collapsed the whole TS client into
// one bucket (35 of 82 findings), so the "top cluster" never changed for five straight rounds.
function clusterOf(file) {
@@ -303,6 +317,17 @@ function clusterOf(file) {
// args.graph: session-computed coupling ranking (rank-explore.mjs). The workflow only reads
// .file - scoring already happened outside, where the filesystem is.
const GRAPH_ROWS = (Array.isArray(ARGS.graph) ? ARGS.graph : []).filter((r) => r && typeof r.file === 'string')
// ---------- coverage mode (spec 2026-08-20) ----------
// Arms only when rows carry the `examined` flag (full inventory from rank-explore.mjs).
// Legacy rows and the churn fallback leave all of this inert: covered stays empty,
// uncoveredCount() is 0, and the loop condition reduces to the old dry-threshold rule.
const HAS_INVENTORY = GRAPH_ROWS.some((r) => 'examined' in r)
const INVENTORY = HAS_INVENTORY ? GRAPH_ROWS.map((r) => r.file) : []
const covered = new Set(HAS_INVENTORY ? GRAPH_ROWS.filter((r) => r.examined).map((r) => r.file) : [])
const PRE_COVERED = covered.size
const RISKY_FILES = HAS_INVENTORY ? GRAPH_ROWS.filter((r) => r.risky).map((r) => r.file) : []
const uncoveredCount = () => (HAS_INVENTORY ? INVENTORY.reduce((n, f) => n + (covered.has(f) ? 0 : 1), 0) : 0)
if (HAS_INVENTORY) log(`coverage: inventory=${INVENTORY.length} preCovered=${PRE_COVERED} risky=${RISKY_FILES.length}`)
const EXPLORE_FILES_PER_LENS = 10
const exploreConsumed = new Set() // within-run consumption: never re-offer a file to a later round
let exploreFallbackLogged = false
@@ -316,9 +341,18 @@ function drawExploreFiles() {
}
pool = churnFiles
}
const files = pool
.filter((f) => !exploreConsumed.has(f) && !seen.some((s) => s.file === f))
.slice(0, EXPLORE_FILES_PER_LENS)
const avail = pool.filter((f) => !exploreConsumed.has(f) && !covered.has(f) && !seen.some((s) => s.file === f))
const files = []
while (files.length < EXPLORE_FILES_PER_LENS && avail.length) {
const head = avail.shift()
files.push(head)
const dir = clusterOf(head)
// pull same-directory siblings forward: one lens reading one module beats ten strangers
for (let i = 0; i < avail.length && files.length < EXPLORE_FILES_PER_LENS; ) {
if (clusterOf(avail[i]) === dir) files.push(avail.splice(i, 1)[0])
else i++
}
}
for (const f of files) exploreConsumed.add(f)
return files
}
@@ -333,7 +367,12 @@ function exploreLens(i) {
`single finding in them - either they are clean or every lens so far walked past them.`
return {
key: `explore-${i}`,
prompt: `${src} Read each one IN FULL with fresh eyes and hunt for real bugs of any class:\n` +
prompt: `${src} Read each one IN FULL with fresh eyes. Hunt every class: concurrency and ` +
`interleaving (races, TOCTOU, lock ordering, stale-closure writes after await); lifecycle and ` +
`teardown (unreleased acquires, use-after-close, missing disposal on error paths); state desync ` +
`(two sources of truth updated by different code paths); error-path data loss (swallowed errors, ` +
`partial writes, silent fallbacks); ordering and boundaries (off-by-one, pagination truncation, ` +
`sequence gaps, inclusive/exclusive disagreements).\n` +
files.map((f) => ` - ${f}`).join('\n'),
files,
}
@@ -348,8 +387,10 @@ function buildAdaptiveLenses(round) {
byCluster[cl].push(c)
}
// Explore-heavy schedule: measured hotspot yield flattened to 0.25 high+med/agent by round 6.
const sweeping = uncoveredCount() > 0
const hotspotQuota = round <= 5 ? 2 : 1
const exploreQuota = round <= 5 ? 2 : 3
// sweep pace: 4 explore lenses x 10 files while inventory files remain uncovered
const exploreQuota = sweeping ? 4 : round <= 5 ? 2 : 3
const hotKey = (cl) => ('hotspot ' + cl).toLowerCase().replace(/[^a-z0-9]+/g, '-')
const picked = Object.entries(byCluster)
.sort((a, b) => b[1].length - a[1].length)
@@ -370,7 +411,7 @@ function buildAdaptiveLenses(round) {
if (shortfall > 0) log(`adaptive: hotspot pool short by ${shortfall} - trying explore backfill`)
const explores = []
for (let i = 1; i <= exploreQuota + shortfall; i++) {
if ((cleanStreak[`explore-${i}`] || 0) >= 2) continue // demoted slot: no substitution, that IS demotion
if (!sweeping && (cleanStreak[`explore-${i}`] || 0) >= 2) continue // demoted slot: no substitution, that IS demotion
const lens = exploreLens(i)
if (!lens) {
log(`adaptive: explore pool exhausted after ${explores.length} lens(es)`)
@@ -424,12 +465,14 @@ function seenBlock(seen) {
const lines = seen.map((s) => ` - ${s.file}:${s.line} [${s.status}] ${s.title}`)
return `\n--- KNOWN FINDINGS (already investigated - do NOT re-report; refuted means examined and rejected) ---\n${lines.join('\n')}\n`
}
function convergenceTable(stats, converged, stoppedOnBudget) {
function convergenceTable(stats, converged, stoppedOnBudget, stalled) {
const verdict = converged
? `CONVERGED after ${stats.length} round(s).`
: stoppedOnBudget
? 'NOT converged - stopped on budget.'
: 'NOT converged - hit the round backstop.'
: stalled
? 'NOT converged - coverage stalled.'
: stoppedOnBudget
? 'NOT converged - stopped on budget.'
: 'NOT converged - hit the round backstop.'
const rows = stats.map(
(s) =>
`| ${s.round} | ${s.family} | ${s.lenses} | ${s.candidates} | ${s.fresh} | ${s.confirmed} | ${s.refuted} | ${s.dryEligible ? 'yes' : 'NO'} | ${s.dryAfter} |`,
@@ -494,6 +537,8 @@ const cleanStreak = {}
let dry = 0
let round = 0
let stoppedOnBudget = false
let coverageStall = 0
let stalledCoverage = false
function finderPrompt(lens, rnd) {
return (
@@ -524,18 +569,35 @@ function verifyPrompt(lensKey, candidates) {
)
}
while (dry < DRY_THRESHOLD && round < MAX_ROUNDS) {
const riskySweepPending = () => HAS_INVENTORY && RISKY_FILES.length > 0 && !riskySweepDone
while ((uncoveredCount() > 0 || riskySweepPending() || dry < DRY_THRESHOLD) && round < MAX_ROUNDS) {
if (BUDGET_TOTAL && remainingBudget() < ROUND_BUDGET_FLOOR) {
stoppedOnBudget = true
log(`Budget floor reached (${Math.round(remainingBudget() / 1000)}k left) - stopping before round ${round + 1}`)
break
}
const family = lensesForRound(round + 1)
if (!family || !family.length) break // nothing to hunt != everything demoted
if (!family || !family.length) {
// Coverage mode with the pool drained and the risky sweep done: an empty family means
// hotspots are demoted/cooled and there is genuinely nothing left to hunt - that IS
// quietness. Count it as a dry round so a late confirm cannot strand a fully-covered
// run one dry round short of its earned convergence. Legacy mode keeps the hard stop:
// an empty family there means "nothing targetable" (no churn, no graph), not "done".
if (HAS_INVENTORY && uncoveredCount() === 0 && !riskySweepPending()) {
round++
dry++
roundStats.push({ round, family: 'exhausted', lenses: 0, candidates: 0, fresh: 0, confirmed: 0, refuted: 0, dryEligible: true, dryAfter: dry, severity: { critical: 0, high: 0, medium: 0, low: 0 }, perLens: {}, filesTouched: 0, filesNew: 0, suppressedLedger: 0, suppressedRun: 0, finderNull: 0, finderEmpty: 0, verifierNull: 0, spentBefore: budget.spent(), spentAfter: budget.spent() })
log(`Round ${round}: nothing left to hunt - counts as a dry round (dry=${dry})`)
continue
}
break // nothing to hunt != everything demoted
}
round++
const uncBefore = uncoveredCount()
const spentBefore = budget.spent()
const counts = { suppressedLedger: 0, suppressedRun: 0, finderNull: 0, finderEmpty: 0, verifierNull: 0 }
const lenses = family.filter((l) => (cleanStreak[l.key] || 0) < 2)
const sweepingNow = uncoveredCount() > 0
const lenses = family.filter((l) => (sweepingNow && /^explore-/.test(l.key)) || (cleanStreak[l.key] || 0) < 2)
if (!lenses.length) {
dry++
roundStats.push({ round, family: familyName(round), lenses: 0, candidates: 0, fresh: 0, confirmed: 0, refuted: 0, dryEligible: true, dryAfter: dry, severity: { critical: 0, high: 0, medium: 0, low: 0 }, perLens: {}, filesTouched: 0, filesNew: 0, ...counts, spentBefore, spentAfter: budget.spent() })
@@ -613,10 +675,13 @@ while (dry < DRY_THRESHOLD && round < MAX_ROUNDS) {
candCount += r.unionCount
freshCount += r.fresh.length
if (r.finderFailed) eligible = false
if (r.finderFailed && r.lens.files) {
// a dead finder read nothing: un-consume its draw so later rounds can re-offer the
// files and the session does not record never-examined files as explored-clean
for (const f of r.lens.files) exploreConsumed.delete(f)
if (r.lens.files) {
// coverage credit (spec: only explicit-file lenses that ran to completion). A lens
// denied credit - dead finder OR candidates left unverified - returns its whole draw
// to the pool: consumed-but-uncovered files would otherwise strand uncoveredCount()
// above zero forever, and a partially-verified draw is not evidence of cleanliness.
if (!r.finderFailed && !r.unmatched.length) for (const f of r.lens.files) covered.add(f)
else for (const f of r.lens.files) exploreConsumed.delete(f)
}
let lensConfirmed = 0
let lensRefuted = 0
@@ -627,6 +692,7 @@ while (dry < DRY_THRESHOLD && round < MAX_ROUNDS) {
const rec = { file: v.file, line: v.line, title: v.title, status: v.refuted ? 'refuted' : 'confirmed' }
if (seen.some((p) => isDup(rec, p))) { counts.suppressedRun++; continue } // cross-lens same-round duplicate
seen.push(rec)
covered.add(rec.file) // any verdict proves the file was read (inert in legacy mode: seen already blocks re-draws)
if (v.refuted) { newRefuted++; lensRefuted++ }
else {
newConfirmed++
@@ -654,9 +720,24 @@ while (dry < DRY_THRESHOLD && round < MAX_ROUNDS) {
// ineligible zero-confirm round: dry unchanged - "we didn't fully look" is not "it's clean"
roundStats.push({ round, family: familyName(round), lenses: lenses.length, candidates: candCount, fresh: freshCount, confirmed: newConfirmed, refuted: newRefuted, dryEligible: eligible, dryAfter: dry, severity: sevMix, perLens, filesTouched: filesTouched.size, filesNew: filesNew.size, ...counts, spentBefore, spentAfter: budget.spent() })
log(`Round ${round} (${familyName(round)}): ${newConfirmed} confirmed, ${newRefuted} refuted, dry=${dry}${eligible ? '' : ' (ineligible)'}`)
// stalled coverage: an adaptive round that failed to shrink a non-empty uncovered pool
// AND confirmed nothing. Only adaptive rounds count - rounds 1-3 never draw explore
// files by design - and a round that confirmed a bug is never a stall: hotspot yield
// does not shrink the pool, and cutting off a still-productive hunt is the one thing
// a bug-finding tool must not do.
const uncAfter = uncoveredCount()
if (HAS_INVENTORY && familyName() === 'adaptive' && uncAfter > 0 && newConfirmed === 0) {
coverageStall = uncAfter < uncBefore ? 0 : coverageStall + 1
if (coverageStall >= 2) {
stalledCoverage = true
log(`Coverage stalled: uncovered=${uncAfter} did not shrink for 2 adaptive rounds - stopping`)
break
}
} else coverageStall = 0
}
const converged = dry >= DRY_THRESHOLD
const converged = uncoveredCount() === 0 && !riskySweepPending() && dry >= DRY_THRESHOLD
// ---------- report (deterministic) ----------
// A report agent silently dropped findings (79 sections for 82 confirmed on 2026-08-12), so the
@@ -665,14 +746,16 @@ const converged = dry >= DRY_THRESHOLD
const RANK = { critical: 0, high: 1, medium: 2, low: 3 }
const confirmedSorted = confirmedAll.slice().sort((a, b) => RANK[a.severity] - RANK[b.severity])
const unverifiedFinal = unverified.filter((u) => !seen.some((p) => isDup(u, p)))
const table = convergenceTable(roundStats, converged, stoppedOnBudget)
const table = convergenceTable(roundStats, converged, stoppedOnBudget, stalledCoverage)
const sum = (k) => roundStats.reduce((n, r) => n + (r[k] || 0), 0)
const runStats = {
config: { maxRounds: MAX_ROUNDS, dryThreshold: DRY_THRESHOLD, customLenses: !!CUSTOM_LENSES, knownCount: (ARGS.known || []).length, graphRows: GRAPH_ROWS.length, budgetTotal: BUDGET_TOTAL },
coverage: HAS_INVENTORY ? { inventory: INVENTORY.length, preCovered: PRE_COVERED, covered: INVENTORY.length - uncoveredCount(), uncoveredAtStop: uncoveredCount() } : null,
spentTotal: budget.spent(),
rounds: roundStats.length,
converged,
stoppedOnBudget,
stalledCoverage,
confirmed: confirmedSorted.length,
refuted: sum('refuted'),
unverified: unverifiedFinal.length,
@@ -686,9 +769,11 @@ const runStats = {
function buildReport() {
const outcome = converged
? `CONVERGED after ${round} round(s).`
: stoppedOnBudget
? `NOT converged - stopped on budget after ${round} round(s).`
: `NOT converged - hit the round backstop after ${round} round(s).`
: stalledCoverage
? `NOT converged - coverage stalled after ${round} round(s).`
: stoppedOnBudget
? `NOT converged - stopped on budget after ${round} round(s).`
: `NOT converged - hit the round backstop after ${round} round(s).`
const sev = { critical: 0, high: 0, medium: 0, low: 0 }
for (const f of confirmedSorted) sev[f.severity] = (sev[f.severity] || 0) + 1
const lines = ['# Bug hunt report', '']
@@ -721,6 +806,13 @@ function buildReport() {
`Agent failures: ${runStats.finderNull} finder null, ${runStats.finderEmpty} finder empty, ${runStats.verifierNull} verifier null.`,
'',
)
if (runStats.coverage)
lines.push(
`Coverage: ${runStats.coverage.covered}/${runStats.coverage.inventory} files ` +
`(${runStats.coverage.preCovered} pre-covered from ledger + live explored-clean); ` +
`${runStats.coverage.uncoveredAtStop} uncovered at stop.`,
'',
)
lines.push('| round | spent | files (new) | suppressed ledger/run | finder null/empty | verifier null |')
lines.push('|---|---|---|---|---|---|')
for (const s of roundStats)
@@ -729,4 +821,4 @@ function buildReport() {
}
const report = buildReport()
return { converged, stoppedOnBudget, rounds: roundStats, confirmed: confirmedSorted, unverified: unverifiedFinal, runStats, exploredFiles: [...exploreConsumed], report }
return { converged, stoppedOnBudget, stalledCoverage, rounds: roundStats, confirmed: confirmedSorted, unverified: unverifiedFinal, runStats, exploredFiles: [...exploreConsumed], report }