mirror of
https://github.com/J3vb/OwnCord.git
synced 2026-09-03 03:50:00 +03:00
feat(bughunt): coverage-driven convergence (#1399)
* feat(bughunt): coverage-driven stop rule and directory-coherent sweep Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bughunt): return uncredited explore draws to the pool An explore lens denied coverage credit (dead finder or unverified candidates) now un-consumes its draw so later rounds re-offer the files; consumed-but-uncovered files could otherwise pin uncoveredCount above zero and block convergence. Directory grouping reuses clusterOf(). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(bughunt): stalled-coverage guard, risky-file class sweep, exhausted-dry convergence Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bughunt): stall guard never stops a still-confirming hunt A round with newConfirmed > 0 resets the coverage-stall counter instead of counting toward it; hotspot yield does not shrink the uncovered pool, and a stuck sweep must not cut off a hunt that is still finding bugs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(bughunt): coverage telemetry in report and operator docs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(bughunt): scoped-hunt coverage trap and current cost estimate Final-review fixes: warn that args.lenses plus an examined-armed inventory still sweeps the whole pool (pass a filtered inventory or legacy rows to truly scope), and align the budget note with the coverage-run estimate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -16,7 +16,7 @@ reach a commit, an issue, or a PR body.
|
|||||||
## 1. Hunt
|
## 1. Hunt
|
||||||
|
|
||||||
**Launch the hunt from a turn that carries a token-budget directive** (recommended:
|
**Launch the hunt from a turn that carries a token-budget directive** (recommended:
|
||||||
`+25M`, comfortably above a full 8-round run). The workflow's cost ceiling is gated
|
`+25M`, comfortably above a full coverage run's ~8-12M). The workflow's cost ceiling is gated
|
||||||
on `budget.total`, which is null without a directive — a directive-less run has **no
|
on `budget.total`, which is null without a directive — a directive-less run has **no
|
||||||
ceiling at all**. The workflow's first log line echoes the state: `budget=25M` means
|
ceiling at all**. The workflow's first log line echoes the state: `budget=25M` means
|
||||||
armed; `budget=NONE - cost ceiling disarmed` means stop the run and relaunch with a
|
armed; `budget=NONE - cost ceiling disarmed` means stop the run and relaunch with a
|
||||||
@@ -26,9 +26,19 @@ Before launching, in order:
|
|||||||
|
|
||||||
1. **Rebuild the graph** (stale coordinates aim the explore lens at moved code):
|
1. **Rebuild the graph** (stale coordinates aim the explore lens at moved code):
|
||||||
`graphify update . --no-cluster` — local tree-sitter, zero LLM cost, ~10.7k nodes.
|
`graphify update . --no-cluster` — local tree-sitter, zero LLM cost, ~10.7k nodes.
|
||||||
2. **Rank explore targets**: `node .superpowers/rank-explore.mjs` — writes
|
2. **Build the inventory**: `node .superpowers/rank-explore.mjs` — writes
|
||||||
`.superpowers/explore-ranking.json`, deprioritizing files recorded clean in
|
`.superpowers/explore-ranking.json`: EVERY non-test source file (~419 rows), each with
|
||||||
`.superpowers/explored-clean.json` and dropping files that no longer exist.
|
`examined` (already carries a ledger finding or a LIVE explored-clean record → the hunt
|
||||||
|
pre-seeds its covered set), `risky` (top coupling ∪ past-bug clusters ∪ top churn,
|
||||||
|
capped at 40 → they get an extra pass through all 5 bug-class lenses), and `churn`.
|
||||||
|
Explored-clean records carry content hashes: editing a file expires its clean record,
|
||||||
|
so re-runs automatically re-hunt what changed. The hunt cannot stop while any inventory
|
||||||
|
file is uncovered, so a full run now takes ~10-20 rounds and ~8-12M tokens — the `+25M`
|
||||||
|
directive still covers it. Regenerate the inventory and read `known` from the ledger in
|
||||||
|
the SAME session step: both derive from `findings-ledger.json`, and every `known` file
|
||||||
|
must be `examined` in the inventory — a `known` file the inventory does not mark
|
||||||
|
examined can never be drawn (the seen-filter blocks it) nor covered, which would
|
||||||
|
strand `uncoveredCount()` above zero and block convergence.
|
||||||
3. Read the ledger and pass every record in as `known`, so the hunt does not
|
3. Read the ledger and pass every record in as `known`, so the hunt does not
|
||||||
re-derive anything already found, fixed, declined, or refuted.
|
re-derive anything already found, fixed, declined, or refuted.
|
||||||
|
|
||||||
@@ -39,7 +49,7 @@ Workflow({
|
|||||||
known: <every record from findings-ledger.json, as {file, line, title, status}>,
|
known: <every record from findings-ledger.json, as {file, line, title, status}>,
|
||||||
graph: <the rows of .superpowers/explore-ranking.json>,
|
graph: <the rows of .superpowers/explore-ranking.json>,
|
||||||
lenses: [ {key, prompt}, ... ], // optional: scope the hunt to one subsystem
|
lenses: [ {key, prompt}, ... ], // optional: scope the hunt to one subsystem
|
||||||
maxRounds: 8,
|
maxRounds: 30, // safety backstop only - coverage + dry is the real stop
|
||||||
dryThreshold: 2,
|
dryThreshold: 2,
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
@@ -49,8 +59,28 @@ If `graph` is omitted or empty the hunt logs
|
|||||||
`explore: args.graph absent/empty - falling back to churn-based fresh eyes` and
|
`explore: args.graph absent/empty - falling back to churn-based fresh eyes` and
|
||||||
still runs — degraded targeting, never a smaller lens family.
|
still runs — degraded targeting, never a smaller lens family.
|
||||||
|
|
||||||
|
`converged: true` now means: every inventory file was covered by a completed
|
||||||
|
explicit-file lens (or carries a verdict), the risky class sweep ran, and then
|
||||||
|
`dryThreshold` consecutive eligible rounds confirmed nothing (a round where the
|
||||||
|
lens family comes up empty with the pool drained counts as dry — family
|
||||||
|
`exhausted`). Rows without the `examined` field fall back to the old
|
||||||
|
quietness-only stop. Two new run outcomes: `stalledCoverage: true` means adaptive
|
||||||
|
rounds stopped shrinking the uncovered pool (usually mass finder failures —
|
||||||
|
investigate before re-running); a budget stop now reports
|
||||||
|
`coverage.uncoveredAtStop` so the next run knows exactly what remains (re-run
|
||||||
|
with the ledger as `known`; live explored-clean records pre-cover what was
|
||||||
|
finished, so the sweep naturally continues where it stopped).
|
||||||
|
|
||||||
Omit `lenses` for a general hunt across the rotating families.
|
Omit `lenses` for a general hunt across the rotating families.
|
||||||
|
|
||||||
|
**Scoping a hunt while coverage mode is armed is a budget trap:** `lenses` only
|
||||||
|
replaces round 1, and inventory rows with `examined` force the coverage stop
|
||||||
|
rule — from round 2 the run sweeps the ENTIRE uncovered pool and the risky
|
||||||
|
sweep before it may converge, at general-hunt cost. For a true scoped hunt,
|
||||||
|
pass a subsystem-filtered inventory as `graph` (only the rows you want swept),
|
||||||
|
or rows without the `examined` field to fall back to the legacy quietness-only
|
||||||
|
stop.
|
||||||
|
|
||||||
Each lens object is `{key, prompt}`. `key` must match `^[a-z0-9-]+$` —
|
Each lens object is `{key, prompt}`. `key` must match `^[a-z0-9-]+$` —
|
||||||
lowercase letters, digits, and hyphens only. Keys get interpolated into agent
|
lowercase letters, digits, and hyphens only. Keys get interpolated into agent
|
||||||
labels of the form `r<N>:hunt:<key>:<model>`, and a key containing uppercase,
|
labels of the form `r<N>:hunt:<key>:<model>`, and a key containing uppercase,
|
||||||
|
|||||||
@@ -73,6 +73,11 @@ export const finding = (n, over = {}) => ({
|
|||||||
})
|
})
|
||||||
export const graphRows = (n) =>
|
export const graphRows = (n) =>
|
||||||
Array.from({ length: n }, (_, i) => ({ file: `Server/gen/g${i}.go`, score: 1 - i / (n + 1), degree: 10, cited: 5 }))
|
Array.from({ length: n }, (_, i) => ({ file: `Server/gen/g${i}.go`, score: 1 - i / (n + 1), degree: 10, cited: 5 }))
|
||||||
|
export const inventoryRows = (n, over = () => ({})) =>
|
||||||
|
Array.from({ length: n }, (_, i) => ({
|
||||||
|
file: `Server/gen/g${i}.go`, degree: 10, cited: 5, score: 1 - i / (n + 1),
|
||||||
|
examined: false, risky: false, ...over(i),
|
||||||
|
}))
|
||||||
export const confirmAll = (cands) => ({
|
export const confirmAll = (cands) => ({
|
||||||
verdicts: cands.map((c) => ({
|
verdicts: cands.map((c) => ({
|
||||||
title: c.title, file: c.file, line: c.line,
|
title: c.title, file: c.file, line: c.line,
|
||||||
@@ -845,6 +850,193 @@ scenarios.s_explore_rewind_on_thrown_stage = async () => {
|
|||||||
assert.equal(result.rounds[3].dryEligible, false, 'a nulled lens result still makes the round ineligible')
|
assert.equal(result.rounds[3].dryEligible, false, 'a nulled lens result still makes the round ineligible')
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// COV1 (spec §3): coverage mode may NOT stop on quietness while inventory files are uncovered.
|
||||||
|
// 60 rows, 10 pre-examined -> 50 to sweep. Nothing is ever found, so dry passes the threshold
|
||||||
|
// at round 2 - the old stop rule would have converged there. The new rule keeps going until
|
||||||
|
// round 4's explore lenses (quota 4 + backfill 2 slots; 5 draw files, the 6th comes up empty)
|
||||||
|
// cover all 50, then exits. Also locks the enriched explore prompt (class checklist).
|
||||||
|
scenarios.s_coverage_blocks_stop = async () => {
|
||||||
|
const inv = inventoryRows(60, (i) => (i >= 50 ? { examined: true } : {}))
|
||||||
|
const { result, calls } = await run({
|
||||||
|
args: { graph: inv },
|
||||||
|
agentStub: makeStub({ hunt: () => none, verify: (r, k, c) => confirmAll(c) }),
|
||||||
|
})
|
||||||
|
assert.equal(result.rounds.length, 4, 'must run past the dry threshold (hit at r2) to sweep in r4')
|
||||||
|
assert.equal(result.converged, true)
|
||||||
|
assert.deepEqual(result.rounds.map((r) => r.dryAfter), [1, 2, 3, 4])
|
||||||
|
assert.deepEqual(result.runStats.coverage, { inventory: 60, preCovered: 10, covered: 60, uncoveredAtStop: 0 })
|
||||||
|
assert.match(result.report, /CONVERGED/)
|
||||||
|
assert.match(result.report, /Coverage: 60\/60 files \(10 pre-covered/)
|
||||||
|
const ep = (calls.find((c) => (c.opts.label || '') === 'r4:hunt:explore-1:opus') || {}).prompt || ''
|
||||||
|
assert.match(ep, /error-path data loss/, 'explore lenses carry the distilled class checklist')
|
||||||
|
}
|
||||||
|
|
||||||
|
// COV2 (spec §2+§4): a dead explore finder's files stay uncovered and get re-offered; the
|
||||||
|
// run only converges after a LIVE lens covers them.
|
||||||
|
scenarios.s_coverage_dead_finder = async () => {
|
||||||
|
const inv = inventoryRows(20)
|
||||||
|
const { result, calls } = await run({
|
||||||
|
args: { graph: inv },
|
||||||
|
agentStub: makeStub({
|
||||||
|
hunt: (round, key) => (round === 4 && key === 'explore-1' ? null : none),
|
||||||
|
verify: (r, k, c) => confirmAll(c),
|
||||||
|
}),
|
||||||
|
})
|
||||||
|
const promptOf = (rnd, key) => (calls.find((c) => (c.opts.label || '') === `r${rnd}:hunt:${key}:opus`) || {}).prompt || ''
|
||||||
|
assert.match(promptOf(4, 'explore-1'), /Server\/gen\/g0\.go/, 'r4 explore-1 drew the head of the pool')
|
||||||
|
assert.match(promptOf(5, 'explore-1'), /Server\/gen\/g0\.go/, 'dead lens files are re-offered next round')
|
||||||
|
assert.equal(result.rounds[3].dryEligible, false, 'dead finder keeps the round ineligible')
|
||||||
|
assert.equal(result.converged, true)
|
||||||
|
assert.deepEqual(result.runStats.coverage, { inventory: 20, preCovered: 0, covered: 20, uncoveredAtStop: 0 })
|
||||||
|
}
|
||||||
|
|
||||||
|
// COV7 (amendment 4): explore draws are directory-coherent - one lens reads one module,
|
||||||
|
// not ten strangers. Cross-file classes (state desync, acquire/release pairs) need siblings
|
||||||
|
// in one agent's context.
|
||||||
|
scenarios.s_directory_coherent_draws = async () => {
|
||||||
|
const inv = inventoryRows(20, (i) => ({ file: i % 2 === 0 ? `Server/alpha/a${i}.go` : `Server/beta/b${i}.go` }))
|
||||||
|
const { calls } = await run({
|
||||||
|
args: { graph: inv },
|
||||||
|
agentStub: makeStub({ hunt: () => none, verify: (r, k, c) => confirmAll(c) }),
|
||||||
|
})
|
||||||
|
const p1 = (calls.find((c) => (c.opts.label || '') === 'r4:hunt:explore-1:opus') || {}).prompt || ''
|
||||||
|
assert.match(p1, /Server\/alpha\/a0\.go/)
|
||||||
|
assert.match(p1, /Server\/alpha\/a18\.go/, 'all ten alpha files ride in the first lens')
|
||||||
|
assert.doesNotMatch(p1, /Server\/beta\//, 'no stranger directories in a coherent draw')
|
||||||
|
}
|
||||||
|
|
||||||
|
// COV8 (Task 5 review finding): an explore lens whose candidates never get a verdict is
|
||||||
|
// denied coverage credit - its draw must return to the pool and be re-offered, or the
|
||||||
|
// consumed-but-uncovered files strand uncoveredCount() above zero and the run can never
|
||||||
|
// converge (it would grind to the round backstop instead).
|
||||||
|
scenarios.s_uncredited_draw_returns_to_pool = async () => {
|
||||||
|
const inv = inventoryRows(20)
|
||||||
|
const { result, calls } = await run({
|
||||||
|
args: { graph: inv },
|
||||||
|
agentStub: makeStub({
|
||||||
|
hunt: (round, key) =>
|
||||||
|
round === 4 && key === 'explore-1'
|
||||||
|
? { findings: [finding(1, { file: 'Server/gen/g0.go', title: 'orphaned candidate one' })] }
|
||||||
|
: none,
|
||||||
|
verify: (round, key, cands) =>
|
||||||
|
round === 4 && key === 'explore-1' ? { verdicts: [] } : confirmAll(cands),
|
||||||
|
}),
|
||||||
|
})
|
||||||
|
const promptOf = (rnd, key) => (calls.find((c) => (c.opts.label || '') === `r${rnd}:hunt:${key}:opus`) || {}).prompt || ''
|
||||||
|
assert.match(promptOf(4, 'explore-1'), /Server\/gen\/g0\.go/)
|
||||||
|
assert.match(promptOf(5, 'explore-1'), /Server\/gen\/g0\.go/, 'uncredited draw is re-offered next round')
|
||||||
|
assert.equal(result.rounds[3].dryEligible, false, 'unverified candidates keep the round ineligible')
|
||||||
|
assert.equal(result.converged, true, 'the run must still converge once a later lens covers the files')
|
||||||
|
assert.deepEqual(result.runStats.coverage, { inventory: 20, preCovered: 0, covered: 20, uncoveredAtStop: 0 })
|
||||||
|
assert.equal(result.unverified.length, 1, 'the orphaned candidate stays reported unverified')
|
||||||
|
}
|
||||||
|
|
||||||
|
// COV3 (spec §4): finders dying every adaptive round -> uncovered never shrinks -> stop with
|
||||||
|
// stalledCoverage after 2 stalled adaptive rounds instead of burning 26 more rounds.
|
||||||
|
scenarios.s_coverage_stall = async () => {
|
||||||
|
const inv = inventoryRows(20)
|
||||||
|
const { result, logs } = await run({
|
||||||
|
args: { graph: inv },
|
||||||
|
agentStub: makeStub({
|
||||||
|
hunt: (round, key) => (/^explore-/.test(key) ? null : none),
|
||||||
|
verify: (r, k, c) => confirmAll(c),
|
||||||
|
}),
|
||||||
|
})
|
||||||
|
assert.equal(result.stalledCoverage, true)
|
||||||
|
assert.equal(result.converged, false)
|
||||||
|
assert.equal(result.rounds.length, 5, 'r1-3 families, then exactly 2 stalled adaptive rounds')
|
||||||
|
assert.equal(result.runStats.coverage.uncoveredAtStop, 20)
|
||||||
|
assert.ok(logs.some((l) => /Coverage stalled/.test(l)))
|
||||||
|
assert.match(result.report, /NOT converged - coverage stalled/)
|
||||||
|
}
|
||||||
|
|
||||||
|
// COV4 (spec §2 smart depth): once the sweep completes, the 5 bug-class lenses run once more
|
||||||
|
// scoped to the risky files - then the run may converge.
|
||||||
|
scenarios.s_risky_sweep = async () => {
|
||||||
|
const inv = inventoryRows(10, (i) => (i < 2 ? { risky: true } : {}))
|
||||||
|
const { result, calls } = await run({
|
||||||
|
args: { graph: inv },
|
||||||
|
agentStub: makeStub({ hunt: () => none, verify: (r, k, c) => confirmAll(c) }),
|
||||||
|
})
|
||||||
|
const r5keys = [...new Set(calls.filter((c) => /^r5:hunt:/.test(c.opts.label || '')).map((c) => c.opts.label.split(':')[2]))]
|
||||||
|
assert.deepEqual(r5keys.sort(), ['risky-concurrency', 'risky-error-paths', 'risky-lifecycle', 'risky-ordering-boundary', 'risky-state-desync'])
|
||||||
|
const p = calls.find((c) => (c.opts.label || '') === 'r5:hunt:risky-concurrency:opus').prompt
|
||||||
|
assert.match(p, /Server\/gen\/g0\.go/)
|
||||||
|
assert.match(p, /Server\/gen\/g1\.go/)
|
||||||
|
assert.doesNotMatch(p, /Server\/gen\/g5\.go/, 'the sweep is scoped to risky files only')
|
||||||
|
assert.equal(result.rounds[4].family, 'risky-sweep')
|
||||||
|
assert.equal(result.converged, true)
|
||||||
|
assert.equal(result.rounds.length, 5, 'dry was already past threshold - the run ends right after the risky sweep')
|
||||||
|
assert.ok(!calls.some((c) => /^r6:hunt:risky-/.test(c.opts.label || '')), 'the risky sweep runs exactly once')
|
||||||
|
}
|
||||||
|
|
||||||
|
// COV5: legacy rows (no `examined` key) leave every new mechanism inert - old stop rule,
|
||||||
|
// no coverage stats, no risky sweep.
|
||||||
|
scenarios.s_legacy_rows_inert = async () => {
|
||||||
|
const { result, calls } = await run({
|
||||||
|
args: { graph: graphRows(30) },
|
||||||
|
agentStub: makeStub({ hunt: () => none, verify: (r, k, c) => confirmAll(c) }),
|
||||||
|
})
|
||||||
|
assert.equal(result.rounds.length, 2, 'legacy mode still converges on the plain dry threshold')
|
||||||
|
assert.equal(result.converged, true)
|
||||||
|
assert.equal(result.runStats.coverage, null)
|
||||||
|
assert.equal(result.stalledCoverage, false)
|
||||||
|
assert.ok(!calls.some((c) => /risky-/.test(c.opts.label || '')))
|
||||||
|
}
|
||||||
|
|
||||||
|
// COV6 (amendment 1): a confirm on the sweep's last round resets dry to 0; hotspot rounds then
|
||||||
|
// go clean, cooldown + an empty pool empty the family - in coverage mode that emptiness IS
|
||||||
|
// quietness and must count dry rounds instead of stranding a fully-covered run at converged:false.
|
||||||
|
scenarios.s_exhausted_counts_dry = async () => {
|
||||||
|
const inv = inventoryRows(10)
|
||||||
|
const { result } = await run({
|
||||||
|
args: { graph: inv },
|
||||||
|
agentStub: makeStub({
|
||||||
|
hunt: (round, key) =>
|
||||||
|
round === 4 && key === 'explore-1'
|
||||||
|
? { findings: [finding(1, { file: 'Server/gen/g0.go', title: 'late sweep bug one' })] }
|
||||||
|
: none,
|
||||||
|
verify: (r, k, c) => confirmAll(c),
|
||||||
|
}),
|
||||||
|
})
|
||||||
|
assert.equal(result.converged, true, 'a fully-covered, fully-quiet run must converge')
|
||||||
|
assert.equal(result.rounds.length, 6)
|
||||||
|
assert.equal(result.rounds[5].family, 'exhausted')
|
||||||
|
assert.equal(result.rounds[5].dryAfter, 2)
|
||||||
|
assert.equal(result.confirmed.length, 1)
|
||||||
|
}
|
||||||
|
|
||||||
|
// COV9 (Task 6 review finding): a stuck pool must NOT stop a hunt that is still confirming.
|
||||||
|
// Dead explore lenses pin uncovered at 20 while hotspot lenses confirm fresh bugs in rounds
|
||||||
|
// 4 and 6 - each productive round resets the stall counter, so the run survives to round 8
|
||||||
|
// and only then stops with stalledCoverage (without the productivity term it would have
|
||||||
|
// stopped at round 5, mid-yield).
|
||||||
|
scenarios.s_stall_deferred_while_productive = async () => {
|
||||||
|
const inv = inventoryRows(20)
|
||||||
|
const { result, logs } = await run({
|
||||||
|
args: { graph: inv },
|
||||||
|
agentStub: makeStub({
|
||||||
|
hunt: (round, key) => {
|
||||||
|
if (round === 1 && key === 'ws-hub')
|
||||||
|
return { findings: [finding(1, { file: 'Server/ws/hub.go', title: 'seed bug alpha one' })] }
|
||||||
|
if (round === 4 && key === 'hotspot-server-ws')
|
||||||
|
return { findings: [finding(2, { file: 'Server/ws/emit.go', title: 'adjacent bug beta two' })] }
|
||||||
|
if (round === 6 && key === 'hotspot-server-ws')
|
||||||
|
return { findings: [finding(3, { file: 'Server/ws/pubsub.go', title: 'adjacent bug gamma three' })] }
|
||||||
|
if (/^explore-/.test(key)) return null // dead explore finders: the pool never shrinks
|
||||||
|
return none
|
||||||
|
},
|
||||||
|
verify: (r, k, c) => confirmAll(c),
|
||||||
|
}),
|
||||||
|
})
|
||||||
|
assert.equal(result.rounds.length, 8, 'productive rounds 4 and 6 must defer the stall to round 8')
|
||||||
|
assert.equal(result.stalledCoverage, true)
|
||||||
|
assert.equal(result.converged, false)
|
||||||
|
assert.equal(result.confirmed.length, 3)
|
||||||
|
assert.equal(result.runStats.coverage.uncoveredAtStop, 20)
|
||||||
|
assert.ok(logs.some((l) => /Coverage stalled/.test(l)))
|
||||||
|
}
|
||||||
|
|
||||||
// ---------- runner ----------
|
// ---------- runner ----------
|
||||||
const only = process.argv[2]
|
const only = process.argv[2]
|
||||||
for (const [name, fn] of Object.entries(scenarios)) {
|
for (const [name, fn] of Object.entries(scenarios)) {
|
||||||
|
|||||||
+125
-33
@@ -15,7 +15,7 @@ const ARGS = (() => {
|
|||||||
}
|
}
|
||||||
return args || {}
|
return args || {}
|
||||||
})()
|
})()
|
||||||
const MAX_ROUNDS = ARGS.maxRounds || 8
|
const MAX_ROUNDS = ARGS.maxRounds || 30
|
||||||
const DRY_THRESHOLD = ARGS.dryThreshold || 2
|
const DRY_THRESHOLD = ARGS.dryThreshold || 2
|
||||||
// A scoped hunt (args.lenses) replaces the round-1 family outright; later rounds still go
|
// A scoped hunt (args.lenses) replaces the round-1 family outright; later rounds still go
|
||||||
// adaptive, so hotspot and explore coverage - and therefore convergence - still work.
|
// adaptive, so hotspot and explore coverage - and therefore convergence - still work.
|
||||||
@@ -281,17 +281,31 @@ const FLOW_LENSES = [
|
|||||||
},
|
},
|
||||||
]
|
]
|
||||||
|
|
||||||
|
let riskySweepDone = false
|
||||||
|
function riskySweepLenses() {
|
||||||
|
if (riskySweepDone || !HAS_INVENTORY || !RISKY_FILES.length || uncoveredCount() > 0) return null
|
||||||
|
riskySweepDone = true // consumed even if this round's finders die: same at-most-once semantics as cooldown
|
||||||
|
const list = RISKY_FILES.map((f) => ` - ${f}`).join('\n')
|
||||||
|
return BUGCLASS_LENSES.map((l) => ({
|
||||||
|
key: `risky-${l.key}`,
|
||||||
|
prompt: `${l.prompt}\n\nScope this sweep to ONLY these highest-risk files (read each one in full):\n${list}`,
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
let currentFamilyName = 'surfaces'
|
||||||
function lensesForRound(round) {
|
function lensesForRound(round) {
|
||||||
if (CUSTOM_LENSES) return round === 1 ? CUSTOM_LENSES : buildAdaptiveLenses(round)
|
const pick = (name, lenses) => { currentFamilyName = name; return lenses }
|
||||||
if (round === 1) return SURFACE_LENSES
|
if (CUSTOM_LENSES) {
|
||||||
if (round === 2) return BUGCLASS_LENSES
|
if (round === 1) return pick('custom', CUSTOM_LENSES)
|
||||||
if (round === 3) return FLOW_LENSES
|
} else {
|
||||||
return buildAdaptiveLenses(round)
|
if (round === 1) return pick('surfaces', SURFACE_LENSES)
|
||||||
}
|
if (round === 2) return pick('bug-classes', BUGCLASS_LENSES)
|
||||||
function familyName(round) {
|
if (round === 3) return pick('flows', FLOW_LENSES)
|
||||||
if (CUSTOM_LENSES) return round === 1 ? 'custom' : 'adaptive'
|
}
|
||||||
return ['surfaces', 'bug-classes', 'flows'][round - 1] || 'adaptive'
|
const risky = riskySweepLenses()
|
||||||
|
if (risky) return pick('risky-sweep', risky)
|
||||||
|
return pick('adaptive', buildAdaptiveLenses(round))
|
||||||
}
|
}
|
||||||
|
function familyName() { return currentFamilyName }
|
||||||
// Directory granularity: the old two/three-segment cluster collapsed the whole TS client into
|
// Directory granularity: the old two/three-segment cluster collapsed the whole TS client into
|
||||||
// one bucket (35 of 82 findings), so the "top cluster" never changed for five straight rounds.
|
// one bucket (35 of 82 findings), so the "top cluster" never changed for five straight rounds.
|
||||||
function clusterOf(file) {
|
function clusterOf(file) {
|
||||||
@@ -303,6 +317,17 @@ function clusterOf(file) {
|
|||||||
// args.graph: session-computed coupling ranking (rank-explore.mjs). The workflow only reads
|
// args.graph: session-computed coupling ranking (rank-explore.mjs). The workflow only reads
|
||||||
// .file - scoring already happened outside, where the filesystem is.
|
// .file - scoring already happened outside, where the filesystem is.
|
||||||
const GRAPH_ROWS = (Array.isArray(ARGS.graph) ? ARGS.graph : []).filter((r) => r && typeof r.file === 'string')
|
const GRAPH_ROWS = (Array.isArray(ARGS.graph) ? ARGS.graph : []).filter((r) => r && typeof r.file === 'string')
|
||||||
|
// ---------- coverage mode (spec 2026-08-20) ----------
|
||||||
|
// Arms only when rows carry the `examined` flag (full inventory from rank-explore.mjs).
|
||||||
|
// Legacy rows and the churn fallback leave all of this inert: covered stays empty,
|
||||||
|
// uncoveredCount() is 0, and the loop condition reduces to the old dry-threshold rule.
|
||||||
|
const HAS_INVENTORY = GRAPH_ROWS.some((r) => 'examined' in r)
|
||||||
|
const INVENTORY = HAS_INVENTORY ? GRAPH_ROWS.map((r) => r.file) : []
|
||||||
|
const covered = new Set(HAS_INVENTORY ? GRAPH_ROWS.filter((r) => r.examined).map((r) => r.file) : [])
|
||||||
|
const PRE_COVERED = covered.size
|
||||||
|
const RISKY_FILES = HAS_INVENTORY ? GRAPH_ROWS.filter((r) => r.risky).map((r) => r.file) : []
|
||||||
|
const uncoveredCount = () => (HAS_INVENTORY ? INVENTORY.reduce((n, f) => n + (covered.has(f) ? 0 : 1), 0) : 0)
|
||||||
|
if (HAS_INVENTORY) log(`coverage: inventory=${INVENTORY.length} preCovered=${PRE_COVERED} risky=${RISKY_FILES.length}`)
|
||||||
const EXPLORE_FILES_PER_LENS = 10
|
const EXPLORE_FILES_PER_LENS = 10
|
||||||
const exploreConsumed = new Set() // within-run consumption: never re-offer a file to a later round
|
const exploreConsumed = new Set() // within-run consumption: never re-offer a file to a later round
|
||||||
let exploreFallbackLogged = false
|
let exploreFallbackLogged = false
|
||||||
@@ -316,9 +341,18 @@ function drawExploreFiles() {
|
|||||||
}
|
}
|
||||||
pool = churnFiles
|
pool = churnFiles
|
||||||
}
|
}
|
||||||
const files = pool
|
const avail = pool.filter((f) => !exploreConsumed.has(f) && !covered.has(f) && !seen.some((s) => s.file === f))
|
||||||
.filter((f) => !exploreConsumed.has(f) && !seen.some((s) => s.file === f))
|
const files = []
|
||||||
.slice(0, EXPLORE_FILES_PER_LENS)
|
while (files.length < EXPLORE_FILES_PER_LENS && avail.length) {
|
||||||
|
const head = avail.shift()
|
||||||
|
files.push(head)
|
||||||
|
const dir = clusterOf(head)
|
||||||
|
// pull same-directory siblings forward: one lens reading one module beats ten strangers
|
||||||
|
for (let i = 0; i < avail.length && files.length < EXPLORE_FILES_PER_LENS; ) {
|
||||||
|
if (clusterOf(avail[i]) === dir) files.push(avail.splice(i, 1)[0])
|
||||||
|
else i++
|
||||||
|
}
|
||||||
|
}
|
||||||
for (const f of files) exploreConsumed.add(f)
|
for (const f of files) exploreConsumed.add(f)
|
||||||
return files
|
return files
|
||||||
}
|
}
|
||||||
@@ -333,7 +367,12 @@ function exploreLens(i) {
|
|||||||
`single finding in them - either they are clean or every lens so far walked past them.`
|
`single finding in them - either they are clean or every lens so far walked past them.`
|
||||||
return {
|
return {
|
||||||
key: `explore-${i}`,
|
key: `explore-${i}`,
|
||||||
prompt: `${src} Read each one IN FULL with fresh eyes and hunt for real bugs of any class:\n` +
|
prompt: `${src} Read each one IN FULL with fresh eyes. Hunt every class: concurrency and ` +
|
||||||
|
`interleaving (races, TOCTOU, lock ordering, stale-closure writes after await); lifecycle and ` +
|
||||||
|
`teardown (unreleased acquires, use-after-close, missing disposal on error paths); state desync ` +
|
||||||
|
`(two sources of truth updated by different code paths); error-path data loss (swallowed errors, ` +
|
||||||
|
`partial writes, silent fallbacks); ordering and boundaries (off-by-one, pagination truncation, ` +
|
||||||
|
`sequence gaps, inclusive/exclusive disagreements).\n` +
|
||||||
files.map((f) => ` - ${f}`).join('\n'),
|
files.map((f) => ` - ${f}`).join('\n'),
|
||||||
files,
|
files,
|
||||||
}
|
}
|
||||||
@@ -348,8 +387,10 @@ function buildAdaptiveLenses(round) {
|
|||||||
byCluster[cl].push(c)
|
byCluster[cl].push(c)
|
||||||
}
|
}
|
||||||
// Explore-heavy schedule: measured hotspot yield flattened to 0.25 high+med/agent by round 6.
|
// Explore-heavy schedule: measured hotspot yield flattened to 0.25 high+med/agent by round 6.
|
||||||
|
const sweeping = uncoveredCount() > 0
|
||||||
const hotspotQuota = round <= 5 ? 2 : 1
|
const hotspotQuota = round <= 5 ? 2 : 1
|
||||||
const exploreQuota = round <= 5 ? 2 : 3
|
// sweep pace: 4 explore lenses x 10 files while inventory files remain uncovered
|
||||||
|
const exploreQuota = sweeping ? 4 : round <= 5 ? 2 : 3
|
||||||
const hotKey = (cl) => ('hotspot ' + cl).toLowerCase().replace(/[^a-z0-9]+/g, '-')
|
const hotKey = (cl) => ('hotspot ' + cl).toLowerCase().replace(/[^a-z0-9]+/g, '-')
|
||||||
const picked = Object.entries(byCluster)
|
const picked = Object.entries(byCluster)
|
||||||
.sort((a, b) => b[1].length - a[1].length)
|
.sort((a, b) => b[1].length - a[1].length)
|
||||||
@@ -370,7 +411,7 @@ function buildAdaptiveLenses(round) {
|
|||||||
if (shortfall > 0) log(`adaptive: hotspot pool short by ${shortfall} - trying explore backfill`)
|
if (shortfall > 0) log(`adaptive: hotspot pool short by ${shortfall} - trying explore backfill`)
|
||||||
const explores = []
|
const explores = []
|
||||||
for (let i = 1; i <= exploreQuota + shortfall; i++) {
|
for (let i = 1; i <= exploreQuota + shortfall; i++) {
|
||||||
if ((cleanStreak[`explore-${i}`] || 0) >= 2) continue // demoted slot: no substitution, that IS demotion
|
if (!sweeping && (cleanStreak[`explore-${i}`] || 0) >= 2) continue // demoted slot: no substitution, that IS demotion
|
||||||
const lens = exploreLens(i)
|
const lens = exploreLens(i)
|
||||||
if (!lens) {
|
if (!lens) {
|
||||||
log(`adaptive: explore pool exhausted after ${explores.length} lens(es)`)
|
log(`adaptive: explore pool exhausted after ${explores.length} lens(es)`)
|
||||||
@@ -424,12 +465,14 @@ function seenBlock(seen) {
|
|||||||
const lines = seen.map((s) => ` - ${s.file}:${s.line} [${s.status}] ${s.title}`)
|
const lines = seen.map((s) => ` - ${s.file}:${s.line} [${s.status}] ${s.title}`)
|
||||||
return `\n--- KNOWN FINDINGS (already investigated - do NOT re-report; refuted means examined and rejected) ---\n${lines.join('\n')}\n`
|
return `\n--- KNOWN FINDINGS (already investigated - do NOT re-report; refuted means examined and rejected) ---\n${lines.join('\n')}\n`
|
||||||
}
|
}
|
||||||
function convergenceTable(stats, converged, stoppedOnBudget) {
|
function convergenceTable(stats, converged, stoppedOnBudget, stalled) {
|
||||||
const verdict = converged
|
const verdict = converged
|
||||||
? `CONVERGED after ${stats.length} round(s).`
|
? `CONVERGED after ${stats.length} round(s).`
|
||||||
: stoppedOnBudget
|
: stalled
|
||||||
? 'NOT converged - stopped on budget.'
|
? 'NOT converged - coverage stalled.'
|
||||||
: 'NOT converged - hit the round backstop.'
|
: stoppedOnBudget
|
||||||
|
? 'NOT converged - stopped on budget.'
|
||||||
|
: 'NOT converged - hit the round backstop.'
|
||||||
const rows = stats.map(
|
const rows = stats.map(
|
||||||
(s) =>
|
(s) =>
|
||||||
`| ${s.round} | ${s.family} | ${s.lenses} | ${s.candidates} | ${s.fresh} | ${s.confirmed} | ${s.refuted} | ${s.dryEligible ? 'yes' : 'NO'} | ${s.dryAfter} |`,
|
`| ${s.round} | ${s.family} | ${s.lenses} | ${s.candidates} | ${s.fresh} | ${s.confirmed} | ${s.refuted} | ${s.dryEligible ? 'yes' : 'NO'} | ${s.dryAfter} |`,
|
||||||
@@ -494,6 +537,8 @@ const cleanStreak = {}
|
|||||||
let dry = 0
|
let dry = 0
|
||||||
let round = 0
|
let round = 0
|
||||||
let stoppedOnBudget = false
|
let stoppedOnBudget = false
|
||||||
|
let coverageStall = 0
|
||||||
|
let stalledCoverage = false
|
||||||
|
|
||||||
function finderPrompt(lens, rnd) {
|
function finderPrompt(lens, rnd) {
|
||||||
return (
|
return (
|
||||||
@@ -524,18 +569,35 @@ function verifyPrompt(lensKey, candidates) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
while (dry < DRY_THRESHOLD && round < MAX_ROUNDS) {
|
const riskySweepPending = () => HAS_INVENTORY && RISKY_FILES.length > 0 && !riskySweepDone
|
||||||
|
while ((uncoveredCount() > 0 || riskySweepPending() || dry < DRY_THRESHOLD) && round < MAX_ROUNDS) {
|
||||||
if (BUDGET_TOTAL && remainingBudget() < ROUND_BUDGET_FLOOR) {
|
if (BUDGET_TOTAL && remainingBudget() < ROUND_BUDGET_FLOOR) {
|
||||||
stoppedOnBudget = true
|
stoppedOnBudget = true
|
||||||
log(`Budget floor reached (${Math.round(remainingBudget() / 1000)}k left) - stopping before round ${round + 1}`)
|
log(`Budget floor reached (${Math.round(remainingBudget() / 1000)}k left) - stopping before round ${round + 1}`)
|
||||||
break
|
break
|
||||||
}
|
}
|
||||||
const family = lensesForRound(round + 1)
|
const family = lensesForRound(round + 1)
|
||||||
if (!family || !family.length) break // nothing to hunt != everything demoted
|
if (!family || !family.length) {
|
||||||
|
// Coverage mode with the pool drained and the risky sweep done: an empty family means
|
||||||
|
// hotspots are demoted/cooled and there is genuinely nothing left to hunt - that IS
|
||||||
|
// quietness. Count it as a dry round so a late confirm cannot strand a fully-covered
|
||||||
|
// run one dry round short of its earned convergence. Legacy mode keeps the hard stop:
|
||||||
|
// an empty family there means "nothing targetable" (no churn, no graph), not "done".
|
||||||
|
if (HAS_INVENTORY && uncoveredCount() === 0 && !riskySweepPending()) {
|
||||||
|
round++
|
||||||
|
dry++
|
||||||
|
roundStats.push({ round, family: 'exhausted', lenses: 0, candidates: 0, fresh: 0, confirmed: 0, refuted: 0, dryEligible: true, dryAfter: dry, severity: { critical: 0, high: 0, medium: 0, low: 0 }, perLens: {}, filesTouched: 0, filesNew: 0, suppressedLedger: 0, suppressedRun: 0, finderNull: 0, finderEmpty: 0, verifierNull: 0, spentBefore: budget.spent(), spentAfter: budget.spent() })
|
||||||
|
log(`Round ${round}: nothing left to hunt - counts as a dry round (dry=${dry})`)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
break // nothing to hunt != everything demoted
|
||||||
|
}
|
||||||
round++
|
round++
|
||||||
|
const uncBefore = uncoveredCount()
|
||||||
const spentBefore = budget.spent()
|
const spentBefore = budget.spent()
|
||||||
const counts = { suppressedLedger: 0, suppressedRun: 0, finderNull: 0, finderEmpty: 0, verifierNull: 0 }
|
const counts = { suppressedLedger: 0, suppressedRun: 0, finderNull: 0, finderEmpty: 0, verifierNull: 0 }
|
||||||
const lenses = family.filter((l) => (cleanStreak[l.key] || 0) < 2)
|
const sweepingNow = uncoveredCount() > 0
|
||||||
|
const lenses = family.filter((l) => (sweepingNow && /^explore-/.test(l.key)) || (cleanStreak[l.key] || 0) < 2)
|
||||||
if (!lenses.length) {
|
if (!lenses.length) {
|
||||||
dry++
|
dry++
|
||||||
roundStats.push({ round, family: familyName(round), lenses: 0, candidates: 0, fresh: 0, confirmed: 0, refuted: 0, dryEligible: true, dryAfter: dry, severity: { critical: 0, high: 0, medium: 0, low: 0 }, perLens: {}, filesTouched: 0, filesNew: 0, ...counts, spentBefore, spentAfter: budget.spent() })
|
roundStats.push({ round, family: familyName(round), lenses: 0, candidates: 0, fresh: 0, confirmed: 0, refuted: 0, dryEligible: true, dryAfter: dry, severity: { critical: 0, high: 0, medium: 0, low: 0 }, perLens: {}, filesTouched: 0, filesNew: 0, ...counts, spentBefore, spentAfter: budget.spent() })
|
||||||
@@ -613,10 +675,13 @@ while (dry < DRY_THRESHOLD && round < MAX_ROUNDS) {
|
|||||||
candCount += r.unionCount
|
candCount += r.unionCount
|
||||||
freshCount += r.fresh.length
|
freshCount += r.fresh.length
|
||||||
if (r.finderFailed) eligible = false
|
if (r.finderFailed) eligible = false
|
||||||
if (r.finderFailed && r.lens.files) {
|
if (r.lens.files) {
|
||||||
// a dead finder read nothing: un-consume its draw so later rounds can re-offer the
|
// coverage credit (spec: only explicit-file lenses that ran to completion). A lens
|
||||||
// files and the session does not record never-examined files as explored-clean
|
// denied credit - dead finder OR candidates left unverified - returns its whole draw
|
||||||
for (const f of r.lens.files) exploreConsumed.delete(f)
|
// to the pool: consumed-but-uncovered files would otherwise strand uncoveredCount()
|
||||||
|
// above zero forever, and a partially-verified draw is not evidence of cleanliness.
|
||||||
|
if (!r.finderFailed && !r.unmatched.length) for (const f of r.lens.files) covered.add(f)
|
||||||
|
else for (const f of r.lens.files) exploreConsumed.delete(f)
|
||||||
}
|
}
|
||||||
let lensConfirmed = 0
|
let lensConfirmed = 0
|
||||||
let lensRefuted = 0
|
let lensRefuted = 0
|
||||||
@@ -627,6 +692,7 @@ while (dry < DRY_THRESHOLD && round < MAX_ROUNDS) {
|
|||||||
const rec = { file: v.file, line: v.line, title: v.title, status: v.refuted ? 'refuted' : 'confirmed' }
|
const rec = { file: v.file, line: v.line, title: v.title, status: v.refuted ? 'refuted' : 'confirmed' }
|
||||||
if (seen.some((p) => isDup(rec, p))) { counts.suppressedRun++; continue } // cross-lens same-round duplicate
|
if (seen.some((p) => isDup(rec, p))) { counts.suppressedRun++; continue } // cross-lens same-round duplicate
|
||||||
seen.push(rec)
|
seen.push(rec)
|
||||||
|
covered.add(rec.file) // any verdict proves the file was read (inert in legacy mode: seen already blocks re-draws)
|
||||||
if (v.refuted) { newRefuted++; lensRefuted++ }
|
if (v.refuted) { newRefuted++; lensRefuted++ }
|
||||||
else {
|
else {
|
||||||
newConfirmed++
|
newConfirmed++
|
||||||
@@ -654,9 +720,24 @@ while (dry < DRY_THRESHOLD && round < MAX_ROUNDS) {
|
|||||||
// ineligible zero-confirm round: dry unchanged - "we didn't fully look" is not "it's clean"
|
// ineligible zero-confirm round: dry unchanged - "we didn't fully look" is not "it's clean"
|
||||||
roundStats.push({ round, family: familyName(round), lenses: lenses.length, candidates: candCount, fresh: freshCount, confirmed: newConfirmed, refuted: newRefuted, dryEligible: eligible, dryAfter: dry, severity: sevMix, perLens, filesTouched: filesTouched.size, filesNew: filesNew.size, ...counts, spentBefore, spentAfter: budget.spent() })
|
roundStats.push({ round, family: familyName(round), lenses: lenses.length, candidates: candCount, fresh: freshCount, confirmed: newConfirmed, refuted: newRefuted, dryEligible: eligible, dryAfter: dry, severity: sevMix, perLens, filesTouched: filesTouched.size, filesNew: filesNew.size, ...counts, spentBefore, spentAfter: budget.spent() })
|
||||||
log(`Round ${round} (${familyName(round)}): ${newConfirmed} confirmed, ${newRefuted} refuted, dry=${dry}${eligible ? '' : ' (ineligible)'}`)
|
log(`Round ${round} (${familyName(round)}): ${newConfirmed} confirmed, ${newRefuted} refuted, dry=${dry}${eligible ? '' : ' (ineligible)'}`)
|
||||||
|
|
||||||
|
// stalled coverage: an adaptive round that failed to shrink a non-empty uncovered pool
|
||||||
|
// AND confirmed nothing. Only adaptive rounds count - rounds 1-3 never draw explore
|
||||||
|
// files by design - and a round that confirmed a bug is never a stall: hotspot yield
|
||||||
|
// does not shrink the pool, and cutting off a still-productive hunt is the one thing
|
||||||
|
// a bug-finding tool must not do.
|
||||||
|
const uncAfter = uncoveredCount()
|
||||||
|
if (HAS_INVENTORY && familyName() === 'adaptive' && uncAfter > 0 && newConfirmed === 0) {
|
||||||
|
coverageStall = uncAfter < uncBefore ? 0 : coverageStall + 1
|
||||||
|
if (coverageStall >= 2) {
|
||||||
|
stalledCoverage = true
|
||||||
|
log(`Coverage stalled: uncovered=${uncAfter} did not shrink for 2 adaptive rounds - stopping`)
|
||||||
|
break
|
||||||
|
}
|
||||||
|
} else coverageStall = 0
|
||||||
}
|
}
|
||||||
|
|
||||||
const converged = dry >= DRY_THRESHOLD
|
const converged = uncoveredCount() === 0 && !riskySweepPending() && dry >= DRY_THRESHOLD
|
||||||
|
|
||||||
// ---------- report (deterministic) ----------
|
// ---------- report (deterministic) ----------
|
||||||
// A report agent silently dropped findings (79 sections for 82 confirmed on 2026-08-12), so the
|
// A report agent silently dropped findings (79 sections for 82 confirmed on 2026-08-12), so the
|
||||||
@@ -665,14 +746,16 @@ const converged = dry >= DRY_THRESHOLD
|
|||||||
const RANK = { critical: 0, high: 1, medium: 2, low: 3 }
|
const RANK = { critical: 0, high: 1, medium: 2, low: 3 }
|
||||||
const confirmedSorted = confirmedAll.slice().sort((a, b) => RANK[a.severity] - RANK[b.severity])
|
const confirmedSorted = confirmedAll.slice().sort((a, b) => RANK[a.severity] - RANK[b.severity])
|
||||||
const unverifiedFinal = unverified.filter((u) => !seen.some((p) => isDup(u, p)))
|
const unverifiedFinal = unverified.filter((u) => !seen.some((p) => isDup(u, p)))
|
||||||
const table = convergenceTable(roundStats, converged, stoppedOnBudget)
|
const table = convergenceTable(roundStats, converged, stoppedOnBudget, stalledCoverage)
|
||||||
const sum = (k) => roundStats.reduce((n, r) => n + (r[k] || 0), 0)
|
const sum = (k) => roundStats.reduce((n, r) => n + (r[k] || 0), 0)
|
||||||
const runStats = {
|
const runStats = {
|
||||||
config: { maxRounds: MAX_ROUNDS, dryThreshold: DRY_THRESHOLD, customLenses: !!CUSTOM_LENSES, knownCount: (ARGS.known || []).length, graphRows: GRAPH_ROWS.length, budgetTotal: BUDGET_TOTAL },
|
config: { maxRounds: MAX_ROUNDS, dryThreshold: DRY_THRESHOLD, customLenses: !!CUSTOM_LENSES, knownCount: (ARGS.known || []).length, graphRows: GRAPH_ROWS.length, budgetTotal: BUDGET_TOTAL },
|
||||||
|
coverage: HAS_INVENTORY ? { inventory: INVENTORY.length, preCovered: PRE_COVERED, covered: INVENTORY.length - uncoveredCount(), uncoveredAtStop: uncoveredCount() } : null,
|
||||||
spentTotal: budget.spent(),
|
spentTotal: budget.spent(),
|
||||||
rounds: roundStats.length,
|
rounds: roundStats.length,
|
||||||
converged,
|
converged,
|
||||||
stoppedOnBudget,
|
stoppedOnBudget,
|
||||||
|
stalledCoverage,
|
||||||
confirmed: confirmedSorted.length,
|
confirmed: confirmedSorted.length,
|
||||||
refuted: sum('refuted'),
|
refuted: sum('refuted'),
|
||||||
unverified: unverifiedFinal.length,
|
unverified: unverifiedFinal.length,
|
||||||
@@ -686,9 +769,11 @@ const runStats = {
|
|||||||
function buildReport() {
|
function buildReport() {
|
||||||
const outcome = converged
|
const outcome = converged
|
||||||
? `CONVERGED after ${round} round(s).`
|
? `CONVERGED after ${round} round(s).`
|
||||||
: stoppedOnBudget
|
: stalledCoverage
|
||||||
? `NOT converged - stopped on budget after ${round} round(s).`
|
? `NOT converged - coverage stalled after ${round} round(s).`
|
||||||
: `NOT converged - hit the round backstop after ${round} round(s).`
|
: stoppedOnBudget
|
||||||
|
? `NOT converged - stopped on budget after ${round} round(s).`
|
||||||
|
: `NOT converged - hit the round backstop after ${round} round(s).`
|
||||||
const sev = { critical: 0, high: 0, medium: 0, low: 0 }
|
const sev = { critical: 0, high: 0, medium: 0, low: 0 }
|
||||||
for (const f of confirmedSorted) sev[f.severity] = (sev[f.severity] || 0) + 1
|
for (const f of confirmedSorted) sev[f.severity] = (sev[f.severity] || 0) + 1
|
||||||
const lines = ['# Bug hunt report', '']
|
const lines = ['# Bug hunt report', '']
|
||||||
@@ -721,6 +806,13 @@ function buildReport() {
|
|||||||
`Agent failures: ${runStats.finderNull} finder null, ${runStats.finderEmpty} finder empty, ${runStats.verifierNull} verifier null.`,
|
`Agent failures: ${runStats.finderNull} finder null, ${runStats.finderEmpty} finder empty, ${runStats.verifierNull} verifier null.`,
|
||||||
'',
|
'',
|
||||||
)
|
)
|
||||||
|
if (runStats.coverage)
|
||||||
|
lines.push(
|
||||||
|
`Coverage: ${runStats.coverage.covered}/${runStats.coverage.inventory} files ` +
|
||||||
|
`(${runStats.coverage.preCovered} pre-covered from ledger + live explored-clean); ` +
|
||||||
|
`${runStats.coverage.uncoveredAtStop} uncovered at stop.`,
|
||||||
|
'',
|
||||||
|
)
|
||||||
lines.push('| round | spent | files (new) | suppressed ledger/run | finder null/empty | verifier null |')
|
lines.push('| round | spent | files (new) | suppressed ledger/run | finder null/empty | verifier null |')
|
||||||
lines.push('|---|---|---|---|---|---|')
|
lines.push('|---|---|---|---|---|---|')
|
||||||
for (const s of roundStats)
|
for (const s of roundStats)
|
||||||
@@ -729,4 +821,4 @@ function buildReport() {
|
|||||||
}
|
}
|
||||||
const report = buildReport()
|
const report = buildReport()
|
||||||
|
|
||||||
return { converged, stoppedOnBudget, rounds: roundStats, confirmed: confirmedSorted, unverified: unverifiedFinal, runStats, exploredFiles: [...exploreConsumed], report }
|
return { converged, stoppedOnBudget, stalledCoverage, rounds: roundStats, confirmed: confirmedSorted, unverified: unverifiedFinal, runStats, exploredFiles: [...exploreConsumed], report }
|
||||||
|
|||||||
Reference in New Issue
Block a user