Fix the 15 weakest skills (audit-driven, fabrication-checked)#32
Merged
Conversation
The multi-agent quality audit (71 skills, 6-dimension rubric, adversarially re-checked) flagged these 15 as weakest (2.17–2.50/5), almost all sharing: grounding=1 (no real citations), thin validation, and confirmed correctness bugs. Process: audit → expert fixer per skill (HARD no-fabrication rules) → adversarial verifier hunting each fixed file for invented CVEs/citations/new errors → manual correction of every flag. 7/15 came back verifier-clean; the other 8 had issues the verifier caught (this is the point of the pass) — all hand-corrected here. Confirmed correctness bugs fixed (examples): - hunt-k8s: broken kubelet /exec (SPDY stream, not plain POST) → /run primitive - hunt-cicd: `cat $GITHUB_TOKEN` (env var, not a file) corrected - hunt-grpc: h2load is not a Rapid-Reset PoC; fixed tool + wrong CLI flags + quiche URL - hunt-cors: wrong regex-bypass example corrected - hunt-websocket: wrong socket.io `nsp=` syntax; removed mischaracterized CVE-2017-16031 - hunt-dom: broken cross-origin Service-Worker payload; corrected DOM-clobbering example - hunt-tls-network: CAA mischaracterization; replaced dead `openssl -ssl2/-ssl3` flags - hunt-ldap: `{SSHA}` hashcat mode 1411→111 (1411 is {SSHA256}) - hunt-lfi: misleading php://input; added filter-chain RCE; fixed invented "Gaztech" credit Grounding: added only well-known, verifiable CVEs (Heartbleed/POODLE/DROWN/FREAK/SWEET32, Apache 2.4.49/.50 path traversal, runc Leaky Vessels, jQuery 11022/11023, Jenkins CVE-2024-23897, etc.). Anti-fabrication enforced: REMOVED two HackerOne report IDs (#226659, #281575) that could not be independently verified — no invented report IDs, payouts, or stats. Validation/false-positive discipline strengthened throughout. All 15: lint clean, descriptions ≤1024, bodies ≤500 lines, YAML-safe (strict/Codex). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pull Bot
pushed a commit
to danielabelski/Claude-BugHunter
that referenced
this pull request
Jun 7, 2026
Second accuracy pass from the multi-agent audit — the confirmed correctness errors in otherwise-adequate/strong skills (the weakest 15 were PR elementalsouls#32). Same pipeline: surgical fix → adversarial verify → hand-correct. 14/21 verifier-clean; 3 flagged (bugcrowd §8.1 stale ordering, meme-coin SUAVE overstatement, hunt-saml partial) all hand-corrected. Diffs are intentionally small (182+/132−). SEVERE — fabrications / wrong identifiers: - okta-attack: fabricated `CVE-2024-VERIFY` -> real CVE-2024-10327 (Okta Verify iOS push-notification bypass; web-verified on NVD) - enterprise-vpn-attack: `CVE-2024-46805` -> CVE-2023-46805 (Ivanti, the real number) - supply-chain-attack-recon: SUNBURST no longer mislabeled CVE-2020-10148; cite CISA AA20-352A - meme-coin-audit: removed invented "35%/25%/20%" stats + phantom tool refs; SUAVE de-overstated - hunt-subdomain: removed an UNVERIFIABLE HackerOne report id (#1487793) — kept the technique - bugcrowd-reporting: fixed backwards chain-submission UUID ordering (§5.1 AND §8.1) - hunt-business-logic: corrected the "HMAC replay with modified payload bypasses" claim HIGH — stale facts / backwards logic / class conflations: - hunt-xss (removed-2019 Chrome XSS Auditor), hunt-xxe (.NET XmlReader DtdProcessing default), triage-validation (CVSS vectors recomputed), hunt-ssti/hunt-springboot (backwards `#{7*7}` logic), hunt-open-redirect (CRLF != open-redirect), hunt-saml (gzip/comment-injection corrected; description de-overclaimed), hunt-cache-poison, hunt-mfa-bypass, hunt-nextjs, redteam-mindset, vmware-vcenter-attack, mid-engagement-ir-detection, hunt-sharepoint. Anti-fabrication enforced: every added identifier is a well-known real CVE (Ivanti 2023-46805, Next.js 2024-34351, SharePoint ToolShell 2025-49704/49706, ruby-saml 2017-11428, Okta 2024-10327) or a real advisory (CISA AA20-352A); the one unverifiable HackerOne id was removed, not kept. All 71 lint clean (incl. the new YAML-safety check), descriptions <=1024, bodies <=500. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rewrites the 15 weakest skills identified by the multi-agent quality audit (71 skills scored on a 6-dimension rubric, weak/error flags adversarially re-checked). These scored 2.17–2.50/5, almost all with
grounding=1(no real citations), thin validation, and confirmed correctness bugs.Process (audit → fix → adversarial verify → hand-correct)
Confirmed correctness bugs fixed (sample)
hunt-k8s/exec(SPDY stream, not plain POST) →/runprimitivehunt-cicdcat $GITHUB_TOKEN(env var, not a file)hunt-grpccloudflare/quicheURLhunt-corshunt-websocketnsp=syntax; removed mischaracterized CVE-2017-16031hunt-domhunt-tls-networkopenssl -ssl2/-ssl3flagshunt-ldap{SSHA}hashcat mode1411→111hunt-lfiphp://input; added filter-chain RCE; fixed invented "Gaztech" creditAnti-fabrication (the #1 risk, given the audit's findings)
2.4.49/.50, runc Leaky Vessels, jQuery11022/11023, JenkinsCVE-2024-23897, …).#226659,#281575) that I could not independently verify (HackerOne pages are JS SPAs — unverifiable via fetch). No invented report IDs, payouts, or statistics.Quality gates
All 15: lint clean (0 errors), descriptions ≤1024, bodies ≤500 lines, YAML-safe (strict/Codex parsers).
🤖 Generated with Claude Code