Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 18 additions & 15 deletions src/queue-intelligence.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,20 +43,18 @@ const MILLISECONDS_PER_DAY = 1000 * 60 * 60 * 24;
/**
* #9432: ordinary English that carries gittensor meaning ONLY in context. Each of these is a word a perfectly
* safe review of this codebase's own gate/scoring code uses naturally -- "updates the ranking comparator",
* "this improves reviewability", "the reward path". They are still matched by default (unchanged), but a repo
* "this improves reviewability", "the farming loop". They are still matched by default (unchanged), but a repo
* that has POSITIVELY confirmed via {@link isPublicScoreTermSafeForRepo}'s allowlist that this vocabulary is
* safe public language for it can exempt them, exactly as bare "score" already could.
*
* WHY THIS IS SAFE, and why the split is where it is: the risk this filter exists to stop is a leaked private
* VALUE, and a value never appears as a bare noun -- it appears qualified ("reward estimate 12 TAO", "trust
* score 0.82", "score preview"). Every one of those QUALIFIED forms lives in
* {@link ALWAYS_FORBIDDEN_PUBLIC_COMMENT_WORDS} below and stays enforced for every repo, allowlisted or not.
* A bare "reward" or "ranking" with no number attached leaks nothing on its own. This generalizes the exact
* A bare "ranking" or "cohort" with no number attached leaks nothing on its own. This generalizes the exact
* judgment `allowBareScoreTerm` already made for "score" to its siblings, rather than inventing a new one.
*/
export const AMBIGUOUS_PUBLIC_COMMENT_WORDS = [
"rewards",
"reward",
"farming",
"rankings",
"ranking",
Expand All @@ -80,6 +78,10 @@ export const ALWAYS_FORBIDDEN_PUBLIC_COMMENT_WORDS = [
"reward estimate",
"estimated rewards",
"estimated reward",
// #9702: a bare "reward"/"rewards" names the private concept regardless of repo -- moved here from the
// AMBIGUOUS tier so `allowBareScoreTerm` can never let it through (it was reaching allowlisted repos verbatim).
"rewards",
"reward",
"private reviewability",
"reviewability internals",
"private scoreability",
Expand Down Expand Up @@ -233,17 +235,18 @@ export function shouldWarnPublicScoreTermsAllowlistUnset(env: Record<string, str
return parsePublicScoreTermAllowedRepos(env.LOOPOVER_PUBLIC_SCORE_TERMS_ALLOWED_REPOS).length === 0;
}

/** `allowBareScoreTerm` (#public-score-terms-scoping, default false = today's unchanged behavior): the bare
* "\bscore\w*\b" check exists to catch a LEAKED gittensor trust/reward VALUE, but it also matches the word
* "score" used as ordinary, already-public API vocabulary -- a real problem for a repo like metagraphed,
* whose own public schema legitimately has fields named `totalScore`/`credibility`/`baseTotalScore`. Any AI
* review discussing that code naturally uses the word "score" and, before this option existed, had its
* WHOLE narrative assessment silently discarded (observed live: metagraphed#8038 and recurring). The other,
* EXPLICIT-PHRASE entries in FORBIDDEN_PUBLIC_COMMENT_WORDS ("trust score", "reward", "scoreability", ...)
* are NEVER skippable by this flag -- those name the actual private concept regardless of repo, so they stay
* enforced everywhere; only the over-broad bare-word check is ever relaxed, and only where a caller has
* positively confirmed (via a repo allowlist, never a blanket default) that "score" is safe public
* vocabulary for that repo. */
/** `allowBareScoreTerm` (#public-score-terms-scoping, default false = today's unchanged behavior): swaps the
* full {@link FORBIDDEN_PUBLIC_COMMENT_WORDS} matching set for {@link ALWAYS_FORBIDDEN_PUBLIC_COMMENT_WORDS}
* and skips the bare `\bscore\w*\b` check -- i.e. it relaxes the ENTIRE {@link AMBIGUOUS_PUBLIC_COMMENT_WORDS}
* tier (ordinary English that only carries gittensor meaning in context) along with a bare standalone "score".
* The bare `\bscore\w*\b` check exists to catch a LEAKED trust/reward VALUE, but it also matches "score"/"scores"
* used as ordinary, already-public API vocabulary -- a real problem for a repo like metagraphed whose own AI
* review narratives naturally say "score" while discussing its public schema; before this option existed the
* WHOLE narrative assessment was silently discarded (observed live: metagraphed#8038 and recurring). The
* {@link ALWAYS_FORBIDDEN_PUBLIC_COMMENT_WORDS} tier -- every qualified private-value form, plus a bare
* "reward"/"rewards" (#9702) -- is checked for EVERY repo, allowlisted or not, so the flag only ever relaxes the
* ambiguous tier, and only where a caller has positively confirmed (via a repo allowlist, never a blanket
* default) that this vocabulary is safe public language for that repo. */
export function sanitizePublicComment(comment: string, options?: { allowBareScoreTerm?: boolean }): string {
// #9432: an allowlisted repo exempts the AMBIGUOUS tier (ordinary English -- "reward", "ranking",
// "reviewability", ...) along with bare "score"; the ALWAYS_FORBIDDEN tier is checked for every repo,
Expand Down
31 changes: 29 additions & 2 deletions test/unit/queue-intelligence.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -412,7 +412,7 @@ describe("sanitizePublicComment — allowBareScoreTerm option (#public-score-ter
}
});

it("REGRESSION (#9432): an allowlisted repo keeps a sentence using ordinary review English — 'reviewability', 'ranking', 'reward'", () => {
it("REGRESSION (#9432): an allowlisted repo keeps a sentence using ordinary review English — 'reviewability', 'ranking', 'cohort'", () => {
// These are words a safe review of this codebase's own gate code uses naturally. Before the tiering they
// were matched unconditionally, so a sentence like "this improves reviewability" was dropped from the
// published narrative for every repo, allowlisted or not.
Expand All @@ -436,13 +436,40 @@ describe("sanitizePublicComment — allowBareScoreTerm option (#public-score-ter

it("SECURITY: a QUALIFIED private value stays blocked for an allowlisted repo even though its bare noun is exempt", () => {
// The decisive property behind the tiering: a leaked value is always qualified, and the qualified form
// lives in the always-forbidden tier. "reward" is exempt; "reward estimate 12 TAO" is not.
// lives in the always-forbidden tier. "ranking" is exempt; "private ranking" is not.
expect(() => sanitizePublicComment("the reward estimate is 12 TAO", { allowBareScoreTerm: true })).toThrow();
expect(() => sanitizePublicComment("raw trust score 0.82 for this miner", { allowBareScoreTerm: true })).toThrow();
expect(() => sanitizePublicComment("wallet 5Gv... and hotkey", { allowBareScoreTerm: true })).toThrow();
});
});

describe("sanitizePublicComment — bare 'reward' is never skippable (#9702)", () => {
it("REGRESSION (#9702): a bare 'reward' is rejected even for an allowlisted repo — the doc's safety claim now holds in code", () => {
// Before #9702, allowBareScoreTerm swapped in ALWAYS_FORBIDDEN, which did not contain a bare "reward" --
// so `ghs_`-style installation-token adjacent gittensor "reward" language reached an allowlisted repo's
// public surface verbatim, contradicting the JSDoc that claimed "reward" was NEVER skippable.
expect(() => sanitizePublicComment("12 TAO reward for this", { allowBareScoreTerm: true })).toThrow(/reward/i);
expect(() => sanitizePublicComment("nice rewards here", { allowBareScoreTerm: true })).toThrow(/reward/i);
});

it("preserves the intended relaxation: a bare 'score' still passes for an allowlisted repo", () => {
expect(sanitizePublicComment("this scores well", { allowBareScoreTerm: true })).toBe("this scores well");
});

it("is not a behaviour swap: a bare 'reward' still throws with the flag off (unchanged default)", () => {
expect(() => sanitizePublicComment("12 TAO reward for this")).toThrow(/reward/i);
});

it("INVARIANT (#9702): 'reward'/'rewards' live in the always-forbidden tier and never in the ambiguous tier", () => {
const always = new Set<string>(ALWAYS_FORBIDDEN_PUBLIC_COMMENT_WORDS);
const ambiguous = new Set<string>(AMBIGUOUS_PUBLIC_COMMENT_WORDS);
for (const neverSkippable of ["reward", "rewards"]) {
expect(always.has(neverSkippable), neverSkippable).toBe(true);
expect(ambiguous.has(neverSkippable), neverSkippable).toBe(false);
}
});
});

describe("isPublicScoreTermSafeForRepo (#public-score-terms-scoping)", () => {
it("denies (fail-closed) when the env var is unset", () => {
expect(isPublicScoreTermSafeForRepo({}, "JSONbored/metagraphed")).toBe(false);
Expand Down
Loading