Skip to content

Commit 115e55d

Browse files
committed
Add more margin for judge context
1 parent 1760568 commit 115e55d

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

evals/git-evals/judge-git-eval.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ import { JudgingAnalysisSchema } from './types'
88

99
import type { EvalRunLog } from './types'
1010

11-
const MAX_TOKENS = 1_000_000 // 1 million token limit
11+
const MAX_TOKENS = 1_000_000 * 0.95 // 1 million token limit, with 5% fudge factor
1212

1313
function buildAnalysisPrompt(
1414
evalRun: EvalRunLog,

0 commit comments

Comments
 (0)