Skip to content

Allow partial credit in choices questions#1475

Merged
inducer merged 3 commits intomainfrom
copilot/update-choice-mode-and-scoring
Apr 12, 2026
Merged

Allow partial credit in choices questions#1475
inducer merged 3 commits intomainfrom
copilot/update-choice-mode-and-scoring

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 10, 2026

ChoiceQuestion and InlineMultiQuestion previously supported only binary correct/incorrect choices. This adds a correctness field to ChoiceDesc enabling fractional credit per choice.

Data model

  • Replaced ChoiceMode.CORRECT/INCORRECT with a unified STANDARD mode plus a correctness: PointCount field (default 0.0)
  • ~CORRECT~ remains backward-compatible (sets correctness=1.0); no-prefix choices continue to mean correctness=0.0
  • Partial credit specified via YAML dict syntax:
choices:
  - ~CORRECT~ Fully correct answer
  - An incorrect answer
  - mode: standard
    correctness: 0.5
    text: A half-credit answer

Scoring changes

  • ChoiceQuestion: grade() now returns choice.correctness directly instead of binary 0/1; page_correct_answer() displays the highest-correctness choice
  • ChoicesAnswer (InlineMultiQuestion): get_correctness() returns choice.correctness directly

Validation

  • ChoiceQuestion: at least one choice must have correctness > 0
  • MultipleChoiceQuestion: partial credit (correctness ∉ {0, 1}) is disallowed via a new model validator

Tests

  • Sandbox tests covering 0%, 50%, and 100% correctness for ChoiceQuestion
  • PythonClassFakeRepo-based validation tests confirming partial-credit pages validate successfully and that invalid configurations (all-zero ChoiceQuestion, partial-credit MultipleChoiceQuestion) raise ValidationError

Copilot AI linked an issue Apr 10, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Update ChoiceMode to allow partial credit in choices questions Allow partial credit in choices questions Apr 10, 2026
Copilot AI requested a review from inducer April 10, 2026 22:39
@inducer inducer force-pushed the copilot/update-choice-mode-and-scoring branch 2 times, most recently from b41213c to 41e9213 Compare April 11, 2026 20:21
@inducer inducer marked this pull request as ready for review April 11, 2026 20:25
@inducer inducer force-pushed the copilot/update-choice-mode-and-scoring branch 3 times, most recently from 59f98ac to 2a8bf44 Compare April 11, 2026 22:11
Copilot AI and others added 3 commits April 11, 2026 17:38
@inducer inducer force-pushed the copilot/update-choice-mode-and-scoring branch from 2a8bf44 to ec914e4 Compare April 11, 2026 22:38
@inducer inducer merged commit 5a21348 into main Apr 12, 2026
18 checks passed
@inducer inducer deleted the copilot/update-choice-mode-and-scoring branch April 12, 2026 02:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow partial credit in choices questions

2 participants