Skip to content

fix(autograder): stop penalizing left-in guide comments and schema-qualified views#8

Merged
lassebenni merged 1 commit into
mainfrom
fix/autograder-todo-and-schema-qualified-views
Jul 6, 2026
Merged

fix(autograder): stop penalizing left-in guide comments and schema-qualified views#8
lassebenni merged 1 commit into
mainfrom
fix/autograder-todo-and-schema-qualified-views

Conversation

@lassebenni

Copy link
Copy Markdown
Contributor

Problem

The Week 9 autograder (.hyf/test.sh) was failing correct student submissions. Running it against the 7 open PRs, 5 of 7 passing students scored 15–30/100 despite complete, correct SQL. Two bugs caused it:

1. file_is_filled() flagged any file containing the word "TODO". The SQL scaffold ships guide comments like -- TODO: GROUP BY the three columns …. Students wrote the correct query below the comment and (reasonably) left the guide comment in place. The grader treated the whole file as an empty stub. The markdown header line …Replace every TODO. tripped it too.

2. View detection missed schema-qualified names. The check grepped VIEW[[:space:]]+vw_dim_zones, which does not match CREATE VIEW dev_halyna.vw_dim_zones — i.e. it penalized the own-schema pattern the assignment explicitly requires. Halyna's correct submission lost 10 points to a false "view not found".

Fix

  • file_is_filled() now strips SQL line-comments and blank lines, requires real remaining content, and only treats a line whose content begins with TODO (an unreplaced markdown placeholder like TODO: what did you ask?) as a stub. Leaving a -- TODO: guide comment above finished SQL is now harmless; …Replace every TODO. is fine.
  • The vw_dim_zones / vw_fact_trips greps now allow an optional <schema>. prefix.

Verified against all 7 open PRs

PR before → after note
#1 15 → 100 guide comments no longer penalized
#2 30 → 100 same
#5 85 → 100 schema-qualified views now detected
#6 30 → 100 same
#3 15 → 95 still flags genuinely unfilled AI reflection prompts ✅
#4 15 → 95 still flags an unfilled AI reflection section ✅
#7 95 → 95 still flags misnamed borough_count.png.png

Real gaps are still caught, so this is not a blanket pass-everything loosening.

Note for future weeks: the blanket "fail on any TODO" gate is an anti-pattern worth avoiding when writing new graders from build_assignment_repo.md — prefer the pattern checks (which already prove the real work) plus a start-of-line placeholder check.

🤖 Generated with Claude Code

…alified views

Two grader bugs in .hyf/test.sh were failing correct submissions:

1. file_is_filled() failed any file containing the word "TODO", so students
   who left the scaffold's "-- TODO:" guide comments above their finished,
   correct queries were scored as empty stubs. Now SQL line-comments are
   stripped before the check and only a line whose content *begins* with TODO
   (an unreplaced markdown placeholder) marks a file as a stub. The instruction
   line "...Replace every TODO." no longer trips it.

2. The vw_dim_zones / vw_fact_trips detection grepped for "VIEW <name>" and
   missed schema-qualified names like "VIEW dev_halyna.vw_dim_zones" — i.e. it
   penalized the own-schema pattern the assignment requires. The regex now
   allows an optional "<schema>." prefix.

Verified against the 7 open student PRs: scores move from 15/30/15/15/85/30/95
to 100/100/95/95/100/100/95. Genuine gaps are still caught (unfilled AI
reflection prompts in two PRs, a misnamed screenshot in one).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lassebenni lassebenni merged commit d4a8283 into main Jul 6, 2026
@lassebenni lassebenni deleted the fix/autograder-todo-and-schema-qualified-views branch July 6, 2026 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant