Human in the loop assignment submission: jdvakil

[Jdvakil]

No description provided.

[Copilot]

Pull request overview

This PR significantly expands the course infrastructure by adding a structured pytest-based grading/public test suite for Scratch-1, updating Scratch-1 synthetic action generation for improved learnability, and introducing new site features (textbook index styling, audit review-mode UI, KaTeX handling, and a “Top Reviewers” dashboard) along with CI/deploy workflow changes.

Changes:

• Add public + internal pytest suites (markers, fixtures, documentation) for Scratch-1 grading and student self-checks.
• Update Scratch-1 dataset generation to use a structured 256-bin action encoding (direction octant + magnitude bin).
• Add site/UI features for audits/textbook and a reviewer leaderboard backed by generated GitHub stats.

Reviewed changes

Copilot reviewed 56 out of 70 changed files in this pull request and generated 13 comments.

Show a summary per file

File	Description
tests/README.md	New documentation for public/internal test suites and usage.
tests/public/test_scratch1_basic.py	Adds public-facing Scratch-1 smoke tests for data gen and provided modules.
tests/public/init.py	Declares public tests package.
tests/internal/test_scratch1_rigor.py	Adds internal rigorous grading tests (imports, attention, RoPE, training).
tests/internal/init.py	Declares internal tests package.
tests/conftest.py	Shared pytest fixtures + attempted solution injection/reset logic.
tests/init.py	Declares overall tests package and structure.
pytest.ini	Adds pytest discovery + marker definitions.
src/assignments/scratch-1/generate_data.py	Updates action encoding + trajectory generation logic for learnability.
src/assignments/scratch-1/backbone.py	Minor typing import formatting change (trailing whitespace).
scripts/README.md	Documents CI/CD scripts and expectations.
scripts/collect_reviewer_data.py	New script to collect GitHub PR comment stats and generate reviewer leaderboard JSON.
scripts/audit_linter.py	Adds stronger MDX frontmatter validation (required fields + non-placeholder checks).
data/reviewer-stats.json	Adds generated reviewer statistics used by the new dashboard.
package.json	Adds recharts dependency for charts.
app/course/top-reviewers/ReviewerCharts.tsx	Client-side Recharts visualizations for reviewer stats.
app/course/top-reviewers/page.tsx	Server page reading reviewer-stats.json and rendering leaderboard.
app/course/page.tsx	Adds link/section entry to the “Top Reviewers” page.
app/course/assignments/capstone/[handle]/page.tsx	Adds dynamic capstone report page rendering MDX per student handle.
content/course/submissions/human-in-the-loop/.gitkeep	Adds placeholder directory for human-in-the-loop submissions.
content/course/assignments/scratch-1.mdx	Updates Scratch-1 assignment expectations (loss convergence guidance, removes draft banner).
content/course/assignments/capstone/zlaouar/Report.mdx	Adds capstone running log page.
content/course/assignments/capstone/yuni-wyx/Report.mdx	Adds capstone running log page.
content/course/assignments/capstone/yi-shiuan-tung/Report.mdx	Adds capstone running log page.
content/course/assignments/capstone/Tr0612/Report.mdx	Adds capstone running log page.
content/course/assignments/capstone/Soorej30/Report.mdx	Adds capstone running log page.
content/course/assignments/capstone/lorinachey/Report.mdx	Adds capstone running log page.
content/course/assignments/capstone/krusnim/Report.mdx	Adds capstone running log page.
content/course/assignments/capstone/kalhamilton/Report.mdx	Adds capstone running log page.
content/course/assignments/capstone/jt7347/Report.mdx	Adds capstone running log page.
content/course/assignments/capstone/jdvakil/Report.mdx	Adds capstone running log page.
content/course/assignments/capstone/himanshugupta1009/Report.mdx	Adds capstone running log page.
content/course/assignments/capstone/Hhy903/Report.mdx	Adds capstone running log page.
content/course/assignments/capstone/gyanigk/Report.mdx	Adds capstone running log page.
content/course/assignments/capstone/carson-jay/Report.mdx	Adds capstone running log page.
content/course/assignments/capstone/aritrach/Report.mdx	Adds capstone running log page.
content/course/assignments/capstone/antony-zhao/Report.mdx	Adds capstone running log page.
app/textbook/page.tsx	Adds a styled textbook index/landing page.
components/textbook/Sidebar.tsx	Updates textbook sidebar styling and adds “Living Textbook” link.
components/textbook/TextbookLayout.tsx	Updates textbook layout right-side TOC styling.
components/audit/AuditLayout.tsx	Adds audit review-mode banner and restyles audit layout.
app/textbook/audits/[...slug]/page.tsx	Adds review-mode wiring, KaTeX handling, and restyles audit header/banner.
components/KatexStyles.tsx	Adds client-side KaTeX CSS injection via CDN.
app/layout.tsx	Reorders KaTeX CSS import to precede globals.css.
app/globals.css	Adds KaTeX and prose styling improvements (code blocks, headings, tables, etc.).
tailwind.config.ts	Extends typography CSS overrides (links, headings, code, KaTeX display, etc.).
README.md	Removes a large “Repository Structure / Workflow” section.
pyproject.toml	Adds Python project metadata, dependencies, and pytest config (duplicated with pytest.ini).
.gitignore	Replaces “private repo files” ignores with narrower ignores and claude output patterns.
.continueignore	Adds ignore patterns for Continue/agent tooling.
.github/workflows/vla-audit.yml	Updates audit CI guidance text; removes shadow test dispatch job.
.github/workflows/shadow-tester.yml	Adds repository_dispatch-driven workflow to run internal tests from a private repo.
.github/workflows/refresh-reviewer-data.yml	Adds scheduled workflow to refresh reviewer stats, open PR, auto-merge, and trigger deploy.
.github/workflows/deploy.yml	Adjusts deploy trigger rules and remote deployment details (host/path/npm install).
.github/workflows/deploy-staging.yml	Adjusts staging deploy host and paths-ignore rules.

Files not reviewed (1)

• pnpm-lock.yaml: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

rehype-katex is configured with trust: true, which enables KaTeX trusted commands (e.g., \href / HTML-like macros). Since audit content is effectively user-authored (students), this increases XSS/phishing surface. Consider keeping trust disabled, or providing a restrictive trust callback that only allows a minimal safe subset of commands.

[Copilot]

The causal mask passed to student attention implementations is always a float 0/1 matrix. Some correct implementations expect a boolean mask (and will do ~mask), which would raise a TypeError and make this test harness unnecessarily brittle. Consider trying both dtypes (e.g., bool then float) or normalizing to the starter-code convention before calling forward().

[Copilot]

There are trailing spaces on this import line. Please remove them to avoid formatting/lint noise and accidental diffs.

[Copilot]

The PR title suggests a narrow change (“Human in the loop assignment submission”), but this diff includes large additions/changes across testing infrastructure, workflows, deployment, UI styling, and new course pages. Please update the PR title/description to reflect the actual scope, or split into smaller PRs to make review and rollback safer.

[Copilot]

KaTeX CSS is already imported globally in app/layout.tsx. Loading it again from a CDN here adds an external runtime dependency and risks duplicate styles / version skew (the existing selector won’t match Next’s bundled CSS href). Prefer removing this component and relying on the bundled KaTeX CSS import.

Suggested change

	'use client';
	import { useEffect } from 'react';
	export function KatexStyles() {
	useEffect(() => {
	// Ensure KaTeX CSS is loaded
	const link = document.createElement('link');
	link.rel = 'stylesheet';
	link.href = 'https://cdn.jsdelivr.net/npm/katex@0.16.27/dist/katex.min.css';
	link.integrity = 'sha384-yp+jpRNKIa0xGrYaVtwImDXkFq7ZOCV5kJZVDg/uAFfYPmtFcKr0sxhVJy1HqnWD';
	link.crossOrigin = 'anonymous';
	// Check if already loaded
	const existing = document.querySelector('link[href*="katex"]');
	if (!existing) {
	document.head.appendChild(link);
	}
	}, []);
	export function KatexStyles() {
	// KaTeX CSS is imported globally in app/layout.tsx; no runtime work is needed here.

[Copilot]

This fixture tries to inject/reset solutions via scripts/dev_utils.py, but that file does not exist in the repository, so solution injection will always be skipped. Either add scripts/dev_utils.py (or switch to an existing script) or adjust the injection logic/docs to reflect the real workflow.

[Copilot]

Pytest configuration is duplicated between this [tool.pytest.ini_options] block and the new pytest.ini. Pytest will only use one config source, so the duplicate can drift and confuse contributors. Prefer consolidating into a single source of truth (either keep pytest.ini or keep the pyproject section).

[Copilot]

This parameter-count expectation assumes both Linear layers have biases (it adds hidden_dim + dim), but the provided FeedForward implementation in src/assignments/scratch-1/backbone.py constructs both nn.Linear(..., bias=False). As written, this test will fail on the starter code. Update the expected count to match the actual layer definitions (or compute it dynamically based on whether biases are enabled).

[Copilot]

This README references scripts/manage_solutions.py, but that script isn’t present in the repository (scripts/ currently contains audit_linter.py, collect_reviewer_data.py, deploy.sh, grade_scratch1.py, review-prs.sh). Either add manage_solutions.py or update this documentation and any callers (e.g., tests/conftest.py) to point to the actual solution-management script.

[Copilot]

This workflow only runs on repository_dispatch (type run-shadow-tests), but this repo no longer contains any workflow/job that emits that dispatch event (the prior trigger in vla-audit.yml was removed). If shadow testing is meant to run automatically, consider re-adding a safe dispatch trigger or adding workflow_dispatch / pull_request triggers; otherwise this file will never run in normal CI.

Suggested change

	on:
	on:
	workflow_dispatch: