Skip to content

Commit fc8633a

Browse files
thomasahleclaude
andcommitted
Add Runtime Assertions chapter, fix npm test CI failure
- Add vitest + `npm test` script to package.json / vitest.config.js - Add unit tests: worker-shim.test.js, lec-model.test.js - New SVA chapter: Runtime Assertions - New lesson: Concurrent Assertions in Simulation (concurrent-sim) - Move vacuous-pass and $isunknown into Runtime Assertions - Rename 'Your First Assertion' → 'Your First Formal Assertion' - immediate-assert stays bmc runner; gains simulation→formal bridge paragraph - Move gear button next to split-view toggle in split editor header - Update e2e/solutions.spec.js to match new chapter/lesson structure - Add e2e test suite files (solutions, formal, lessons, waveform, etc.) - Add playwright.config.js, scripts, and CI workflow Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 7c40160 commit fc8633a

144 files changed

Lines changed: 4709 additions & 1471 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.env.example

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,11 @@ VITE_CIRCT_SIM_JS_URL=/circt/circt-sim.js
66
VITE_CIRCT_SIM_WASM_URL=/circt/circt-sim.wasm
77
VITE_CIRCT_BMC_JS_URL=/circt/circt-bmc.js
88
VITE_CIRCT_BMC_WASM_URL=/circt/circt-bmc.wasm
9+
# VPI-capable circt-sim (for cocotb lessons — built with Asyncify + VPI exports):
10+
VITE_CIRCT_SIM_VPI_JS_URL=/circt/circt-sim-vpi.js
11+
VITE_CIRCT_SIM_VPI_WASM_URL=/circt/circt-sim-vpi.wasm
12+
# Pyodide CDN URL (used by cocotb runner):
13+
# VITE_PYODIDE_URL=https://cdn.jsdelivr.net/pyodide/v0.27.0/full/pyodide.js
914
#
1015
# Optional args for each stage (JSON array preferred):
1116
# VITE_CIRCT_VERILOG_ARGS=[\"--ir-llhd\",\"--timescale\",\"1ns/1ns\",\"--single-unit\"]

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,5 @@ public/surfer/
1313
*.pdf
1414
.playwright-mcp/
1515
playwright-console.log
16+
playwright-report/
17+
test-results/

.nvmrc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
22

README.md

Lines changed: 96 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -2,60 +2,110 @@
22

33
Svelte-based interactive tutorial shell for SystemVerilog, SVA, and UVM.
44

5-
## CIRCT Fork
5+
## Reproducible Requirements
66

7-
This project is pinned to the CIRCT fork with wasm support:
7+
Pinned versions are centralized in `scripts/toolchain.lock.sh`:
88

9-
- `git@github.com:thomasnormal/circt.git`
9+
- Node major: `22`
10+
- Emscripten (emsdk): `4.0.21`
11+
- CIRCT repo: `https://github.com/thomasnormal/circt.git`
12+
- CIRCT ref: `8e8ca87dcda1c8abd47103ae7789c8ed261d5de3`
13+
- LLVM submodule ref: `972cd847efb20661ea7ee8982dd19730aa040c75`
14+
15+
Host tools:
16+
17+
- `git`
18+
- `node`/`npm` (Node 22.x)
19+
- `cmake` (>= 3.24)
20+
- `ninja` (>= 1.10)
21+
- `python3` (>= 3.10)
22+
- `rsync`
23+
- `unzip`
24+
25+
Check prerequisites:
26+
27+
- `npm run check:req` (frontend/tooling only)
28+
- `npm run check:req:wasm` (includes emsdk/emcc verification)
29+
30+
## Reproducible Local Build
31+
32+
1. Activate emsdk `4.0.21` (or ensure `emcc -v` reports `4.0.21`).
33+
2. Run the one-shot bootstrap:
34+
- `npm run bootstrap:repro`
35+
36+
Equivalent step-by-step:
37+
38+
1. `npm ci`
39+
2. `scripts/setup-surfer.sh`
40+
3. `npm run setup:circt`
41+
4. `npm run build:circt`
42+
5. `npm run sync:circt`
43+
6. `npm run build`
44+
45+
The CIRCT wasm build script enforces conservative resource limits by default:
46+
47+
- Ninja parallelism: `CIRCT_WASM_JOBS=4`
48+
- Configure timeout: `CIRCT_WASM_CONFIGURE_TIMEOUT_MIN=30`
49+
- Build timeout: `CIRCT_WASM_BUILD_TIMEOUT_MIN=120`
50+
- Force clean build dir first: `CIRCT_WASM_CLEAN_BUILD=1`
51+
- Build targets override: `CIRCT_WASM_TARGETS="circt-verilog circt-sim circt-bmc"`
52+
- Optional virtual memory cap: `CIRCT_WASM_MEMORY_LIMIT_KB=<kb>`
53+
54+
To also build `circt-sim-vpi` (required for cocotb lessons):
55+
56+
```
57+
CIRCT_WASM_TARGETS="circt-verilog circt-sim circt-bmc circt-sim-vpi" npm run build:circt
58+
```
1059

1160
## Scripts
1261

13-
- `npm install`
1462
- `npm run dev`
15-
- `npm run build`
1663
- `npm run preview`
17-
- `scripts/setup-circt.sh` (clone/update the CIRCT fork checkout)
18-
19-
## CIRCT WASM Runtime Setup
20-
21-
1. Clone/update the fork checkout:
22-
- `scripts/setup-circt.sh`
23-
2. Build wasm artifacts from that checkout.
24-
3. Copy artifacts into this app:
25-
- `cp vendor/circt/build-wasm/bin/circt-verilog.js public/circt/circt-verilog.js`
26-
- `cp vendor/circt/build-wasm/bin/circt-verilog.wasm public/circt/circt-verilog.wasm`
27-
- `cp vendor/circt/build-wasm/bin/circt-sim.js public/circt/circt-sim.js`
28-
- `cp vendor/circt/build-wasm/bin/circt-sim.wasm public/circt/circt-sim.wasm`
29-
- `cp vendor/circt/build-wasm/bin/circt-bmc.js public/circt/circt-bmc.js`
30-
- `cp vendor/circt/build-wasm/bin/circt-bmc.wasm public/circt/circt-bmc.wasm`
31-
4. Runtime artifact lookup order:
32-
- `public/circt/circt-verilog.js`
33-
- `public/circt/circt-verilog.wasm`
34-
- `public/circt/circt-sim.js`
35-
- `public/circt/circt-sim.wasm`
36-
- `public/circt/circt-bmc.js`
37-
- `public/circt/circt-bmc.wasm`
38-
5. Optional mock shim (explicit opt-in only):
39-
- `public/circt/circt.js`
40-
- `public/circt/circt.wasm`
41-
- not used by default runtime
42-
6. Optional URL overrides in `.env`:
43-
- `VITE_CIRCT_VERILOG_JS_URL`
44-
- `VITE_CIRCT_VERILOG_WASM_URL`
45-
- `VITE_CIRCT_SIM_JS_URL`
46-
- `VITE_CIRCT_SIM_WASM_URL`
47-
- `VITE_CIRCT_BMC_JS_URL`
48-
- `VITE_CIRCT_BMC_WASM_URL`
49-
- `VITE_CIRCT_VERILOG_ARGS`
50-
- `VITE_CIRCT_SIM_ARGS`
51-
- `VITE_CIRCT_BMC_ARGS`
52-
53-
## Notes
54-
55-
- This repo currently includes a starter tutorial UI and lesson data model.
64+
- `npm run test:e2e`
65+
- `npm run setup:circt`
66+
- `npm run build:circt`
67+
- `npm run sync:circt`
68+
- `npm run bootstrap:repro`
69+
70+
## Optional Runtime Overrides
71+
72+
In `.env` (copy `.env.example`):
73+
74+
- `VITE_CIRCT_VERILOG_JS_URL`
75+
- `VITE_CIRCT_VERILOG_WASM_URL`
76+
- `VITE_CIRCT_SIM_JS_URL`
77+
- `VITE_CIRCT_SIM_WASM_URL`
78+
- `VITE_CIRCT_BMC_JS_URL`
79+
- `VITE_CIRCT_BMC_WASM_URL`
80+
- `VITE_CIRCT_SIM_VPI_JS_URL` (cocotb lessons)
81+
- `VITE_CIRCT_SIM_VPI_WASM_URL` (cocotb lessons)
82+
- `VITE_PYODIDE_URL` (cocotb lessons, default: jsdelivr CDN)
83+
- `VITE_CIRCT_VERILOG_ARGS`
84+
- `VITE_CIRCT_SIM_ARGS`
85+
- `VITE_CIRCT_BMC_ARGS`
86+
87+
## Runtime Notes
88+
5689
- Runtime uses a real 2-stage wasm toolchain by default:
5790
- `circt-verilog` lowers SV/SVA/UVM source to MLIR
5891
- `circt-sim` executes lowered MLIR and emits VCD for the waveform pane
5992
- Tool invocations run in isolated Web Workers to avoid global Emscripten symbol collisions and re-entry issues.
60-
- The UI includes a `self-check` action in the Runtime panel to validate artifact compatibility.
61-
- Waveform pane is metadata-driven (`off`, `optional`, `required`) and prepared for Surfer integration.
93+
- UI includes a `self-check` action in the Runtime panel to validate artifact compatibility.
94+
- Waves tab appears automatically only when a valid VCD is generated.
95+
96+
## CI
97+
98+
- `.github/workflows/ci.yml` builds with pinned toolchain requirements:
99+
1. install Node 22
100+
2. install emsdk 4.0.21
101+
3. run `scripts/setup-circt.sh` (pinned CIRCT ref)
102+
4. run `scripts/build-circt-wasm.sh`
103+
5. run `npm run sync:circt`
104+
6. run `npm run build`
105+
106+
## E2E
107+
108+
- `npm run test:e2e -- e2e/waveform.spec.js`
109+
- Includes:
110+
- non-mock pipeline check (`circt-verilog` + `circt-sim`, waveform generated)
111+
- Surfer waveform render check when browser WebGL support is available in the test environment
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Browser-Worker UVM Repro (`Malformed attribute storage object`)
2+
3+
This reproduces the CIRCT wasm UVM failure in the **browser worker** path
4+
without using lesson UI clicks.
5+
6+
## Preconditions
7+
8+
1. `vendor/circt` is checked out and wasm artifacts are built.
9+
2. Artifacts are synced into `public/circt`:
10+
11+
```bash
12+
scripts/setup-circt.sh
13+
npm run sync:circt
14+
```
15+
16+
## Repro command
17+
18+
```bash
19+
node scripts/repro-uvm-browser-worker-assert.mjs
20+
```
21+
22+
## What it does
23+
24+
- starts Vite dev server on `http://127.0.0.1:4173` (`--strictPort`)
25+
- opens Chromium headless via Playwright
26+
- imports `createCirctWasmAdapter` from `/src/runtime/circt-adapter.js`
27+
- runs compile-only (`simulate: false`) on minimal UVM files:
28+
- `/src/my_test.sv`
29+
- `/src/tb_top.sv`
30+
- uses full UVM path (`--uvm-path /circt/uvm-core -I /circt/uvm-core/src`)
31+
32+
## Expected failure signature
33+
34+
- run result `ok=false`
35+
- logs include both:
36+
- `Aborted(`
37+
- `Malformed attribute storage object`
38+
- logs do **not** reach `$ circt-sim`
39+
40+
If this signature appears, the script exits `0` and prints:
41+
42+
`REPRODUCED: browser-worker UVM compile hit malformed-attribute abort`
43+
44+
If not, it exits non-zero.

e2e/cocotb.spec.js

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
/**
2+
* E2E tests for the cocotb runner (circt-verilog → circt-sim + Pyodide).
3+
*/
4+
import { test, expect } from '@playwright/test';
5+
6+
async function goToLesson(page, lessonName) {
7+
await page.goto('/');
8+
await page.getByRole('button', { name: 'cocotb Basics' }).click();
9+
await page.getByRole('button', { name: new RegExp(lessonName) }).click();
10+
await expect(page.getByRole('heading', { level: 2, name: lessonName })).toBeVisible();
11+
}
12+
13+
test('cocotb: shows "test" button, not "run"', async ({ page }) => {
14+
await goToLesson(page, 'Your First cocotb Test');
15+
await expect(page.getByTestId('run-button')).toHaveText('test');
16+
});
17+
18+
test('cocotb: Your First cocotb Test — solution passes all assertions', async ({ page }) => {
19+
await goToLesson(page, 'Your First cocotb Test');
20+
21+
// The Python test file is the starter; apply the SV solution (correct adder).
22+
await page.getByTestId('solve-button').click();
23+
await page.getByTestId('run-button').click();
24+
25+
const logs = page.getByTestId('runtime-logs');
26+
// Pyodide download can be slow on first run — generous timeout.
27+
await expect(logs).toContainText('[cocotb] Simulation complete', { timeout: 180_000 });
28+
await expect(logs).not.toContainText('[cocotb] Python error');
29+
await expect(logs).not.toContainText('# cocotb error');
30+
await expect(logs).not.toContainText('AssertionError');
31+
});
32+
33+
test('cocotb: Clock and Timing — solution passes all assertions', async ({ page }) => {
34+
await goToLesson(page, 'Clock and Timing');
35+
36+
await page.getByTestId('solve-button').click();
37+
await page.getByTestId('run-button').click();
38+
39+
const logs = page.getByTestId('runtime-logs');
40+
await expect(logs).toContainText('[cocotb] Simulation complete', { timeout: 180_000 });
41+
await expect(logs).not.toContainText('[cocotb] Python error');
42+
await expect(logs).not.toContainText('AssertionError');
43+
});

e2e/debug-cocotb.spec.js

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
import { test, expect } from '@playwright/test';
2+
3+
test('cocotb: debug - check wasmTable', async ({ page }) => {
4+
const consoleLogs = [];
5+
page.on('console', msg => {
6+
const text = `[${msg.type()}] ${msg.text()}`;
7+
consoleLogs.push(text);
8+
});
9+
10+
await page.goto('/');
11+
await page.getByRole('button', { name: 'cocotb Basics' }).click();
12+
await page.getByRole('button', { name: /Your First cocotb Test/ }).click();
13+
await page.getByTestId('solve-button').click();
14+
await page.getByTestId('run-button').click();
15+
16+
await page.waitForTimeout(20000);
17+
18+
const logs = page.getByTestId('runtime-logs');
19+
const text = await logs.textContent();
20+
21+
console.log('=== Runtime logs ===');
22+
console.log(text);
23+
console.log('=== Console logs (last 40) ===');
24+
consoleLogs.slice(-40).forEach(l => console.log(l));
25+
26+
expect(text).not.toContain('# cocotb error');
27+
});

e2e/formal.spec.js

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
import { test, expect } from '@playwright/test';
2+
3+
// Helper: expand a chapter in the sidebar and click a lesson.
4+
async function goToLesson(page, chapterName, lessonName) {
5+
await page.goto('/');
6+
await page.getByRole('button', { name: chapterName }).click();
7+
await page.getByRole('button', { name: lessonName }).click();
8+
await expect(page.getByRole('heading', { level: 2, name: lessonName })).toBeVisible();
9+
}
10+
11+
// ── BMC: Bounded Model Checking ───────────────────────────────────────────────
12+
13+
test('BMC: starter code has no assertion → verify does not prove anything', async ({ page }) => {
14+
await goToLesson(page, 'Implication & BMC', 'Bounded Model Checking');
15+
16+
await page.getByTestId('verify-button').click();
17+
18+
const logs = page.getByTestId('runtime-logs');
19+
await expect(logs).toContainText('$ circt-verilog', { timeout: 120_000 });
20+
await expect(logs).toContainText('$ circt-bmc', { timeout: 120_000 });
21+
// No assertion in the module → bmc may exit with 0 but won't say "proved"
22+
await expect(logs).not.toContainText('# circt-verilog exit code: 1');
23+
});
24+
25+
test('BMC: solution proves all properties within the bound', async ({ page }) => {
26+
await goToLesson(page, 'Implication & BMC', 'Bounded Model Checking');
27+
28+
await page.getByTestId('solve-button').click();
29+
await page.getByTestId('verify-button').click();
30+
31+
const logs = page.getByTestId('runtime-logs');
32+
await expect(logs).toContainText('$ circt-verilog', { timeout: 120_000 });
33+
await expect(logs).toContainText('$ circt-bmc', { timeout: 120_000 });
34+
await expect(logs).toContainText('[z3] unsat', { timeout: 120_000 });
35+
await expect(logs).not.toContainText('# circt-verilog exit code: 1');
36+
await expect(logs).not.toContainText('# circt-bmc exit code: 1');
37+
});
38+
39+
test('BMC: shows only verify button, no run button', async ({ page }) => {
40+
await goToLesson(page, 'Implication & BMC', 'Bounded Model Checking');
41+
42+
await expect(page.getByTestId('verify-button')).toBeVisible();
43+
await expect(page.getByTestId('run-button')).toHaveCount(0);
44+
});
45+
46+
// ── BMC: assume property ──────────────────────────────────────────────────────
47+
48+
test('assume property: solution proves property with constraint', async ({ page }) => {
49+
await goToLesson(page, 'Formal Verification', 'assume property');
50+
51+
await page.getByTestId('solve-button').click();
52+
await page.getByTestId('verify-button').click();
53+
54+
const logs = page.getByTestId('runtime-logs');
55+
await expect(logs).toContainText('[z3] unsat', { timeout: 120_000 });
56+
});
57+
58+
// ── LEC: Logical Equivalence Checking ────────────────────────────────────────
59+
60+
test('LEC: only shows verify (LEC) button, no run button', async ({ page }) => {
61+
await goToLesson(page, 'Formal Verification', 'Logical Equivalence Checking');
62+
63+
await expect(page.getByTestId('verify-button')).toBeVisible();
64+
await expect(page.getByTestId('verify-button')).toHaveText('verify (LEC)');
65+
await expect(page.getByTestId('run-button')).toHaveCount(0);
66+
});
67+
68+
test('LEC: buggy Impl is detected as NOT equivalent', async ({ page }) => {
69+
await goToLesson(page, 'Formal Verification', 'Logical Equivalence Checking');
70+
71+
await page.getByTestId('verify-button').click();
72+
73+
const logs = page.getByTestId('runtime-logs');
74+
await expect(logs).toContainText('$ circt-verilog', { timeout: 120_000 });
75+
await expect(logs).toContainText('$ circt-lec', { timeout: 120_000 });
76+
await expect(logs).toContainText('[z3] sat', { timeout: 120_000 });
77+
await expect(logs).not.toContainText('# circt-verilog exit code: 1');
78+
});
79+
80+
test('LEC: fixed Impl is proved equivalent to Spec', async ({ page }) => {
81+
await goToLesson(page, 'Formal Verification', 'Logical Equivalence Checking');
82+
83+
await page.getByTestId('solve-button').click();
84+
await page.getByTestId('verify-button').click();
85+
86+
const logs = page.getByTestId('runtime-logs');
87+
await expect(logs).toContainText('[z3] unsat', { timeout: 120_000 });
88+
await expect(logs).not.toContainText('# circt-verilog exit code: 1');
89+
await expect(logs).not.toContainText('# circt-lec exit code: 1');
90+
});

0 commit comments

Comments
 (0)