Skip to content

Commit 158bea0

Browse files
committed
Raise structural examples toward quality target
1 parent bf2f2bb commit 158bea0

24 files changed

Lines changed: 7356 additions & 6793 deletions

.github/workflows/preview-viz.yml

Lines changed: 1 addition & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -49,26 +49,7 @@ jobs:
4949
--message "${{ github.sha }}" \
5050
--json
5151
- name: Smoke test deployed Preview
52-
run: |
53-
set -euo pipefail
54-
base="https://viz-pythonbyexample.adewale-883.workers.dev"
55-
for path in \
56-
"/" \
57-
"/examples/values" \
58-
"/examples/async-await" \
59-
"/examples/networking" \
60-
"/examples/subprocesses" \
61-
"/journeys/workers" \
62-
"/prototyping/journey-figures-gestalt"; do
63-
url="${base}${path}"
64-
echo "Checking ${url}"
65-
curl --fail --show-error --silent --location --output /tmp/preview-smoke.html --write-out "%{http_code} %{url_effective}\n" "${url}"
66-
if grep -qiE "error code: 1101|PythonError|Traceback" /tmp/preview-smoke.html; then
67-
echo "Preview rendered an exception for ${url}"
68-
head -200 /tmp/preview-smoke.html
69-
exit 1
70-
fi
71-
done
52+
run: scripts/smoke_deployment.py https://viz-pythonbyexample.adewale-883.workers.dev
7253
- name: Dump wrangler logs on failure
7354
if: failure()
7455
run: |

docs/lessons-learned.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,7 @@ git diff --check
112112
- **Lines must terminate AT elements, not in their gaps or interiors.** A 1.5px gap between a tree edge and a leaf dot reads as "the tree is disconnected" (the `exception-group-peel` bug). A line endpoint 2px inside a circle reads as "the arrow pierces the node" (the `context-bowtie` bug). When connecting to a dot, end the line at the dot's centre and let the dot draw on top — the visual termination is the circumference, with zero gap or overshoot.
113113
- **Journey pages render section figures inline.** `SECTION_FIGURES` lives in `src/marginalia.py` (single source of truth, keyed by section title) and `render_for_section(title)` is invoked from `render_journey_page` between each section's meta and its example list. The same paint code that produces the `/prototyping/journey-figures-gestalt` review page renders on production journey pages; drift between the two is structurally impossible. Contract 10 asserts every section in `JOURNEYS` has a figure and every figure name resolves.
114114
- **An explicit comparison loop should iterate over the topic's whole spectrum.** When a cell teaches by doing `for label, value in [(...), (...)]: print(...)`, the bracketed list IS the lesson. Two items is a binary contrast; three items reads as a progression. The strings example presented English (pure ASCII, 1 byte/char) against Thai (3 bytes/char) but skipped the Latin-extended middle (French `café`: 4 code points, 5 bytes — `é` is 2 UTF-8 bytes). Adding the middle row turned the cell from "ASCII vs non-Latin" into "1-byte / 2-byte / 3-byte progression." The rule is narrow — most examples spread categories across cells, which is also a valid pattern — but when a comparison loop exists, fill it with the topic's actual spectrum, not just the endpoints.
115-
- **Quality debt must be tracked, not normalized away.** `docs/example-quality-rubric.md` sets a 9.0 target and `scripts/check_quality_scores.py` enforces the score registry: pages below the hard minimum need a concrete improvement backlog entry, and Hello World is the only standing waiver because first examples are traditionally tiny. A score below target is allowed only when the remaining work is named.
115+
- **Quality debt must be tracked, not normalized away.** `docs/example-quality-rubric.md` sets a 9.0 target and `scripts/check_quality_scores.py` enforces the score registry: pages below the hard minimum need a concrete improvement backlog entry, stale backlog entries fail once a page clears the gate, and Hello World is the only standing waiver because first examples are traditionally tiny. A score below target is allowed only when the remaining work is named.
116116
- **No-figure decisions need a registry.** Some examples should not have figures, but that cannot be an invisible omission. `scripts/check_no_figure_rationales.py` validates `no_figure_rationales` so future constraint-shaped pages can opt out explicitly instead of shipping weak diagrams.
117117
- **Journey sections need outcome contracts.** `scripts/check_journey_outcomes.py` ties each journey section to learner outcomes and support examples so journey pages stay mental maps rather than catalog slices.
118-
- **Deployment smoke belongs beside CI.** `scripts/smoke_deployment.py` checks rendered Worker pages, runtime-boundary pages, journey pages, and prototype review pages for HTTP failures and exception markers. Build success is not enough; the deployed Worker must render.
118+
- **Deployment smoke belongs beside CI.** `scripts/smoke_deployment.py` checks rendered Worker pages, runtime-boundary pages, journey pages, prototype review pages, and representative Dynamic Worker POST runs for HTTP failures, exception markers, and stale edited-code output. Build success is not enough; the deployed Worker must render and execute edited examples.

docs/quality-registries.toml

Lines changed: 0 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -204,38 +204,6 @@ next_action = "add a cell contrasting naming convention with typing.Final and ru
204204
cause = "truth-value protocol is under-linked to booleans and special methods"
205205
next_action = "add a boundary cell that predicts bool() for empty containers, None, and custom __bool__"
206206

207-
[quality_improvement_backlog.object-lifecycle]
208-
cause = "single-cell page compresses references, identity, and garbage collection"
209-
next_action = "split into reference lifetime, identity after rebinding, and cleanup boundary cells"
210-
211-
[quality_improvement_backlog.guard-clauses]
212-
cause = "single-cell page does not show the before/after flattening payoff"
213-
next_action = "add a nested conditional contrast beside the guard-clause version"
214-
215-
[quality_improvement_backlog.for-loops]
216-
cause = "foundational loop page lacks enough protocol and alternative framing"
217-
next_action = "add a cell contrasting direct iteration, range(), and enumerate() with links to iterators"
218-
219-
[quality_improvement_backlog.sentinel-iteration]
220-
cause = "single-cell page hides the read-until-sentinel problem shape"
221-
next_action = "add a callable state cell and a while-loop contrast"
222-
223-
[quality_improvement_backlog.collections-module]
224-
cause = "aggregator page names a broad standard-library surface but has one cell"
225-
next_action = "turn it into a map cell for Counter, defaultdict, deque, and namedtuple or split pages"
226-
227-
[quality_improvement_backlog.copying-collections]
228-
cause = "important shallow/deep footgun has one cell despite nested-state boundary"
229-
next_action = "split into aliasing, shallow copy, and deepcopy cells"
230-
231-
[quality_improvement_backlog.partial-functions]
232-
cause = "single-cell page does not show the before/after callable API pressure"
233-
next_action = "add a repeated-argument function call before partial() and a callable-object neighbor"
234-
235-
[quality_improvement_backlog.warnings]
236-
cause = "single-cell page misses filter/escalate/ignore boundaries"
237-
next_action = "add cells for warn(), simplefilter('error'), and when logging/exceptions are better"
238-
239207
[quality_improvement_backlog.virtual-environments]
240208
cause = "runtime-boundary page is constrained by Dynamic Workers and needs stronger standard-Python path"
241209
next_action = "teach venv creation/activation as unsupported Standard Python, then show local dependency evidence"
@@ -248,30 +216,10 @@ next_action = "split value restriction from rebinding restriction and show runti
248216
cause = "single-cell advanced typing page hides why ordinary Callable loses parameter shape"
249217
next_action = "add a decorator typed with Callable[..., T] before the ParamSpec-preserving version"
250218

251-
[quality_improvement_backlog.overloads]
252-
cause = "single-cell page needs static-only overload declarations vs one runtime implementation"
253-
next_action = "add cells for overload stubs, implementation, and runtime annotations"
254-
255219
[quality_improvement_backlog.number-parsing]
256220
cause = "parsing page lacks enough failure/recovery shape"
257221
next_action = "add cells for int base handling, ValueError recovery, and validation boundary"
258222

259-
[quality_improvement_backlog.subprocesses]
260-
cause = "runtime-boundary page needs clearer standard subprocess use before portable evidence"
261-
next_action = "show standard subprocess.run contract once, then Dynamic Worker boundary and captured-output surrogate"
262-
263-
[quality_improvement_backlog.threads-and-processes]
264-
cause = "runtime-boundary page blends thread/process concepts with environment caveat"
265-
next_action = "split shared-memory thread idea, process isolation idea, and Worker unsupported boundary"
266-
267-
[quality_improvement_backlog.networking]
268-
cause = "runtime-boundary page needs stronger protocol-vs-socket contrast"
269-
next_action = "teach request/response byte protocol, show standard socket form once, then local parser surrogate"
270-
271-
[quality_improvement_backlog.csv-data]
272-
cause = "single-cell row-shaped data page lacks dialect/header/error boundaries"
273-
next_action = "add cells for DictReader, writer output, and JSON/regular-expression boundary"
274-
275223
[quality_improvement_backlog.values]
276224
cause = "foundational page is graph-linked now but still needs a sharper object/type mental model"
277225
next_action = "add or revise a cell that connects value, type, and operation with a nearby See also path"
@@ -384,10 +332,6 @@ next_action = "add a small pipeline with take/filter/map output proving values a
384332
cause = "property syntax is shown but method-vs-attribute API boundary needs stronger rationale"
385333
next_action = "add a before/after cell changing a public attribute into a property without caller changes"
386334

387-
[quality_improvement_backlog.descriptors]
388-
cause = "descriptor protocol is compressed into one cell despite being a data-model mechanism"
389-
next_action = "split __get__ lookup from __set__ validation and property relationship"
390-
391335
[quality_improvement_backlog.exceptions]
392336
cause = "try/except structure is shown but bare-except and cleanup boundaries need sharper evidence"
393337
next_action = "add a cell contrasting specific exception handling with overbroad catching"
@@ -400,10 +344,6 @@ next_action = "add a cell comparing Enum identity/name/value with plain strings"
400344
cause = "custom exception class is shown but when not to create one is underdeveloped"
401345
next_action = "add a boundary cell contrasting domain error with built-in ValueError"
402346

403-
[quality_improvement_backlog.logging]
404-
cause = "single-cell infrastructure page misses levels, logger names, and handler/format boundaries"
405-
next_action = "split logger level, message context, and testing/log-capture shape"
406-
407347
[quality_improvement_backlog.datetime]
408348
cause = "date/time operations are shown but timezone and naive-aware boundaries need more clarity"
409349
next_action = "add a timezone-aware datetime cell or explicitly scope the page to naive values"

scripts/check_quality_scores.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,9 @@ def main() -> int:
7070
continue
7171
if slug in waivers:
7272
errors.append(f"{slug} cannot be both waiver and improvement backlog")
73+
score = EXAMPLE_QUALITY_SCORES.get(slug, (0.0, ""))[0]
74+
if score >= hard_min:
75+
errors.append(f"quality backlog {slug} is stale because score is now {score:.1f}")
7376
if not _entry_has_text(backlog[slug], "cause", "next_action"):
7477
errors.append(f"quality backlog {slug} must include cause and next_action")
7578

scripts/smoke_deployment.py

Lines changed: 59 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
import argparse
1010
import sys
1111
import urllib.error
12+
import urllib.parse
1213
import urllib.request
1314
from urllib.parse import urljoin
1415

@@ -21,6 +22,13 @@
2122
"/journeys/workers",
2223
"/prototyping/production-figures-gestalt",
2324
]
25+
POST_SMOKES = [
26+
("values", "print('runtime-smoke-values')\n", "runtime-smoke-values"),
27+
("values", "print('runtime-smoke-values-edited')\n", "runtime-smoke-values-edited"),
28+
("async-await", "import asyncio\n\nasync def main():\n return 'runtime-smoke-async'\n\nprint(asyncio.run(main()))\n", "runtime-smoke-async"),
29+
("networking", "print('runtime-smoke-networking-boundary')\n", "runtime-smoke-networking-boundary"),
30+
("subprocesses", "print('runtime-smoke-subprocess-boundary')\n", "runtime-smoke-subprocess-boundary"),
31+
]
2432
ERROR_MARKERS = ["error code: 1101", "PythonError", "Traceback"]
2533

2634

@@ -31,10 +39,35 @@ def fetch(url: str) -> tuple[int, str]:
3139
return response.status, body
3240

3341

42+
def post_code(url: str, code: str) -> tuple[int, str]:
43+
data = urllib.parse.urlencode({"code": code}).encode()
44+
request = urllib.request.Request(
45+
url,
46+
data=data,
47+
headers={
48+
"User-Agent": "pythonbyexample-smoke/1.0",
49+
"Content-Type": "application/x-www-form-urlencoded",
50+
},
51+
method="POST",
52+
)
53+
with urllib.request.urlopen(request, timeout=30) as response:
54+
body = response.read().decode("utf-8", errors="replace")
55+
return response.status, body
56+
57+
58+
def has_exception_marker(body: str) -> str | None:
59+
lowered = body.lower()
60+
for marker in ERROR_MARKERS:
61+
if marker.lower() in lowered:
62+
return marker
63+
return None
64+
65+
3466
def main() -> int:
3567
parser = argparse.ArgumentParser()
3668
parser.add_argument("base_url", help="deployment origin, e.g. https://www.pythonbyexample.dev")
3769
parser.add_argument("--path", action="append", dest="paths", help="additional path to check")
70+
parser.add_argument("--skip-post", action="store_true", help="check rendered pages only")
3871
args = parser.parse_args()
3972

4073
base = args.base_url.rstrip("/") + "/"
@@ -53,18 +86,37 @@ def main() -> int:
5386
continue
5487
if status != 200:
5588
failures.append(f"{url}: HTTP {status}")
56-
lowered = body.lower()
57-
for marker in ERROR_MARKERS:
58-
if marker.lower() in lowered:
59-
failures.append(f"{url}: rendered exception marker {marker!r}")
60-
break
61-
print(f"{status} {url}")
89+
marker = has_exception_marker(body)
90+
if marker:
91+
failures.append(f"{url}: rendered exception marker {marker!r}")
92+
print(f"GET {status} {url}")
93+
94+
if not args.skip_post:
95+
for slug, code, expected in POST_SMOKES:
96+
url = urljoin(base, f"examples/{slug}")
97+
try:
98+
status, body = post_code(url, code)
99+
except urllib.error.HTTPError as exc:
100+
failures.append(f"POST {url}: HTTP {exc.code}")
101+
continue
102+
except Exception as exc: # pragma: no cover - diagnostic path
103+
failures.append(f"POST {url}: {exc!r}")
104+
continue
105+
if status != 200:
106+
failures.append(f"POST {url}: HTTP {status}")
107+
marker = has_exception_marker(body)
108+
if marker:
109+
failures.append(f"POST {url}: rendered exception marker {marker!r}")
110+
if expected not in body:
111+
failures.append(f"POST {url}: missing edited-code output {expected!r}")
112+
print(f"POST {status} {url} -> {expected}")
62113

63114
if failures:
64115
for failure in failures:
65116
print(failure, file=sys.stderr)
66117
return 1
67-
print(f"Deployment smoke OK ({len(paths)} paths).")
118+
post_count = 0 if args.skip_post else len(POST_SMOKES)
119+
print(f"Deployment smoke OK ({len(paths)} GETs, {post_count} POSTs).")
68120
return 0
69121

70122

src/asset_manifest.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
# Generated by scripts/fingerprint_assets.py. Do not edit by hand.
22
ASSET_PATHS = {'SITE_CSS': '/site.57a55415849b.css', 'SYNTAX_JS': '/syntax-highlight.3b6c7f730d46.js', 'EDITOR_JS': '/editor.a4a7766e1b9b.js'}
3-
HTML_CACHE_VERSION = '1bf90019812a'
3+
HTML_CACHE_VERSION = 'd56bf0e86233'

src/example_sources/collections-module.md

Lines changed: 38 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,19 @@ title = "Collections Module"
44
section = "Collections"
55
summary = "collections provides specialized containers for common data shapes."
66
doc_path = "/library/collections.html"
7+
see_also = [
8+
"dicts",
9+
"lists",
10+
"tuples",
11+
"sets",
12+
]
713
+++
814

9-
collections provides specialized containers for common data shapes. It exists to make a common boundary explicit instead of leaving the behavior implicit in a larger program.
15+
`collections` provides specialized containers for common shapes that would otherwise require repetitive plumbing. Use it when the shape has a name: counting, grouping, queueing, or lightweight records.
1016

11-
Use it when the problem shape matches the example, and prefer simpler neighboring tools when the extra machinery would hide the intent. The notes call out the boundary so the feature stays practical rather than decorative.
17+
These types are not replacements for `list`, `dict`, `tuple`, and `set`. They are small standard-library tools for cases where an ordinary container would hide the intent behind manual bookkeeping.
1218

13-
The example is small, deterministic, and focused on the semantic point. The complete source is editable below, while the walkthrough pairs the source with its output.
19+
The examples below map each type to the question it answers.
1420

1521
:::program
1622
```python
@@ -34,18 +40,42 @@ print(Point(2, 3).x)
3440
:::
3541

3642
:::cell
37-
Use `Counter` when counting is the data shape.
43+
Use `Counter` when the question is "how many times did each value appear?"
3844

3945
```python
40-
from collections import Counter, defaultdict, deque, namedtuple
46+
from collections import Counter
4147

4248
counts = Counter("banana")
4349
print(counts.most_common(2))
50+
```
51+
52+
```output
53+
[('a', 3), ('n', 2)]
54+
```
55+
:::
56+
57+
:::cell
58+
Use `defaultdict(list)` when each key gathers multiple values and the missing-key case should create an empty list automatically.
59+
60+
```python
61+
from collections import defaultdict
4462

4563
groups = defaultdict(list)
4664
for name, team in [("Ada", "red"), ("Grace", "blue"), ("Lin", "red")]:
4765
groups[team].append(name)
4866
print(dict(groups))
67+
```
68+
69+
```output
70+
{'red': ['Ada', 'Lin'], 'blue': ['Grace']}
71+
```
72+
:::
73+
74+
:::cell
75+
Use `deque` for queue operations at both ends, and `namedtuple` when a tiny immutable record needs names as well as positions.
76+
77+
```python
78+
from collections import deque, namedtuple
4979

5080
queue = deque(["first"])
5181
queue.append("second")
@@ -56,15 +86,13 @@ print(Point(2, 3).x)
5686
```
5787

5888
```output
59-
[('a', 3), ('n', 2)]
60-
{'red': ['Ada', 'Lin'], 'blue': ['Grace']}
6189
first
6290
2
6391
```
6492
:::
6593

6694
:::note
67-
- Use `Counter` when counting is the data shape.
68-
- Use `defaultdict` when grouping values by key.
69-
- Use `deque` for efficient queue operations and `namedtuple` for lightweight named records.
95+
- `Counter` counts, `defaultdict` groups, `deque` queues, and `namedtuple` names record fields.
96+
- Prefer the built-in containers until a specialized shape makes the code clearer.
97+
- For new structured records with defaults and methods, consider `dataclasses` instead of `namedtuple`.
7098
:::

0 commit comments

Comments
 (0)