backlog: file #1221 -- a doc lint crashes printing a hit that contains a glyph - #330
Merged
Conversation
…tains a glyph
The coordinator asked whether this belongs under #1030 or needs its own
number. It needs its own, and the reason is measured rather than argued.
#1030 is 'keep the corpus cp1252-clean, because per-file gating lets the
class recur'. THAT CANNOT FIX THIS, because the corpus this lint reads is
REQUIRED to contain cp1252-unsafe characters: asvs_tally_lint walks
rglob('*.md') over docs/ (:245) and explicitly reads docs/BACKLOG.md
(:63-64), whose sanctioned banner alphabet is CLAUDE.md section 11's one
machine-parsed holdout and every glyph of which is cp1252-unsafe. Its own
comment at :153 records it already redding on backlog content.
So a clean corpus is not achievable HERE even in principle. #1030 is
about the CONTENT; this is about the INSTRUMENT. Both are needed and
neither substitutes for the other.
The defect shape is the interesting half: it fires ONLY on the failure
path -- printing the hits -- so it is INVISIBLE WHILE THE LINT IS GREEN,
and it converts a legible FAIL into a traceback at exactly the moment
someone needs to read which document tripped it. A gate that is correct
until it has something to say, and then unreadable.
The item states the real decision rather than prescribing a patch: force
UTF-8 on the script's own stdout -- an idiom already proven in-tree, since
messagefoundry/__main__.py hardens stdout/stderr for the whole CLI and the
harness and a bench script carry the same remedy -- or treat the crash as
the corpus's problem. PYTHONIOENCODING=utf-8 is a workaround, not a fix,
because CI and a developer shell will not both set it.
And it names where to prove the fix: MAKE THE LINT FIRE on a document
containing a glyph and read the output. A green run exercises none of it,
which is the whole point.
Fourth cp1252 surface in one day -- a U+21D2 in a ci.yml comment, three
pre-existing U+2192 already on main in that same file, one in the
coordinator playbook, and now this. The first three are #1030's subject;
this one is not.
Gates: 485 items each declaring exactly one status; ledger_check clean.
cp1252 dogfood on my own added lines: the only unsafe character is
U+1F522, the sanctioned banner glyph, which is the exemption recorded
under #1030.
… lint names The coordinator measured the corpus after I filed, and it moves the item from 'fires on the failure path in principle' to 'fires on a large fraction of possible failures in the primary corpus'. main's docs/BACKLOG.md -- the file asvs_tally_lint names at :63-64 -- carries 799 cp1252-unsafe codepoints across 29 distinct characters, including U+2192 x140. So the crash is reachable from any hit whose line carries an arrow, and there are 140 chances from that character alone. I re-derived the census rather than transcribing it, and that surfaced a decoding correction now recorded in the item: the first pass reported U+D83D x193 and U+DD22 x171, which are UTF-16 SURROGATE HALVES rather than characters. Read as codepoints they are U+1F522 x171 and U+1F6A7 x16. Same underlying data, but anyone grepping for U+D83D would find nothing and reasonably conclude the census was wrong. Count codepoints, not UTF-16 units. The item also now carries the in-tree hardening precedent with its edge cases, which is worth more than the idiom alone: __main__.py:52-55 uses getattr(stream, 'reconfigure', None) and calls it with errors='replace', with a comment naming the environments that lack or reject reconfigure -- PYTHONLEGACYWINDOWSSTDIO and pytest capture. Whoever builds this inherits that scar tissue instead of rediscovering it. Gates: 485 items each declaring exactly one status; #1221 still open.
#1220 landed while #1221 was in flight, and both insert at the same
docs/BACKLOG.md line, so the tail conflicts. Resolved keep-both, #1220
then #1221, in numeric order.
KEEP-BOTH HAS BEEN THE WRONG ANSWER TWICE TONIGHT, so I checked the
property that makes it right here rather than reaching for it:
conflict blocks : 1
OURS carries : {1221} only
THEIRS carries : {1220} only
intersection : EMPTY
Two disjoint appends of different items, neither referencing the other's
text. That is a different situation from #302, where keep-both would have
resurrected 38 archived items, and from #1220's own subject, where it
ships seam 19 describing two contract changes. The phrase is not the
answer; the property is.
RE-DERIVED ON THE RESULT rather than trusting the resolution, as the
coordinator asked:
live 250 | archive 236 | namespace 486 | in BOTH files 0
duplicate headings in live: 0
#1220 and #1221 both live, neither archived
backlog_status_check: 486 items, each declaring exactly one status
ledger_check: clean
doc guards: 48 passed (link resolution, banner status, dast claims)
484 -> 486, nothing duplicated, which is what a correct two-append merge
must produce.
My own earlier advice to batch these was semantically right and
mechanically incomplete: I checked whether either item invalidated the
other's claims and never asked whether their HUNKS collided. They insert
at the same line. Two PRs bought the same parallelism at the cost of
exactly this one resolve.
wshallwshall
force-pushed
the
w3-file-tallylint
branch
from
August 11, 2026 17:32
0fdf0ce to
0302ccf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Files BACKLOG #1221 -- a doc lint that crashes while printing its own findings.
scripts/docs/asvs_tally_lint.pyraisesUnicodeEncodeError: 'charmap' codec can't encode characterwhena hit's text contains a glyph. It fires only on the failure path, so it is invisible while the lint
is green, and it converts a legible FAIL into a traceback at exactly the moment someone needs to read
which document tripped it. A gate that is correct until it has something to say, and then unreadable.
Measured, and this is what moves it from theoretical to likely. The lint walks
rglob("*.md")overdocs/, andmain'sdocs/BACKLOG.mdcarries 799 cp1252-unsafe codepoints across 29 distinctcharacters -- including
U+2192x140. So the crash is reachable from any hit whose line carries anarrow, in the primary corpus, with 140 chances.
Why this is not
#1030evidence, and needs its own number.#1030asks that the corpus be keptcp1252-clean. That cannot fix this, because this lint's corpus is required to contain cp1252-unsafe
characters:
docs/BACKLOG.mdcarries the sanctioned banner alphabet -- CLAUDE.md section 11's onemachine-parsed holdout, which
backlog_status_check.pyrequires -- and all five of those glyphs arecp1252-unsafe.
#1030is about the CONTENT; this is about the INSTRUMENT. Both are needed; neithersubstitutes.
It wants a decision, not a patch: should a repo script force UTF-8 on its own stdout, or should the
crash be treated as the corpus's problem? The first is already an in-tree idiom --
messagefoundry/__main__.py:52-55hardens stdout/stderr viagetattr(stream, "reconfigure", None)witherrors="replace", and comments the environments that lackor reject it (
PYTHONLEGACYWINDOWSSTDIO, pytest capture).PYTHONIOENCODINGis a workaround, not a fix:CI and a developer shell will not both set it.
How to prove any fix: make the lint fire on a document containing a glyph and read the output. A
green run exercises none of this, which is the entire point.
Ledger-only. Verified:
#1221free in the live ledger and the archive, merges clean onto currentmain.