Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions docs/ooxml_corner_cases.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ This document tracks edge cases and quirks in Open XML document processing where
- [Misleading Deflate Hints Cause Compression Loss](#misleading-deflate-hints-cause-compression-loss)
4. [Paragraph Layout](#paragraph-layout)
- [`w:lineRule="auto"` is a multiple of the FONT's line box, not of font-size](#wlineruleauto-is-a-multiple-of-the-fonts-line-box-not-of-font-size)
- [An accumulated line-spacing error can resemble a top-margin deviation](#an-accumulated-line-spacing-error-can-resemble-a-top-margin-deviation)
- [LibreOffice ignores the `Hyperlink` character style on TOC field results](#libreoffice-ignores-the-hyperlink-character-style-on-toc-field-results)
5. [Theme Colors](#theme-colors)
- [`w:color`/`w:fill` are a CACHE; `w:themeColor`/`w:themeFill` are the authority](#wcolorwfill-are-a-cache-wthemecolorwthemefill-are-the-authority)
Expand Down Expand Up @@ -1288,6 +1289,59 @@ which is what pagination parity needed at the time; populated paragraphs kept th
fallback. Issues #396 (DrawingML textbox auto-fit height) and #397 (TOC line height) were both
traced to that remaining fallback.

### An accumulated line-spacing error can resemble a top-margin deviation

#### Symptom

In `DB012-Lists-With-Different-Numberings.docx`, later list lines once appeared about 28px lower
in Docxodus than in LibreOffice. Because the section declares a 1701-twip top margin and the first
content is a list, the shift was initially attributed to different top-margin import behavior.

#### Relevant XML

```xml
<w:sectPr>
<w:pgSz w:w="11906" w:h="16838"/>
<w:pgMar w:top="1701" w:right="1134" w:bottom="1701" w:left="1134"
w:header="708" w:footer="708" w:gutter="0"/>
</w:sectPr>
```

At 96 DPI, 1701 twips is 113.4px. Glyph ink begins a few pixels below that content edge because
the ink bounds measure the font's painted pixels, not the top of its line box.

#### Renderer comparison

The fixture was rendered through Microsoft Graph's Word DOCX-to-PDF conversion and rasterized
under the benchmark's 96-DPI contract. The current Docxodus and LibreOffice artifacts use the
same page size and font-substitution contract.

| Renderer | Page (px) | First ink row | Last ink row |
|---|---:|---:|---:|
| Microsoft Graph Word conversion | 794 × 1123 | **117** | 365 |
| LibreOffice 25.8.7.3 | 794 × 1123 | **117** | 365 |
| Docxodus | 794 × 1123 | **118** | 365 |

The one-row first-glyph difference is rasterization, not layout. Docxodus and LibreOffice have
exact tolerant ink geometry (F1 1.00000), while Word independently confirms the same page-top
position. There is no top-margin deviation to emulate or renderer fix to make.

#### Analysis

The apparent 28px displacement grew with each line. That is the signature of the automatic line
spacing defect described above, not a constant page-origin offset. Once `w:lineRule="auto"` was
measured against the font's native line box, the accumulated displacement disappeared without any
change to `w:pgMar` handling. The `numbered-lists` corpus case is therefore an `environment`
residual (substituted-font rasterization), not a `reference-deviation`.

#### Evidence and tests

- `npm/tests/visual-parity/word-reference.json` records the Word page geometry, ink bounds,
fixture hash, capture environment, and first-line measurement.
- `npm/tests/visual-parity-word-reference.spec.ts` validates that the corpus disposition cannot
cite Word evidence unless the corresponding measurement is committed.
- `npm/tests/visual-parity/ratchet.json` records the current Docxodus/LibreOffice F1 of 1.00000.

### LibreOffice ignores the `Hyperlink` character style on TOC field results

#### Symptom
Expand Down
37 changes: 24 additions & 13 deletions npm/tests/visual-parity-word-reference.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,17 @@ import {

const __dirname = dirname(fileURLToPath(import.meta.url));
const repoRoot = resolve(__dirname, '../..');
// Synthetic records below intentionally measure only one or two cases. Strip the committed
// corpus's real citations here; the dedicated citation test adds its own, while the committed-
// record test validates the real corpus unchanged.
const UNIT_CORPUS: VisualCorpusEntry[] = VISUAL_PARITY_CORPUS.map(entry => ({
...entry,
disposition: {
kind: entry.disposition.kind,
rationale: entry.disposition.rationale,
...(entry.disposition.reference ? { reference: entry.disposition.reference } : {}),
},
}));

function image(width: number, height: number, draw?: (data: Uint8Array) => void): RgbaImage {
const data = new Uint8Array(width * height * 4).fill(255);
Expand Down Expand Up @@ -55,7 +66,7 @@ function measuredCase(overrides: Partial<WordReferenceCase> = {}): WordReference

function recordWith(cases: WordReferenceCase[]): WordReferenceRecord {
let record: WordReferenceRecord = {
...emptyRecord(VISUAL_PARITY_CORPUS),
...emptyRecord(UNIT_CORPUS),
environment: { word: 'Word 365', os: 'Windows 11' },
};
for (const entry of cases) record = upsertCase(record, entry);
Expand Down Expand Up @@ -90,55 +101,55 @@ test.describe('word-reference measurement', () => {

test.describe('word-reference validation', () => {
test('the seeded record validates against the corpus', () => {
expect(validateWordReference(emptyRecord(VISUAL_PARITY_CORPUS), VISUAL_PARITY_CORPUS))
expect(validateWordReference(emptyRecord(UNIT_CORPUS), UNIT_CORPUS))
.toEqual([]);
});

test('a measured case validates and coexists with pending rows', () => {
expect(validateWordReference(recordWith([measuredCase()]), VISUAL_PARITY_CORPUS)).toEqual([]);
expect(validateWordReference(recordWith([measuredCase()]), UNIT_CORPUS)).toEqual([]);
});

test('a missing corpus row is a problem — coverage cannot silently shrink', () => {
const record = emptyRecord(VISUAL_PARITY_CORPUS);
const record = emptyRecord(UNIT_CORPUS);
record.cases = record.cases.filter(entry => entry.id !== 'nested-table');
expect(validateWordReference(record, VISUAL_PARITY_CORPUS).join(' '))
expect(validateWordReference(record, UNIT_CORPUS).join(' '))
.toContain('nested-table has no word-reference row');
});

test('a row matching no corpus case is a problem — evidence cannot outlive its case', () => {
const record = recordWith([measuredCase({ id: 'retired-case' } as WordReferenceCase)]);
expect(validateWordReference(record, VISUAL_PARITY_CORPUS).join(' '))
expect(validateWordReference(record, UNIT_CORPUS).join(' '))
.toContain('retired-case matches no corpus case');
});

test('a pending row carrying measurement fields is a problem', () => {
const record = recordWith([
{ id: 'shape', status: 'pending', pageCount: 1 } as WordReferenceCase,
]);
expect(validateWordReference(record, VISUAL_PARITY_CORPUS).join(' '))
expect(validateWordReference(record, UNIT_CORPUS).join(' '))
.toContain('shape is pending but carries measurement fields');
});

test('measured rows need a fixture hash and internally consistent pages', () => {
expect(validateWordReference(
recordWith([measuredCase({ fixtureSha256: undefined })]), VISUAL_PARITY_CORPUS).join(' '))
recordWith([measuredCase({ fixtureSha256: undefined })]), UNIT_CORPUS).join(' '))
.toContain('without a valid fixtureSha256');
expect(validateWordReference(
recordWith([measuredCase({ pageCount: 2 })]), VISUAL_PARITY_CORPUS).join(' '))
recordWith([measuredCase({ pageCount: 2 })]), UNIT_CORPUS).join(' '))
.toContain('pageCount/pages disagree');
expect(validateWordReference(
recordWith([measuredCase({
pages: [{ page: 1, width: 816, height: 1056, inkBounds: { left: 500, top: 0, right: 10, bottom: 5 }, inkPixelRatio: 0.01 }],
})]), VISUAL_PARITY_CORPUS).join(' '))
})]), UNIT_CORPUS).join(' '))
.toContain('ink bounds are inconsistent');
});

test('a wordEvidence citation without a measurement is a problem — the #402 contract', () => {
const corpusWithCitation: VisualCorpusEntry[] = VISUAL_PARITY_CORPUS.map(entry =>
const corpusWithCitation: VisualCorpusEntry[] = UNIT_CORPUS.map(entry =>
entry.id === 'nested-table'
? { ...entry, disposition: { ...entry.disposition, wordEvidence: 'Word suppresses the spacing' } }
: entry);
expect(validateWordReference(emptyRecord(VISUAL_PARITY_CORPUS), corpusWithCitation).join(' '))
expect(validateWordReference(emptyRecord(UNIT_CORPUS), corpusWithCitation).join(' '))
.toContain('nested-table disposition cites wordEvidence but the word-reference row is pending');
// The same citation is fine once the row is measured.
expect(validateWordReference(
Expand All @@ -147,7 +158,7 @@ test.describe('word-reference validation', () => {

test('measured cases without a recorded Word/OS environment are a problem', () => {
const record = { ...recordWith([measuredCase()]), environment: null };
expect(validateWordReference(record, VISUAL_PARITY_CORPUS).join(' '))
expect(validateWordReference(record, UNIT_CORPUS).join(' '))
.toContain('no environment');
});

Expand Down
5 changes: 4 additions & 1 deletion npm/tests/visual-parity/BASELINE.md
Original file line number Diff line number Diff line change
Expand Up @@ -282,7 +282,10 @@ The corpus rerun also dissolved a standing attribution. `numbered-lists` was `re
on the reading that "the whole content is about 28px lower, so LibreOffice must import the 1701-twip
top margin differently". It was the accumulated line-spacing error: the case is now close with
**exact ink geometry (F1 1.00000)**, so the two engines never disagreed about the margin. Issue #398
was opened to obtain Word evidence for that margin; the premise is gone.
obtained the independent Word evidence: the Microsoft Graph conversion's first ink is row 117 at
96 DPI, exactly matching LibreOffice and within one raster row of Docxodus (118). The premise is
gone, and the case is correctly attributed to the substituted-font environment rather than a
reference deviation.

**One regression, found and fixed by the change itself.** With taller line boxes, `footnote`'s ink
F1 fell 0.70330 → 0.58697: CA008's single body line sat 2px lower, and on a page carrying ~1230 ink
Expand Down
3 changes: 3 additions & 0 deletions npm/tests/visual-parity/corpus.ts
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,9 @@ export const VISUAL_PARITY_CORPUS: VisualCorpusEntry[] = [
'growing amount down the page. With auto spacing measured against the font line box the ' +
'case is close and ink geometry is exact (F1 1.00000), so the margin was never in ' +
'disagreement. The residual is substituted-font rasterization.',
reference: 'https://github.com/JSv4/Docxodus/issues/398',
wordEvidence: 'At 96 DPI, the Microsoft Graph conversion starts first-line ink at row 117, ' +
'matching LibreOffice exactly and lying within one raster row of Docxodus (118).',
},
},
{
Expand Down
31 changes: 29 additions & 2 deletions npm/tests/visual-parity/word-reference.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@
"schemaVersion": 1,
"description": "Word-reference evidence store (issue #402). Numbers only: page counts, page geometry, ink extents, and named per-case measurements taken from Microsoft Word PDF exports of the tracked corpus fixtures, rasterized under the same 96-DPI Poppler contract as the benchmark. No binaries and no image corpus are ever committed. Word renders with the genuine Office fonts (not the issue-#379 substitutes), so these measurements decide STRUCTURAL questions; pixel scores against Word are advisory. See WORD_REFERENCE.md for the capture procedure.",
"procedure": "npm/tests/visual-parity/WORD_REFERENCE.md",
"environment": null,
"environment": {
"word": "Microsoft Graph v1.0 DOCX-to-PDF service (build undisclosed)",
"os": "Remote conversion service (server OS undisclosed)"
},
"cases": [
{ "id": "chart", "status": "pending" },
{ "id": "chart-line", "status": "pending" },
Expand All @@ -17,7 +20,31 @@
{ "id": "merged-table", "status": "pending" },
{ "id": "multi-section", "status": "pending" },
{ "id": "nested-table", "status": "pending" },
{ "id": "numbered-lists", "status": "pending" },
{
"id": "numbered-lists",
"status": "measured",
"fixtureSha256": "eb7474e7c8971176c8462f57c2f4fe7d64169f44a408e4f67d92209c4b6fe78e",
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 794,
"height": 1123,
"inkBounds": {
"left": 100,
"top": 117,
"right": 130,
"bottom": 365
},
"inkPixelRatio": 0.00072
}
],
"keyMeasurements": {
"firstLineInkTopPx": 117
},
"notes": "The first line begins at the same ink row as LibreOffice and within one raster row of Docxodus; the former 28 px claim was accumulated automatic-line-spacing error, not top-margin import.",
"capturedAt": "2026-08-13"
},
{ "id": "running-content", "status": "pending" },
{ "id": "shape", "status": "pending" },
{ "id": "text-formatting", "status": "pending" },
Expand Down
Loading