Skip to content

test(qa): hold every template's DOCX export to a line-by-line corpus baseline - #805

Merged
DemchaAV merged 2 commits into
2.5-devfrom
test/docx-fidelity-corpus
Oct 1, 2026
Merged

DemchaAV merged 2 commits into
2.5-devfrom
test/docx-fidelity-corpus

Conversation

@DemchaAV

@DemchaAV DemchaAV commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner

Why

A DOCX export change was checked by hand against a corpus kept outside the repository. A fix for one template moved lines in another, and nothing failed until someone opened the document. Nothing held the export to where it already stood.

What changed

  • Corpus, in qa. Each template family lists its presets on the fixtures their own tests render, in a *DocxCorpus class in its package (the fixtures are package-private). There are 62 documents: 26 CVs, 15 cover letters, 15 invoices with their long variants, 4 proposals, a receipt and a rota.

  • DocxFidelityCorpusTest.

    • The engine draws each document to PDF and exports it to DOCX.
    • LibreOffice converts all of them in one headless process, under a profile of its own (LibreOfficeConverter). PDFs from an earlier run are deleted first: LibreOffice can exit cleanly when a file fails to load, and a stale PDF would be measured in its place.
  • Lines (PdfLines, FidelityMeasurement).

    • Each document is measured against the page. A line is found by its page and its letters, read in drawing order.
    • A line ends where the next glyph changes baseline, steps back left, or starts more than one and a half times its size further on, so two columns on one baseline are two lines.
    • Lines of the same letters on one page are paired each with the editor's nearest. These are a rota's shifts or a CV's date ranges. One lost or moved among them is not hidden behind another.
  • FidelityBaseline, committed in qa/src/test/resources/docx-fidelity/: a row per document and a row per found line (4,064). A document fails when any of these happens:

    • a line the baseline found is no longer found (set at other words, or on another page);
    • a found line drifts more than 0.5pt further from the page;
    • it is set on a page count further from the engine's;
    • the baseline does not hold it, or holds it and the corpus no longer does.

    A line file that is missing, or that holds another number of a document's lines than its row says, is refused. It is not read as a baseline holding no lines.

    The check is line by line, not by counts per document: NavySidebar, which LibreOffice sets about 3pt off throughout, lost its indented lists without its counts moving.

  • Running it.

    • It runs only when asked for: -Dgraphcompose.docxFidelity=libreoffice. Asked for without LibreOffice, it fails rather than passing unrun.
    • -Dgraphcompose.docxFidelity.update=true rewrites the baseline for a change that moves documents nearer the page. It prints what it writes over first, and its diff shows which documents moved.
    • The baseline records the LibreOffice build it was taken with.
    • Documents, PDFs and the measurements are left in qa/target/docx-fidelity.
  • Docs. CONTRIBUTING.md (testing expectations); CHANGELOG.md under Tests.

Verification

  • Full reactor gate: ./mvnw -B -ntp clean verify -pl :graph-compose-core,:graph-compose-render-pdf,:graph-compose-render-docx,:graph-compose-render-pptx,:graph-compose-templates,:graph-compose-testing,:graph-compose-qa,:graph-compose-coverage -am gives BUILD SUCCESS (qa 1809, the corpus test skipped as not asked for).

    • After install, examples run 93 green.
    • The knowledge checks pass.
  • DocxFidelityGateTest (16 tests, run in every build). On PDFs drawn to differ in just one way:

    • a document measured against itself does not drift;
    • lines set 6pt lower drift 6pt;
    • a lost paragraph is not found;
    • a page more is counted;
    • two columns on one baseline are two lines.

    On lines given directly: two lines of the same letters, the editor drawing one of them, pair it with the line that stands where it stood.

    Against a baseline:

    • a line 0.6pt further fails, also one already 2.8pt off;
    • 0.5pt further passes and 0.51pt fails;
    • a line no longer found fails;
    • a page more fails;
    • a document the baseline does not hold fails, and so does one the corpus no longer has;
    • every move nearer the page passes;
    • a written baseline reads back as written, and a line moved 0.6pt further against the read-back one fails;
    • a line file with a line missing, or no line file, is refused;
    • a malformed document row or line row is refused by name.
  • Each piece fails its own test when taken out. Pairing lines in drawing order rather than nearest-first fails the pairing test. Dropping the count check fails the line-file test.

  • The corpus, with LibreOffice 26.8 on Windows:

  • It catches a real regression. Reverting fix(docx): stand NavySidebar where the page sets it in Word — placed picture size, ring set-in, header margin, list sides #802's rule that a list's own margin indents its items fails cv-navy_sidebar: 2 lines are no longer found ("increased website traffic…", "strategy.") and others are set about 12pt higher. The baseline's counts for that document hardly move under the same revert.

Known limits

  • Only a Windows baseline is committed. CI runs no LibreOffice yet. The next PR adds a CI job and takes its Linux baseline from that job's first run, as the editor sets text a little differently per platform and build.
  • Word is not measured here. It needs COM on a Windows machine, and a Word run follows separately.
  • Lines the editor does not find are not held. A document whose lines are mostly set at other words is held to the few it finds: OrangeOps matches 5 of its 108 lines in LibreOffice. A document's new lines, set where the page does since the baseline was written, are held only once the baseline is rewritten.
  • A line already well off can cross the page's line. A drift of +12pt moving to −12pt is not further from the page, and passes.

Lane: test (qa). No change to main sources.

@DemchaAV
DemchaAV merged commit e6bf221 into 2.5-dev Oct 1, 2026
10 checks passed
@DemchaAV
DemchaAV deleted the test/docx-fidelity-corpus branch October 1, 2026 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant