test(qa): hold every template's DOCX export to a line-by-line corpus baseline - #805
Merged
Merged
Conversation
…ine by line, in LibreOffice
…t its lines, and never measure a stale PDF
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A DOCX export change was checked by hand against a corpus kept outside the repository. A fix for one template moved lines in another, and nothing failed until someone opened the document. Nothing held the export to where it already stood.
What changed
Corpus, in
qa. Each template family lists its presets on the fixtures their own tests render, in a*DocxCorpusclass in its package (the fixtures are package-private). There are 62 documents: 26 CVs, 15 cover letters, 15 invoices with their long variants, 4 proposals, a receipt and a rota.DocxFidelityCorpusTest.LibreOfficeConverter). PDFs from an earlier run are deleted first: LibreOffice can exit cleanly when a file fails to load, and a stale PDF would be measured in its place.Lines (
PdfLines,FidelityMeasurement).FidelityBaseline, committed inqa/src/test/resources/docx-fidelity/: a row per document and a row per found line (4,064). A document fails when any of these happens:A line file that is missing, or that holds another number of a document's lines than its row says, is refused. It is not read as a baseline holding no lines.
The check is line by line, not by counts per document:
NavySidebar, which LibreOffice sets about 3pt off throughout, lost its indented lists without its counts moving.Running it.
-Dgraphcompose.docxFidelity=libreoffice. Asked for without LibreOffice, it fails rather than passing unrun.-Dgraphcompose.docxFidelity.update=truerewrites the baseline for a change that moves documents nearer the page. It prints what it writes over first, and its diff shows which documents moved.qa/target/docx-fidelity.Docs.
CONTRIBUTING.md(testing expectations);CHANGELOG.mdunder Tests.Verification
Full reactor gate:
./mvnw -B -ntp clean verify -pl :graph-compose-core,:graph-compose-render-pdf,:graph-compose-render-docx,:graph-compose-render-pptx,:graph-compose-templates,:graph-compose-testing,:graph-compose-qa,:graph-compose-coverage -amgives BUILD SUCCESS (qa 1809, the corpus test skipped as not asked for).DocxFidelityGateTest(16 tests, run in every build). On PDFs drawn to differ in just one way:On lines given directly: two lines of the same letters, the editor drawing one of them, pair it with the line that stands where it stood.
Against a baseline:
Each piece fails its own test when taken out. Pairing lines in drawing order rather than nearest-first fails the pairing test. Dropping the count check fails the line-file test.
The corpus, with LibreOffice 26.8 on Windows:
It catches a real regression. Reverting fix(docx): stand NavySidebar where the page sets it in Word — placed picture size, ring set-in, header margin, list sides #802's rule that a list's own margin indents its items fails
cv-navy_sidebar: 2 lines are no longer found ("increased website traffic…", "strategy.") and others are set about 12pt higher. The baseline's counts for that document hardly move under the same revert.Known limits
OrangeOpsmatches 5 of its 108 lines in LibreOffice. A document's new lines, set where the page does since the baseline was written, are held only once the baseline is rewritten.Lane: test (qa). No change to
mainsources.