On Mon, Feb 11, 2019 at 2:24 PM Chris Murphy <lists@xxxxxxxxxxxxxxxxx> wrote:
On Mon, Feb 11, 2019 at 9:58 AM Stephen Gallagher <sgallagh@xxxxxxxxxx> wrote:
Sorry that it's taken me so long to get back to this.
I think the feedback on this has been mostly positive on the Beta
criteria, but I'd like to tweak the phrasing a bit and see if this
comes off more favorable:
I'd like to propose that we add the following criteria to Beta for Fedora 30+:
* Printing must work on at least one printer available to Fedora QA.
"Work" is defined as the output from the device matching a preview
shown on the GNOME print preview display. (Note that differences in
color reproduction are not considered "non-working".)
Does the criterion pply strictly to the printing of text and line
art, or does it also apply to gross departures in photographs? If the
latter:
^minor differences in color reproduction are not considered "non-working"; or
^only major differences in color reproduction are considered "non-working"
Major defined as any of:
obvious and grossly incorrect scaling (e.g. +/- 20%)
color inversion, torqued primaries (white becomes black, black becomes
white; red becomes blue, blue becomes green, etc)
tone reproduction that obliterates relevant identifying detail in two
or more test images
With that language I'm trying to carve out only remarkable, WTF level,
bugs as blockers.
I think we can *probably* leave this as a thing to be decided at a
blocker bug review. I really want to avoid trying to set a hard line
on a topic that is inherently subjective. In general, I think we can
just rely on the "last blocker at Go/No-Go" test for this.
Next question is what applications to use for printing, since the
initiating application matters. What if there's a bug in just one
application? That shouldn't be a printing blocker (it might be a basic
functionality blocker for that application if it's included in default
installations). So I'd say pick two. Firefox and LibreOffice? Firefox
and evince?
How about "Desktop environment's 'test page' functionality" and
whichever basic text editor comes with it.
Next question, test document(s). European Color Initiative has several
test PDFs already prepared, perhaps the most applicable for our
purposes is the visual test (and a subset of it).And for font scaling
and reproduction, Ghent Working Group has test GWG 9.1 which tests
various encodings of TrueType, PostScript, and OpenType rendering.
Also, there's a suite of LibreOffice test files, and while I haven't
gone through it, I'm willing to bet there's one or two that'd serve as
a decent sanity tester (in any case I'm not proposing printing out
entire test suites):
https://github.com/freedesktop/libreoffice-test-files
The nice thing about standardized tests is the far lower risk of bugs
in the test file itself, and for sure the applicable developers are
familiar with them so as they get escalated, it eliminates the kick
back "how did you create this test file? can you attach it to the
bug?" etc.
This sounds useful for automating the tests, but I think in general we
don't need to write this into the criteria. They don't need to be that
specific.
and this to Final for Fedora 30+:
* Printing must work on at least one printer using each of the
following drivers:
- The built-in print-to-PDF driver
- The generic IPP driver
To clarify, this does not mean that all printers need to function
properly that use the IPP driver, just that at least one does (so we
know that printing as a whole is unbroken). Contrary to the first
proposal, we won't specify any particular hardware makes or models
that must work.
I agree with this. One possible sanity test:
1. "Print" the standardized test file to a PDF file (using the
built-in print to PDF driver)
2. Print both the resulting PDF from 1, and the original standardized
test file, to the designated IPP printer.
i.e. two physical prints on paper. And within some ballpark on
scaling, they should appear the same. Some of the subcriteria:
a. PDF file is created from test document
b. PDF file is viewable with the default PDF viewer
c. PDF file is printed
d. Test document is printed
e. minor differences aside: b, c, and d should not cause a WTF
reaction by a human
That seems reasonable, though I'd rather have Master Wordsmith Adam
Williamson phrase that better.
_______________________________________________
test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx
