Hi Junio, On Mon, 23 May 2022, Junio C Hamano wrote: > [...] the test output got a lot shorter by discarding the "ok" output > and keeping only the failures and skips. Because the readers are mostly > interested in seeing failures (they can download the full log if > they want to), and this design decision probably makes sense to me. For the record, Victoria suggested to group by file rather than by failed test case. However, I do speak from a lot of experience diagnosing test failures in CI/PR runs when I say: it is frequently very helpful to have a look at one failed test case at a time. I'd much rather suffer a minor lag while scrolling than having to find the boundaries manually, in particular when `test_expect_failure` test cases are present (which are reported as "broken" in the current iteration instead of "failed"). Besides, the scroll issue is probably similar between both approaches to grouping (and may be independent of the grouping, as you pointed out by reporting similar issues in the current `print-test-failures` step), and is something I hope the Actions engineers are working on. > Common to the both approaches, folding output from each test piece > to one line (typically "ok" but sometimes "failed" heading) may be > the source of UI responsiveness irritation I have been observing, > but I wonder, with the removal of all "ok" pieces, it may make sense > not to fold anything and instead give a flat "here are the traces of > all failed and skipped tests". As I mentioned above, I'd rather keep the grouping by failed test case. Obviously, the ideal way to decide would be to set up some A/B testing with real people, but I have no way to set up anything like that. > In any case, either implementation seems to give us a good improvement > over what is in 'master'. There are two things I would like to add: - In the current iteration's summary page, you will see the failed test cases' titles in the errors, and they are clickable (and will get you to the corresponding part of the logs). I find this very convenient. - The addition of the suggestion to look at the run's artifacts for the full logs might not look like a big deal, but I bet that it will help in particular new contributors. This was yet another great suggestion by Victoria. Thanks, Dscho
