Hi Eric,
On Mon, 3 Sep 2018, Eric Sunshine wrote:
> On Mon, Sep 3, 2018 at 5:10 PM Johannes Schindelin via GitGitGadget
> <gitgitgadget@xxxxxxxxx> wrote:
> > This will come in handy when publishing the results of Git's test suite
> > during an automated VSTS CI run.
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@xxxxxx>
> > ---
> > diff --git a/t/test-lib.sh b/t/test-lib.sh
> > @@ -431,11 +434,24 @@ trap 'exit $?' INT
> > test_failure_ () {
> > + if test -n "$write_junit_xml"
> > + then
> > + junit_insert="<failure message=\"not ok $test_count -"
> > + junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
> > + junit_insert="$junit_insert $(xml_attr_encode \
> > + "$(printf '%s\n' "$@" | sed 1d)")"
> > + junit_insert="$junit_insert</failure>"
>
> This is a genuine failure, so you're creating a <failure> node. Okay.
>
> > + write_junit_xml_testcase "$1" " $junit_insert"
> > + fi
> > @@ -444,11 +460,19 @@ test_failure_ () {
> > test_known_broken_ok_ () {
> > + if test -n "$write_junit_xml"
> > + then
> > + write_junit_xml_testcase "$* (breakage fixed)"
> > + fi
> > test_fixed=$(($test_fixed+1))
> > say_color error "ok $test_count - $@ # TODO known breakage vanished"
> > }
>
> This was expected to fail but didn't, which means it probably needs
> some sort of attention. test_known_broken_ok_() prints this result in
> the 'error' color, and test_done() re-inforces that by printing a
> message, also in 'error' color:
>
> 42 known breakage(s) vanished; please update test(s)
>
> So, should this emit a <failure> node also, perhaps with 'type'
> attribute set to "warning" or something? (<failure type="WARNING"
> message="...">)
My primary aim is to display the test results in the web interface, see
e.g.
https://git.visualstudio.com/git/_build/results?buildId=128&view=ms.vss-test-web.test-result-details
The parser for JUnit XML (and in fact, the JUnit XML schema itself) do not
allow for such a warning. If you add a `<failure>`, then the build fails.
And we do not want the build to fail. Historically, I saw quite a couple
of "vanished" breakages depending on the platform where I ran the tests.
> > @@ -758,9 +793,58 @@ test_at_end_hook_ () {
> > +xml_attr_encode () {
> > + # We do not translate CR to 
 because BSD sed does not handle
> > + # \r in the regex. In practice, the output should not even have any
> > + # carriage returns.
> > + printf '%s\n' "$@" |
> > + sed -e 's/&/\&/g' -e "s/'/\'/g" -e 's/"/\"/g' \
> > + -e 's/</\</g' -e 's/>/\>/g' \
> > + -e 's/ /\	/g' -e 's/$/\
/' -e '$s/
$//' |
> > + tr -d '\012\015'
> > +}
>
> It's possible to insert a literal CR in the 'sed' expression, which
> does match correctly on BSD (and MacOS). For instance:
>
> CR=$(printf "\r")
> sed -e "s/$CR/\
/g"
Okay. But since we are talking about displaying some chunk of text, I
would rather just delete the CR here anyway.
Ciao,
Dscho
