For a long time already, we have Git's source code continuously tested via Travis CI, see e.g. https://travis-ci.org/git/git/builds/421738884. It has served us well, and more and more developers actually pay attention and benefit from the testing this gives us. It is also an invaluable tool for contributors who can validate their code contributions via PRs on GitHub, e.g. to verify that their tests do actually run on macOS (i.e. with the BSD family of Unix tools instead of the GNU one). The one sad part about this is the Windows support. Travis lacks it, and we work around that by using Visual Studio Team Services (VSTS) indirectly: one phase in Travis would trigger a build, wait for its log, and then paste that log. As Git's Windows builds (and tests!) take quite a bit of time, Travis often timed out, or somehow the trigger did not work, and for security reasons (the Windows builds are performed in a private pool of containers), the Windows builds are completely disabled for Pull Requests on GitHub. One might ask why we did not use Visual Studio Team Services directly. There were a couple of reasons for that: * most notably, VSTS's build logs could not be viewed anonymously, * while VSTS had Linux and Windows agents, it lacked macOS agents, * etc The main two reasons no longer apply: macOS agents are available now [https://docs.microsoft.com/en-us/vsts/release-notes/2018/jul-10-vsts], and there is a limited preview of "public projects" [https://blogs.msdn.microsoft.com/devops/2018/04/27/vsts-public-projects-limited-preview/] , i.e. it is possible to configure a VSTS project so that anybody can view the logs. I had secured such a public project for Git for Windows already, and I recently also got one for Git. For now, the latter is hooked up with my personal git.git fork on GitHub, but it is my hope that I convince y'all that these VSTS builds are a good idea, and then hook it up with https://github.com/git/git. As a special treat, this patch series adds the ability to present the outcome of Git's test suite as JUnit-style .xml files. This allows the VSTS build to present fun diagrams, trends, and makes it a lot easier to drill down to test failures than before. See for example https://git.visualstudio.com/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details [https://git.visualstudio.com/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details] (you can click on the label of the failed test, and then see the detailed output in the right pane). This patch series took way more time than I had originally planned, but I think that in particular the advanced display of the test results was worth it. Please let me know what you think about this. Johannes Schindelin (9): ci: rename the library of common functions ci/lib.sh: encapsulate Travis-specific things test-date: add a subcommand to measure times in shell scripts tests: optionally write results as JUnit-style .xml ci/lib.sh: add support for VSTS CI Add a build definition for VSTS tests: include detailed trace logs with --write-junit-xml upon failure tests: record more stderr with --write-junit-xml in case of failure README: add a build badge (status of the VSTS build) .vsts-ci.yml | 296 +++++++++++++++++++++++++++++++++ README.md | 2 + ci/install-dependencies.sh | 5 +- ci/{lib-travisci.sh => lib.sh} | 67 ++++++-- ci/mount-fileshare.sh | 26 +++ ci/print-test-failures.sh | 4 +- ci/run-build-and-tests.sh | 2 +- ci/run-linux32-docker.sh | 2 +- ci/run-static-analysis.sh | 2 +- ci/run-windows-build.sh | 2 +- ci/test-documentation.sh | 3 +- t/.gitignore | 1 + t/helper/test-date.c | 12 ++ t/test-lib.sh | 142 ++++++++++++++++ 14 files changed, 544 insertions(+), 22 deletions(-) create mode 100644 .vsts-ci.yml rename ci/{lib-travisci.sh => lib.sh} (61%) create mode 100755 ci/mount-fileshare.sh base-commit: 2f743933341f276111103550fbf383a34dfcfd38 Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-31%2Fdscho%2Fvsts-ci-v1 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-31/dscho/vsts-ci-v1 Pull-Request: https://github.com/gitgitgadget/git/pull/31 -- gitgitgadget
