On Fri, May 22, 2020 at 2:40 AM Ondrej Mosnacek <omosnace@xxxxxxxxxx> wrote: > > On Thu, May 21, 2020 at 4:12 PM William Roberts > <bill.c.roberts@xxxxxxxxx> wrote: > > On Thu, May 21, 2020 at 7:58 AM Ondrej Mosnacek <omosnace@xxxxxxxxxx> wrote: > > > > > > On Thu, May 21, 2020 at 2:52 PM Stephen Smalley > > > <stephen.smalley.work@xxxxxxxxx> wrote: > [...] > > > > Last I looked, his script builds and installs the userspace code on > > > > top of the Fedora libraries and programs (make LIBDIR=... install...) > > > > and then runs the testsuite. That was my suggestion. > > > > > > Ah, yes, I can see that line now. Sorry, somehow I missed it before. > > > > > > > While it is the > > > > kernel testsuite, it exercises a lot of SELinux userspace > > > > functionality that isn't tested by the userspace tests. > > > > > > OK, I suppose it's better than nothing... > > > > > > > Stephen pointed out the additional ways userspace gets tested, and > > perhaps my title and description > > of the patch could be better. But the main point is to increase the > > test coverage > > and perform the testing steps we expect are done before a release in > > the CI. We should have > > the testing coverage and the confidence to release userspace from master at any > > point. We also have forward facing proof that tests are being executed > > and we can make sure > > nothing regresses. > > > > My ultimate goal here, is to help make sure that if Petr gets hit by a > > bus, releases will > > move forward without worry and without any change in quality among the various > > maintainers. > > > > Additionally, we pick up some cross project testing and can find other > > surprises. > > Ah, so you want to move an integration test, which we would normally > run only before release, down to per-commit testing, which is fine > because it is quite fast... OK, it started to make sense to me now :) Exactly, plus we pick up a little more test coverage on the userspace bits by swapping out what's installed in the VM with the current build and running the tests. The speed is less important, it's just fast enough where our CI isn't going to take years to complete. CI doesn't need to be super snappy per se, but it also cannot take a fortnight. > > If I find time I'll have a closer look at the scripts. I already see > some tiny possible improvements... I have Nicolas's last comments addressed and staged, so ill wait a few days and see what you come back with and re-send a V3.
