On Wed, Mar 25, 2020 at 12:27:33PM +0100, Andrea Bolognani wrote: > On Wed, 2020-03-25 at 10:44 +0000, Daniel P. Berrangé wrote: > > On Wed, Mar 25, 2020 at 11:27:31AM +0100, Andrea Bolognani wrote: > > > On Wed, 2020-03-25 at 09:35 +0100, Erik Skultety wrote: > > > > Related to the availability of shared runners, for merge_requests/post_merge > > > > builds only, the PSI/OpenShift infra may actually help in order to achieve > > > > more stable execution times due to the usage of private runners. > > > > > > As explained by Dan, that will not help for the merge request builds, > > > because those are executed in the context of the submitter's fork and > > > as such don't have access to any runners we might deploy ourselves. > > > > > > It would help speed up the post merge phase, but once we move to a > > > merge request workflow we'll want to test as much as possible at the > > > time the merge request is submitted, and there won't be much that > > > needs to happen after the changes hit master... So dedicated runners > > > might actually not be useful outside of supporting non-Linux targets. > > > > NB, our current testing is all just build+unit testing. We want to expand > > to do integration testing, and that will also benefit from dedicated runners > > so that we can actually run stuff in a real OS, as opposed to a container > > where our functionality is much reduced. > > Good point, I was just thinking in terms of migrating what we already > have to GitLab rather than building upon it. > > This kind of testing will still be limited by the fact that merge > requests will not be able to use these dedicated runners. Compare > this to something like KubeVirt's CI, where proper integrations tests > are executed for each and every merge request... I didn't know that merge requests are run in the context of the submitter, I thought that once submitted, it moves over to the original project. Anyhow, yes, the test coverage (more specifically the integration tests) are going to be limited in this regard, however, even if we could achieve what kubevirt is doing, we certainly don't have the capacity to cover all the existing forks out there. That's where lcitool comes in ;), before you submit a pull request, here, run the test suite via lcitool in an automated fashion and then submit the pull request - not perfect, but IMO a massive improvement. Erik
