On Fri, Feb 25, 2022 at 03:02:49AM -0800, Andrea Bolognani wrote: > On Fri, Feb 25, 2022 at 11:31:39AM +0100, Erik Skultety wrote: > > On Thu, Feb 24, 2022 at 06:49:29AM -0800, Andrea Bolognani wrote: > > > I'm not entirely convinced that requiring the use of lcitool for this > > > task is necessarily the best idea though. Ideally, it should be > > > possible for developers to reproduce and debug CI failures locally > > > without having to clone a second repository. It's fine to require > > > lcitool for tasks that are performed by maintainers, but casual > > > contributors should find all they need in libvirt.git itself. > > > > Unless they also want to test with VMs as well in which case lcitool is highly > > recommended. The thing is that right now ci/helper wraps the Makefile. Let's > > say you (or someone else) replaces it with a python equivalent, I don't think > > that the number of contributors suddenly starting to use it will differ much > > compared to the situation right now, but maybe I'm wrong :). > > My motivation to add the support to lcitool is simple - right now there's an > > RFC on adopting some infrastructure in order to run TCK-based functional tests > > as part of GitLab. That's just the first stage, ultimately we want developers > > to be able to run the same test suite locally without the need to wait for > > gitlab and since the current design of the above is using nested virt, it is an > > achievable goal. There was also a suggestion to adopt the Avocado framework for > > a new test framework which would be largely based on TCK [1]. > > Now, let's get to the point I'm trying to make :) - my expectation/vision is > > that both the remote and local functional test executions will revolve around > > lcitool and so eventually with the adoption of *some* framework it is going to > > become a requirement that libvirt developers do perform functional testing > > before submitting a patch series. With that said, it makes more sense in my > > mind to focus on adding the container runtime support to lcitool in order to > > move towards the bigger goal and have everything consolidated in a single > > place. > > Alright, that's a pretty solid argument. And even if we ultimately > decide that we don't want to require using lcitool, once we have a > project-agnostic Python implementation of this logic it would be > reasonably straightforward to turn it into a standalone tool similar > to the current ci/helper, so we have an exit strategy. > > > > Another thing that has been bothering me is that neither 'ci/helper > > > build' nor 'lcitool build' will actually perform the exact same build > > > steps as the corresponding CI job, making them only mildly effective > > > as debugging tools for CI failures. And of course each of these build > > > recipes is maintained separately, so we have three similar but not > > > quite identical scripts floating around. > > > > Yes, but on its own, I think that is a problem which can be solved separately. > > Agreed that the two tasks are not necessarily depending on each > other, but there's significant overlap. I think it makes sense to > rationalize how the build steps for a project are defined and > maintained as part of adding "build in a local container" support to > lcitool. > > Do you have any high-level concerns about the ci/build approach I > vaguely described? The finer details are of course far from being set > in stone, but I think the overall idea is solid and we should aim for > it being implemented as we evolve our CI tooling. No, I think that was a solid proposal, I would probably think of along those lines as well had I ever come to that idea myself :). Having each repo define their own build script which can be consumed both during local test executions and copied to the Dockerfile for a gitlab job to consume makes complete sense. Top it off with something like 'lcitool container --script <path_to_build_script>' and I think we have a solid ground for future work. Erik
