On Thu, Feb 22, 2018 at 03:50:38PM +0100, Pierre-Yves Chibon wrote: > On Wed, Feb 14, 2018 at 05:28:20PM +0100, Petr Šplíchal wrote: > > Hi! > > > > During the last days there have been concerns raised regarding > > what is an appropriate content for the tests namespace. [1] My > > original idea was to enable sharing tests even across branches of > > the same component, not only for tests to be used by completely > > different packages. The initial examples might have been a bit > > misleading in this respect. One of the main points still holds: > > > > > * Tests might follow a different branching pattern than the > > > dist-git repo, leading to code duplication > > > > From the feedback from developers I feel they always keep on mind > > and care a lot about the maintenance costs. So it perfectly makes > > sense to me if they want to keep and maintain tests in a separate > > repo instead of merging/cherry-picking between dist-git branches. > > > > When, for a particular package, it is the most efficient way to > > maintain tests in a separate repo why should we discourage from > > this approach? There are packages where it makes more sense to > > store test code directly in dist-git. And it is still an option. > > But why should we enforce this for all? > > My worries are basically that this mechanism is hiding the tests from the > package maintainers. It splits the concerns between people maintaining the > artifacts and people maintaining the tests, which is exactly what the original > plan to bring CI was *not* about. > The idea has always been to bring the tests where the code lives, so that both > can be worked on as one, to make tests be a concern of the package maintainer > very much like updating the artifact (rpms, image..) is, while getting support > from QE for them (the tests). > > In addition, this is what I fear most: > The tests for the package are stored elsewhere, we're not sure where (the tests > namespace, a random git repo on the internet...), the packager update package > and the tests fail. > What do you think will happen? > a) the packager will look for $random_place_of_the_internet where tests are and > spend time trying to fix them. > b) the packager will turn off the tests because they get in the way > > If the packager wants to turn off the tests, they will have to go through > dist-git to do it, they may very well end up turning the tests off anyway, but > if the tests are right there, they may as well have a quick look at them to see > if they can fix them quickly before deciding. > > In addition, if the packager turn the tests off and the people maintaining the > tests do not realize that, they will end up spending time maintaining > $somewhere_else tests that aren't being used. > Having them interact directly with the dist-git repo will increase the chances > that they see/realize when something is turned off. > > If that means we have less tests in dist-git because the maintainers do not want > them, I'd say so be it. > In the long term this is their loss, they are the ones who will get the bug > reports and will have to deal with them, they are the ones who will have to dive > into old code rather than going back into something that is still fresh in their > mind. > As long as, it is clear that they have been approached and that it is their > choice to not merge pull-requests adding tests, I think the people offering to > help should not be the ones blamed. Hi, pingou, I share your thinking in general, but I think your concern is overstated. Petr's original e-mail suggests that the new separate namespace should be used *in* *preference* to the in-dist-git, and your reply concentrates on that. But I think that we want to have both possibilities, because both will work best for different cases, and it's just the emphasis that needs to change. I agree that having tests in a separate namespace is not any simpler or more manageable for most packages. Thus, I think that *by* *default* we should put tests in dist-git. IMHO this will apply to 90% of all tests and packages. OTOH, there will be cases where it'll be useful to split the tests out into a separate area. The documentation for tests/ has the example of shared tests for bash/zhs/dash. I'm sure that there will be other cases, like testing of stacks consisting of multiple packages apache+mysql+php or glibc+gcc+binutils, etc, where it'll be beneficial to keep the test separate. The management of permissions in this case will be _much_ more complicated, because to meaningfully update the tests sometimes it'll be necessary to change the test repo and _each_ of the packages. But having them separate is better than randomly sticking the shared tests in dist-git for one of the involved packages. Nevertheless, because of the overhead of more repos and additional permissions, IMHO this should never be advertised as the default approach, but as an "advanced" possibility. Zbyszek _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
