On Thu, May 26, 2022 at 10:27:41AM -0700, Darrick J. Wong wrote: > /me looks and sees a large collection of expunge lists, along with > comments about how often failures occur and/or reasons. Neat! > > Leah mentioned on the ext4 call this morning that she would have found > it helpful to know (before she started working on 5.15 backports) which > tests were of the flaky variety so that she could better prioritize the > time she had to look into fstests failures. (IOWS: saw a test fail a > small percentage of the time and then burned a lot of machine time only > to figure out that 5.15.0 also failed a percentage of th time). See my proposal to try to make this easier to parse: https://lore.kernel.org/all/YoW0ZC+zM27Pi0Us@xxxxxxxxxxxxxxxxxxxxxx/ > We talked about where it would be most useful for maintainers and QA > people to store their historical pass/fail data, before settling on > "somewhere public where everyone can review their colleagues' notes" and > "somewhere minimizing commit friction". At the time, we were thinking > about having people contribute their notes directly to the fstests > source code, but I guess Luis has been doing that in the kdevops repo > for a few years now. > > So, maybe there? For now sure, I'm happy to add others the linux-kdevops org on github and they get immediate write access to the repo. This is working well so far. Long term we need to decide if we want to spin off the expunge list as a separate effort and make it a git subtree (note it is different than a git sub module). Another example of a use case for a git subtree, to use it as an example, is the way I forked kconfig from Linux into a standalone git tree so to allow any project to bump kconfig code with just one command. So different projects don't need to fork kconfig as they do today. The value in doing the git subtree for expunges is any runner can use it. I did design kdevops though to run on *any* cloud, and support local virtualization tech like libvirt and virtualbox. The linux-kdevops git org also has other projects which both fstest and blktests depend on, so for example dbench which I had forked and cleaned up a while ago. It may make sense to share keeping oddball efforts like thse which are no longer maintained in this repo. There is other tech I'm evaluating for this sort of collaborative test efforts such as ledgers, but that is in its infancy at this point in time. I have a sense though it may be a good outlet for collection of test artifacts in a decentralized way and also allow *any* entity to participate in bringing confidence to stable kernel branches or dev branches prior to release. Luis
