On Mon, Apr 18, 2022 at 7:52 AM Zorro Lang <zlang@xxxxxxxxxx> wrote: > > On Sun, Apr 17, 2022 at 08:40:22PM +0300, Amir Goldstein wrote: > > Introduce helpers _known_issue_before_kernel() and > > _fixed_by_kernel_commit() that can be used to hint testers why a test > > might be failing and aid in auto-generating of expunge lists for > > downstream kernel testing. > > > > Annotate fix kernel commits for some overlayfs tests and fix kernel > > version for some overlayfs tests testing for legacy behavior whose > > fixes are not likely to be backported to stable kernels. > > > > This is modeled after LTP's 'make filter-known-fails' and > > print_failure_hints() using struct tst_tag annotations. > > > > Signed-off-by: Amir Goldstein <amir73il@xxxxxxxxx> > > --- > > The _fixed_by_kernel_commit is fine, just not all cases for kernel patch, > some of them cover patch from userspace packages. So you might change it > to _fixed_by_upstream_commit, then let its arguments point out the commit > from which upstream project. > I thought about it too and I agree we should have _fixed_by_upstream_commit In fact, LTP tests are annotated like this: {"linux-git", "8edc6e1688fc"}. But I decided that even if and when we add _fixed_by_upstream_commit we had better leave the wrapper _fixed_by_kernel_commit for brevity because it is by far going to be the most used annotation. I can add this helper now or it could be added later. > The _known_issue_before_kernel() is a little ... hard to say. If it's only > for one base line of a project (e.g. mainline), then it make sense. But if > for lots of downstream projects, the version number will not totally > different. Some downstream projects might backport lots of upstream patches, > without change its base verion. > > If it's just used as a hint output, then it more likes a weak version of > _fixed_by_kernel_commit. When we can't make sure a case cover which specified > commits, then we use a big version number directly. > > If it doesn't aim to be a hint output simply, then we face the problem I > said above, it just can be used for versions from one base line of a project. That's exactly what it is. A hint. Note that I used the release tag as the argument, so the only difference from _fixed_by_kernel_commit is the hint text. could have also used: _fixed_by_kernel_commit v4.19 "overlayfs stacked file operations" Anyway, if one is testing a downstream kernel and sees a hint like this: You _MAY_ be missing a kernel fix... You _MAY_ be hit by a known issue on kernel < ... All it is is a hint. Same as in LTP, which is used to test downstream kernels. And if downstream kernel did backport the fix or the entire feature, then there is a good chance that the test will pass and hint would not be displayed at all. Thanks, Amir.