On Thu, Sep 1, 2022 at 4:09 PM Chuck Lever III <chuck.lever@xxxxxxxxxx> wrote: > > > > > On Sep 1, 2022, at 9:51 AM, Olga Kornievskaia <aglo@xxxxxxxxx> wrote: > > > > On Tue, Aug 30, 2022 at 1:49 PM Jeff Layton <jlayton@xxxxxxxxxx> wrote: > >> > >> On Tue, 2022-08-30 at 13:14 -0400, Olga Kornievskaia wrote: > >>> Hi folks, > >>> > >>> Is this a known nfsd kernel oops in 6.0-rc1. Was running xfstests on > >>> pre-rhel-9.1 client against 6.0-rc1 server when it panic-ed. > >>> > >>> [ 5554.769159] BUG: KASAN: null-ptr-deref in kernel_sendpage+0x60/0x220 > >>> [ 5554.770526] Read of size 8 at addr 0000000000000008 by task nfsd/2590 > >>> [ 5554.771899] > >> > >> No, I haven't seen this one. I'm guessing the page pointer passed to > >> kernel_sendpage was probably NULL, so this may be a case where something > >> walked off the end of the rq_pages array? > >> > >> Beyond that I can't tell much from just this stack trace. It might be > >> nice to see what line of code kernel_sendpage+0x60 refers to on your > >> kernel. > > > > After getting debug symbols this is what gdb told me... > > > > (gdb) l *(kernel_sendpage+0x60) > > 0xffffffff81cbd570 is in kernel_sendpage (./include/linux/page-flags.h:487). > > 482 TESTCLEARFLAG(LRU, lru, PF_HEAD) > > 483 PAGEFLAG(Active, active, PF_HEAD) __CLEARPAGEFLAG(Active, active, PF_HEAD) > > 484 TESTCLEARFLAG(Active, active, PF_HEAD) > > 485 PAGEFLAG(Workingset, workingset, PF_HEAD) > > 486 TESTCLEARFLAG(Workingset, workingset, PF_HEAD) > > 487 __PAGEFLAG(Slab, slab, PF_NO_TAIL) > > 488 __PAGEFLAG(SlobFree, slob_free, PF_NO_TAIL) > > 489 PAGEFLAG(Checked, checked, PF_NO_COMPOUND) /* Used by some filesystems */ > > 490 > > 491 /* Xen */ > > > > I'm unable to complete a git bisect. Could somebody suggest what to do > > when a git bisect midpoint results in an unbootable kernel? I can't > > "skip" this point can I? I'm not sure if marking it "bad" makes sense > > since it's not relevant to the actual problem. > > Try skipping. I think "git bisect" is supposed to be smart enough > to figure things out after a skip. > > If not, start over and pick endpoints that you think might avoid > the unbootable kernels. Even adjusting one endpoint a little > might be enough to make bisect pick new targets that avoid the > unbootable kernels. Thanks Chuck. I first, based on a hunch, narrowed down that it's coming from Al Viro's merge commit. Then I git bisected his 32patches to the following commit f0f6b614f83dbae99d283b7b12ab5dd2e04df979 [aglo@ipa84 linux-nfs]$ git bisect log git bisect start # bad: [f30adc0d332fdfe5315cb98bd6a7ff0d5cf2aa38] Merge tag 'pull-work.iov_iter-rebased' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs git bisect bad f30adc0d332fdfe5315cb98bd6a7ff0d5cf2aa38 # good: [5d5d353bed32dc3ea52e2619e0d1c60b17133b91] Merge tag 'rproc-v5.20' of git://git.kernel.org/pub/scm/linux/kernel/git/remoteproc/linux git bisect good 5d5d353bed32dc3ea52e2619e0d1c60b17133b91 # good: [68fe506f3731ecf7881de9512cc5f4c14fd13f3a] unify xarray_get_pages() and xarray_get_pages_alloc() git bisect good 68fe506f3731ecf7881de9512cc5f4c14fd13f3a # good: [dc5801f60b269a73fcce789856c99d1845f75827] af_alg_make_sg(): switch to advancing variant of iov_iter_get_pages() git bisect good dc5801f60b269a73fcce789856c99d1845f75827 # good: [746de1f86fcd33464acac047f111eea877f2f7a0] pipe_get_pages(): switch to append_pipe() git bisect good 746de1f86fcd33464acac047f111eea877f2f7a0 # bad: [f0f6b614f83dbae99d283b7b12ab5dd2e04df979] copy_page_to_iter(): don't split high-order page in case of ITER_PIPE git bisect bad f0f6b614f83dbae99d283b7b12ab5dd2e04df979 > -- > Chuck Lever > > >