On Tue, Oct 1, 2019 at 9:12 PM Jeff Layton <jlayton@xxxxxxxxxx> wrote: > > On Tue, 2019-10-01 at 15:04 -0400, Sasha Levin wrote: > > On Tue, Oct 01, 2019 at 01:54:45PM -0400, Jeff Layton wrote: > > > On Tue, 2019-10-01 at 19:03 +0200, Ilya Dryomov wrote: > > > > On Tue, Oct 1, 2019 at 6:41 PM Kenneth Van Alstyne > > > > <kvanalstyne@xxxxxxxxxxxxxxx> wrote: > > > > > All: > > > > > I’m not sure this should go to LKML or here, but I’ll start here. After upgrading from Linux kernel 4.19.60 to 4.19.75 (or 76), I started running into kernel panics in the “ceph” module. Based on the call trace, I believe I was able to narrow it down to the following commit in the Linux kernel 4.19 source tree: > > > > > > > > > > commit 812810399999a673d30f9d04d38659030a28051a > > > > > Author: Yan, Zheng <zyan@xxxxxxxxxx> > > > > > Date: Sun Jun 2 09:45:38 2019 +0800 > > > > > > > > > > ceph: use ceph_evict_inode to cleanup inode's resource > > > > > > > > > > [ Upstream commit 87bc5b895d94a0f40fe170d4cf5771c8e8f85d15 ] > > > > > > > > > > remove_session_caps() relies on __wait_on_freeing_inode(), to wait for > > > > > freeing inode to remove its caps. But VFS wakes freeing inode waiters > > > > > before calling destroy_inode(). > > > > > > > > > > Cc: stable@xxxxxxxxxxxxxxx > > > > > Link: https://tracker.ceph.com/issues/40102 > > > > > Signed-off-by: "Yan, Zheng" <zyan@xxxxxxxxxx> > > > > > Reviewed-by: Jeff Layton <jlayton@xxxxxxxxxx> > > > > > Signed-off-by: Ilya Dryomov <idryomov@xxxxxxxxx> > > > > > Signed-off-by: Sasha Levin <sashal@xxxxxxxxxx> > > > > > > > > > > > > > > > Backing this patch out and recompiling my kernel has since resolved my issues (as far as I can tell thus far). The issue was fairly easy to create by simply creating and deleting files. I tested using ‘dd’ and was pretty consistently able to reproduce the issue. Since the issue occurred in a VM, I do have a screenshot of the crashed machine and to avoid attaching an image, I’ll link to where they are: http://kvanals.kvanals.org/.ceph_kernel_panic_images/ > > > > > > > > > > Am I way off base or has anyone else run into this issue? > > > > > > > > Hi Kenneth, > > > > > > > > This might be a botched backport. The first version of this patch had > > > > a conflict with Al's change that introduced ceph_free_inode() and Zheng > > > > had to adjust it for that. However, it looks like it has been taken to > > > > 4.19 verbatim, even though 4.19 does not have ceph_free_inode(). > > > > > > > > Zheng, Jeff, please take a look ASAP. > > > > > > > > > > (Sorry for the resend -- I got Sasha's old addr) > > > > > > Thanks Ilya, > > > > > > I think you're right -- this patch should not have been merged on any > > > pre-5.2 kernels. We should go ahead and revert this for now, and do a > > > one-off backport for v4.19. > > > > > > Sasha, what do we need to do to make that happen? > > > > I think the easiest would be to just revert the broken one and apply a > > clean backport which you'll send me? > > > > Thanks, Sasha. You can revert the old patch as soon as you're ready. > It'll take me a bit to put together and test a proper backport, but > I'll try to have something ready within the next day or so. Kenneth, this is now fixed in 4.19.77. Thanks for the report! Ilya _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |