[Bug 217769] New: XFS crash on mount on kernels >= 6.1

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



https://bugzilla.kernel.org/show_bug.cgi?id=217769

            Bug ID: 217769
           Summary: XFS crash on mount on kernels >= 6.1
           Product: File System
           Version: 2.5
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P3
         Component: XFS
          Assignee: filesystem_xfs@xxxxxxxxxxxxxxxxxxxxxx
          Reporter: xani666@xxxxxxxxx
        Regression: No

Created attachment 304789
  --> https://bugzilla.kernel.org/attachment.cgi?id=304789&action=edit
full boot sequence where XFS fails

Background:

I've seen that happen on few machines now, both consumer (my personal laptop)
and enterprise (VMs on server with ECC memory) so I think any kind of disk
corruption could be ruled out here.

Common things between the machines is that all of them were on Debian,
dist-upgraded multiple times. IIRC the filesystem was originally formatted on
4.9 kernel. So basically XFS was formatted long time ago:

xfs_db> version
versionnum [0xb4b4+0x8a] =
V4,NLINK,DIRV2,ATTR,ALIGN,LOGV2,EXTFLG,MOREBITS,ATTR2,LAZYSBCOUNT,PROJID32BIT

xfs_info /dev/vda2 
meta-data=/dev/vda2              isize=256    agcount=4, agsize=624064 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=0        finobt=0, sparse=0, rmapbt=0
         =                       reflink=0    bigtime=0 inobtcount=0 nrext64=0
data     =                       bsize=4096   blocks=2496256, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=0
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

And now *some* of them started to crash near-immediately on boot. After
upgrading to Debian's 6.1 I get the crash, booting the old version (either
5,17, 5.10 or 4.19) works fine. I also tried 6.5~rc4 from experimental to the
same effect, and 6.3 on my laptop (where it only crashes one of the partitions,
as in attachment

The other interesting part is that the server mentioned in logs have a HA pair,
that was formatted at the same date, used near-same but doesn't (so far) have
same behaviour.

In both cases XFS complains about in-memory corruption but I don't believe
that's the case, aside of one of the systems having ECC memory (and dozen other
VMs running without problem) I also memtested them  for ~hour just to be sure.
I also booted that VM on other machine with same effect.

The log in question: 

[   16.115185] ------------[ cut here ]------------
[   16.115856] WARNING: CPU: 2 PID: 646 at fs/xfs/xfs_inode.c:1831
xfs_iunlink+0x165/0x1e0 [xfs]
[   16.118200] Modules linked in: tcp_diag inet_diag binfmt_misc ext4
ghash_clmulni_intel sha512_ssse3 sha512_generic crc16 mbcache jbd2 aesni_intel
crypto_simd cryptd cirrus drm_shmem_helper drm_kms_helper i6300esb
virtio_balloon watchdog pcspkr button joydev evdev serio_raw loop fuse drm
efi_pstore dm_mod configfs qemu_fw_cfg virtio_rng rng_core ip_tables x_tables
autofs4 hid_generic usbhid hid xfs libcrc32c crc32c_generic ata_generic
virtio_net virtio_blk net_failover failover uhci_hcd ata_piix crct10dif_pclmul
crct10dif_common ehci_hcd libata virtio_pci crc32_pclmul floppy crc32c_intel
virtio_pci_legacy_dev virtio_pci_modern_dev virtio scsi_mod usbcore psmouse
scsi_common virtio_ring usb_common i2c_piix4
[   16.129279] CPU: 2 PID: 646 Comm: 6_dirty_io_sche Not tainted 6.5.0-0-amd64
#1  Debian 6.5~rc4-1~exp1
[   16.129290] Hardware name: hq hqblade212.non.3dart.com, BIOS 0.5.1
01/01/2007
[   16.129293] RIP: 0010:xfs_iunlink+0x165/0x1e0 [xfs]
[   16.134173] Code: 89 4c 24 04 72 2d e8 ea 5f f1 d8 8b 74 24 04 48 8d bd e0
00 00 00 e8 8a 40 94 d9 48 89 c3 48 85 c0 74 07 48 83 78 20 00 75 26 <0f> 0b e8
24 9c f1 d8 eb 13 f3 0f 1e fa 48 c7 c6 be fe 64 c0 4c 89
[   16.134177] RSP: 0018:ffffae6740c67b60 EFLAGS: 00010246
[   16.134189] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
0000000000000006
[   16.134192] RDX: 0000000000000000 RSI: ffff9810817bfd98 RDI:
0000000000088b40
[   16.134194] RBP: ffff9810809cd800 R08: ffff9810817bff28 R09:
ffff9810809cd8e0
[   16.134196] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9810861e2ae0
[   16.134199] R13: 00000000000101c0 R14: ffff981081642900 R15:
ffff981084217c00
[   16.146474] FS:  00007f89734e06c0(0000) GS:ffff981119d00000(0000)
knlGS:0000000000000000
[   16.146478] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   16.146480] CR2: 00007f8972b04000 CR3: 00000001024a6002 CR4:
00000000000206e0
[   16.146493] Call Trace:
[   16.146533]  <TASK>
[   16.146536]  ? xfs_iunlink+0x165/0x1e0 [xfs]
[   16.153113]  ? __warn+0x81/0x130
[   16.153193]  ? xfs_iunlink+0x165/0x1e0 [xfs]
[   16.154796]  ? report_bug+0x191/0x1c0
[   16.155433]  ? handle_bug+0x3c/0x80
[   16.155468]  ? exc_invalid_op+0x17/0x70
[   16.155472]  ? asm_exc_invalid_op+0x1a/0x20
[   16.155498]  ? xfs_iunlink+0x165/0x1e0 [xfs]
[   16.159010]  xfs_rename+0xaf9/0xe50 [xfs]
[   16.159285]  xfs_vn_rename+0xfe/0x170 [xfs]
[   16.161080]  ? __pfx_bpf_lsm_inode_permission+0x10/0x10
[   16.161137]  vfs_rename+0xb7e/0xd40
[   16.163241]  ? do_renameat2+0x57a/0x5f0
[   16.163256]  do_renameat2+0x57a/0x5f0
[   16.163277]  __x64_sys_rename+0x43/0x50
[   16.165707]  do_syscall_64+0x60/0xc0
[   16.165740]  ? do_syscall_64+0x6c/0xc0
[   16.165746]  ? do_syscall_64+0x6c/0xc0
[   16.165749]  ? do_syscall_64+0x6c/0xc0
[   16.165754]  entry_SYSCALL_64_after_hwframe+0x6e/0xd8
[   16.165776] RIP: 0033:0x7f89b7277997
[   16.170373] Code: e8 ce 0f 0a 00 f7 d8 19 c0 5b c3 0f 1f 84 00 00 00 00 00
b8 ff ff ff ff 5b c3 66 0f 1f 84 00 00 00 00 00 b8 52 00 00 00 0f 05 <48> 3d 00
f0 ff ff 77 01 c3 48 8b 15 39 94 17 00 f7 d8 64 89 02 b8
[   16.170377] RSP: 002b:00007f89734dfc88 EFLAGS: 00000246 ORIG_RAX:
0000000000000052
[   16.170382] RAX: ffffffffffffffda RBX: 00007f89734dfca0 RCX:
00007f89b7277997
[   16.170384] RDX: 00007f8972b5fbb9 RSI: 00007f8972b5fb80 RDI:
00007f8972b5fb30
[   16.170386] RBP: 00007f8975544480 R08: 0000000000000009 R09:
0000000000000fa0
[   16.170388] R10: 0000000000000000 R11: 0000000000000246 R12:
00007f89734dfca0
[   16.170399] R13: 00007f89734dfcd0 R14: 00007f8972f5e3e8 R15:
00007f8975544480
[   16.170407]  </TASK>
[   16.170408] ---[ end trace 0000000000000000 ]---
[   16.170419] XFS (vda2): Internal error xfs_trans_cancel at line 1104 of file
fs/xfs/xfs_trans.c.  Caller xfs_rename+0x613/0xe50 [xfs]
[   16.201330] CPU: 2 PID: 646 Comm: 6_dirty_io_sche Tainted: G        W       
  6.5.0-0-amd64 #1  Debian 6.5~rc4-1~exp1
[   16.201335] Hardware name: hq hqblade212.non.3dart.com, BIOS 0.5.1
01/01/2007
[   16.201337] Call Trace:
[   16.201377]  <TASK>
[   16.201382]  dump_stack_lvl+0x47/0x60
[   16.201412]  xfs_trans_cancel+0x131/0x150 [xfs]
[   16.214058]  xfs_rename+0x613/0xe50 [xfs]
[   16.218956]  xfs_vn_rename+0xfe/0x170 [xfs]
[   16.218956]  ? __pfx_bpf_lsm_inode_permission+0x10/0x10
[   16.222083]  vfs_rename+0xb7e/0xd40
[   16.222083]  ? do_renameat2+0x57a/0x5f0
[   16.222083]  do_renameat2+0x57a/0x5f0
[   16.222083]  __x64_sys_rename+0x43/0x50
[   16.222083]  do_syscall_64+0x60/0xc0
[   16.222083]  ? do_syscall_64+0x6c/0xc0
[   16.222083]  ? do_syscall_64+0x6c/0xc0
[   16.222083]  ? do_syscall_64+0x6c/0xc0
[   16.222083]  entry_SYSCALL_64_after_hwframe+0x6e/0xd8
[   16.222083] RIP: 0033:0x7f89b7277997
[   16.222083] Code: e8 ce 0f 0a 00 f7 d8 19 c0 5b c3 0f 1f 84 00 00 00 00 00
b8 ff ff ff ff 5b c3 66 0f 1f 84 00 00 00 00 00 b8 52 00 00 00 0f 05 <48> 3d 00
f0 ff ff 77 01 c3 48 8b 15 39 94 17 00 f7 d8 64 89 02 b8
[   16.222083] RSP: 002b:00007f89734dfc88 EFLAGS: 00000246 ORIG_RAX:
0000000000000052
[   16.222083] RAX: ffffffffffffffda RBX: 00007f89734dfca0 RCX:
00007f89b7277997
[   16.222083] RDX: 00007f8972b5fbb9 RSI: 00007f8972b5fb80 RDI:
00007f8972b5fb30
[   16.222083] RBP: 00007f8975544480 R08: 0000000000000009 R09:
0000000000000fa0
[   16.222083] R10: 0000000000000000 R11: 0000000000000246 R12:
00007f89734dfca0
[   16.222083] R13: 00007f89734dfcd0 R14: 00007f8972f5e3e8 R15:
00007f8975544480
[   16.222083]  </TASK>
[   16.223078] XFS (vda2): Corruption of in-memory data (0x8) detected at
xfs_trans_cancel+0x14a/0x150 [xfs] (fs/xfs/xfs_trans.c:1105).  Shutting down
filesystem.
[   16.223941] systemd-journald[251]:
/var/log/journal/48d6a6734183b423cc1f60686f4553bb/system.journal: IO error,
rotating.
[   16.224548] XFS (vda2): Please unmount the filesystem and rectify the
problem(s)

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.



[Index of Archives]     [XFS Filesystem Development (older mail)]     [Linux Filesystem Development]     [Linux Audio Users]     [Yosemite Trails]     [Linux Kernel]     [Linux RAID]     [Linux SCSI]


  Powered by Linux