On Mon, Jun 20 2016 at 3:40pm -0400, Mike Snitzer <snitzer@xxxxxxxxxx> wrote: > # dd if=/dev/zero of=/mnt/dax/meh bs=1024K oflag=direct > [11729.754671] XFS (dm-4): Metadata corruption detected at xfs_agf_read_verify+0x70/0x120 [xfs], xfs_agf block 0x45a808 > [11729.766423] XFS (dm-4): Unmount and run xfs_repair > [11729.771774] XFS (dm-4): First 64 bytes of corrupted metadata buffer: > [11729.778869] ffff8800b8038000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > [11729.788582] ffff8800b8038010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > [11729.798293] ffff8800b8038020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > [11729.808002] ffff8800b8038030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > [11729.817715] XFS (dm-4): metadata I/O error: block 0x45a808 ("xfs_trans_read_buf_map") error 117 numblks 8 > > When this XFS corruption occurs corruption then also manifests in lvm2's > metadata: > > # vgremove pmem > Do you really want to remove volume group "pmem" containing 1 logical volumes? [y/n]: y > Do you really want to remove active logical volume lv? [y/n]: y > Incorrect metadata area header checksum on /dev/pmem0p1 at offset 4096 > WARNING: Failed to write an MDA of VG pmem. > Incorrect metadata area header checksum on /dev/pmem0p2 at offset 4096 > WARNING: Failed to write an MDA of VG pmem. > Failed to write VG pmem. > Incorrect metadata area header checksum on /dev/pmem0p2 at offset 4096 > Incorrect metadata area header checksum on /dev/pmem0p1 at offset 4096 > > If I don't use XFS, and only issue IO directly to the /dev/pmem/lv, I > don't see this corruption. I did the same test with ext4 instead of xfs and it resulted in the same type of systemic corruption (lvm2 metadata corrupted too): [12816.407147] EXT4-fs (dm-4): DAX enabled. Warning: EXPERIMENTAL, use at your own risk [12816.416123] EXT4-fs (dm-4): mounted filesystem with ordered data mode. Opts: dax [12816.766855] EXT4-fs error (device dm-4): ext4_mb_generate_buddy:758: group 9, block bitmap and bg descriptor inconsistent: 32768 vs 32395 free clusters [12816.782016] EXT4-fs error (device dm-4): ext4_mb_generate_buddy:758: group 10, block bitmap and bg descriptor inconsistent: 32768 vs 16384 free clusters [12816.797491] JBD2: Spotted dirty metadata buffer (dev = dm-4, blocknr = 0). There's a risk of filesystem corruption in case of system crash. # vgremove pmem Do you really want to remove volume group "pmem" containing 1 logical volumes? [y/n]: y Do you really want to remove active logical volume lv? [y/n]: y Incorrect metadata area header checksum on /dev/pmem0p1 at offset 4096 WARNING: Failed to write an MDA of VG pmem. Incorrect metadata area header checksum on /dev/pmem0p2 at offset 4096 WARNING: Failed to write an MDA of VG pmem. Failed to write VG pmem. Incorrect metadata area header checksum on /dev/pmem0p2 at offset 4096 Incorrect metadata area header checksum on /dev/pmem0p1 at offset 4096 -- dm-devel mailing list dm-devel@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/dm-devel