Re: [REGRESSION] LVM-on-LVM: error while submitting device barriers

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 29/02/2024 21.22, Patrick Plenefisch wrote:
On Thu, Feb 29, 2024 at 2:56 PM Goffredo Baroncelli <kreijack@xxxxxxxxx> wrote:

Your understanding is correct. The only thing that comes to my mind to
cause the problem is asymmetry of the SATA devices. I have one 8TB
device, plus a 1.5TB, 3TB, and 3TB drives. Doing math on the actual
extents, lowerVG/single spans (3TB+3TB), and
lowerVG/lvmPool/lvm/brokenDisk spans (3TB+1.5TB). Both obviously have
the other leg of raid1 on the 8TB drive, but my thought was that the
jump across the 1.5+3TB drive gap was at least "interesting"


what about lowerVG/works ?


That one is only on two disks, it doesn't span any gaps

Sorry, but re-reading the original email I found something that I missed before:

BTRFS error (device dm-75): bdev /dev/mapper/lvm-brokenDisk errs: wr
0, rd 0, flush 1, corrupt 0, gen 0
BTRFS warning (device dm-75): chunk 13631488 missing 1 devices, max
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tolerance is 0 for writable mount
BTRFS: error (device dm-75) in write_all_supers:4379: errno=-5 IO
failure (errors while submitting device barriers.)

Looking at the code, it seems that if a FLUSH commands fails, btrfs
considers that the disk is missing. The it cannot mount RW the device.

I would investigate with the LVM developers, if it properly passes
the flush/barrier command through all the layers, when we have an
lvm over lvm (raid1). The fact that the lvm is a raid1, is important because
a flush command to be honored has to be honored by all the
devices involved.



However yes, I agree that the pair of disks involved may be the answer
of the problem.

Could you show us the output of

$ sudo pvdisplay -m



I trimmed it, but kept the relevant bits (Free PE is thus not correct):


   --- Physical volume ---
   PV Name               /dev/lowerVG/lvmPool
   VG Name               lvm
   PV Size               <3.00 TiB / not usable 3.00 MiB
   Allocatable           yes
   PE Size               4.00 MiB
   Total PE              786431
   Free PE               82943
   Allocated PE          703488
   PV UUID               7p3LSU-EAHd-xUg0-r9vT-Gzkf-tYFV-mvlU1M

   --- Physical Segments ---
   Physical extent 0 to 159999:
     Logical volume      /dev/lvm/brokenDisk
     Logical extents     0 to 159999
   Physical extent 160000 to 339199:
     Logical volume      /dev/lvm/a
     Logical extents     0 to 179199
   Physical extent 339200 to 349439:
     Logical volume      /dev/lvm/brokenDisk
     Logical extents     160000 to 170239
   Physical extent 349440 to 351999:
     FREE
   Physical extent 352000 to 460026:
     Logical volume      /dev/lvm/brokenDisk
     Logical extents     416261 to 524287
   Physical extent 460027 to 540409:
     FREE
   Physical extent 540410 to 786430:
     Logical volume      /dev/lvm/brokenDisk
     Logical extents     170240 to 416260


   --- Physical volume ---
   PV Name               /dev/sda3
   VG Name               lowerVG
   PV Size               <2.70 TiB / not usable 3.00 MiB
   Allocatable           yes
   PE Size               4.00 MiB
   Total PE              707154
   Free PE               909
   Allocated PE          706245
   PV UUID               W8gJ0P-JuMs-1y3g-b5cO-4RuA-MoFs-3zgKBn

   --- Physical Segments ---
   Physical extent 0 to 52223:
     Logical volume      /dev/lowerVG/single_corig_rimage_0_iorig
     Logical extents     629330 to 681553
   Physical extent 52224 to 628940:
     Logical volume      /dev/lowerVG/single_corig_rimage_0_iorig
     Logical extents     0 to 576716
   Physical extent 628941 to 628941:
     Logical volume      /dev/lowerVG/single_corig_rmeta_0
     Logical extents     0 to 0
   Physical extent 628942 to 628962:
     Logical volume      /dev/lowerVG/single_corig_rimage_0_iorig
     Logical extents     681554 to 681574
   Physical extent 628963 to 634431:
     Logical volume      /dev/lowerVG/single_corig_rimage_0_imeta
     Logical extents     0 to 5468
   Physical extent 634432 to 654540:
     FREE
   Physical extent 654541 to 707153:
     Logical volume      /dev/lowerVG/single_corig_rimage_0_iorig
     Logical extents     576717 to 629329

   --- Physical volume ---
   PV Name               /dev/sdf2
   VG Name               lowerVG
   PV Size               <7.28 TiB / not usable 4.00 MiB
   Allocatable           yes
   PE Size               4.00 MiB
   Total PE              1907645
   Free PE               414967
   Allocated PE          1492678
   PV UUID               my0zQM-832Z-HYPD-sNfW-68ms-nddg-lMyWJM

   --- Physical Segments ---
   Physical extent 0 to 0:
     Logical volume      /dev/lowerVG/single_corig_rmeta_1
     Logical extents     0 to 0
   Physical extent 1 to 681575:
     Logical volume      /dev/lowerVG/single_corig_rimage_1_iorig
     Logical extents     0 to 681574
   Physical extent 681576 to 687044:
     Logical volume      /dev/lowerVG/single_corig_rimage_1_imeta
     Logical extents     0 to 5468
   Physical extent 687045 to 687045:
     Logical volume      /dev/lowerVG/lvmPool_rmeta_0
     Logical extents     0 to 0
   Physical extent 687046 to 1049242:
     Logical volume      /dev/lowerVG/lvmPool_rimage_0
     Logical extents     0 to 362196
   Physical extent 1049243 to 1056551:
     FREE
   Physical extent 1056552 to 1473477:
     Logical volume      /dev/lowerVG/lvmPool_rimage_0
     Logical extents     369506 to 786431
   Physical extent 1473478 to 1480786:
     Logical volume      /dev/lowerVG/lvmPool_rimage_0
     Logical extents     362197 to 369505
   Physical extent 1480787 to 1907644:
     FREE

   --- Physical volume ---
   PV Name               /dev/sdb3
   VG Name               lowerVG
   PV Size               1.33 TiB / not usable 3.00 MiB
   Allocatable           yes (but full)
   PE Size               4.00 MiB
   Total PE              349398
   Free PE               0
   Allocated PE          349398
   PV UUID               Ncmgdw-ZOXS-qTYL-1jAz-w7zt-38V2-f53EpI

   --- Physical Segments ---
   Physical extent 0 to 0:
     Logical volume      /dev/lowerVG/lvmPool_rmeta_1
     Logical extents     0 to 0
   Physical extent 1 to 349397:
     Logical volume      /dev/lowerVG/lvmPool_rimage_1
     Logical extents     0 to 349396


   --- Physical volume ---
   PV Name               /dev/sde2
   VG Name               lowerVG
   PV Size               2.71 TiB / not usable 3.00 MiB
   Allocatable           yes
   PE Size               4.00 MiB
   Total PE              711346
   Free PE               255111
   Allocated PE          456235
   PV UUID               xUG8TG-wvp0-roBo-GPo7-sbvn-aE7I-NAHU07

   --- Physical Segments ---
   Physical extent 0 to 416925:
     Logical volume      /dev/lowerVG/lvmPool_rimage_1
     Logical extents     369506 to 786431
   Physical extent 416926 to 437034:
     Logical volume      /dev/lowerVG/lvmPool_rimage_1
     Logical extents     349397 to 369505
   Physical extent 437035 to 711345:
     FREE


Finally, I am not sure if it's relevant, but I did struggle to expand
the raid1 volumes across gaps when creating this setup. I did file a
bug about that, though I am not sure if it's relevant, as I removed
integrity and cache for brokenDisk & lvmPool:
https://gitlab.com/lvmteam/lvm2/-/issues/6

Patrick


--
gpg @keyserver.linux.it: Goffredo Baroncelli <kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5





[Index of Archives]     [Linux Kernel]     [Kernel Development Newbies]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite Hiking]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux