Hi Phil,
On 17/10/2017 10:39 PM, Phil Turmel wrote:
Hi Liwei,
On 10/17/2017 01:09 AM, Liwei wrote:
Hi list, tldr: Have a mixed-sized (2TB and 6TB) RAID6 array, realised
there is now more potential space (using only 6TB drives) than
running the array based on the 2TB "smallest-drive" size. Is there a
way to convert to using only the 6TB drives, without a rebuild?
It depends. /-:
Certainly not in one step. If you are running LVM on top of your array,
and have the space split up into various LVs, none extremely large,
there are some possibilities.
You might need one more 6TB drive to help with the juggling.
Please use lsdrv[1] to document your running setup so we can see what
might be possible. Past the output into your reply in *text* mode, with
line wrapping turned off. (It would help if you composed your mails in
Thunderbird or some client that handles text mode properly -- TBird
supports gmail accounts.)
Phil
[1] https://github.com/pturmel/lsdrv
Whoops, we do run LVM on top, but currently it is one gigantic LV. (We converted to btrfs and started using subvolumes instead) My guess is you're thinking of creating another array with the unused space, and then migrating LVs over?
The lsdrv output follows, with a lot of irrelevant/empty nodes removed. Also, I realised I misread the storage size of two of the 2TB drives, so we're actually at 9x 2TB and 4x 6TB.
_However_, I do have one 6TB "hot spare" plugged in, and 3 proper 6TB SAS drives arriving next week, so there may still be a way to do this, perhaps?
Output of lsdrv is also attached, in case the formatting still gets truncated because I didn't set things up properly.
PCI [mpt3sas] 02:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS3224 PCI-Express Fusion-MPT SAS-3 (rev 01)
├scsi 0:0:0:0 ATA WDC WD2001FASS-0 {WD-WMAY01416344}
│└sda 1.82t [8:0] Partitioned (gpt)
│ └sda1 1.82t [8:1] MD raid6 (7/13) (w/ sdo1,sdm1,sdl1,sdk1,sdj1,sdi1,sdh1,sdg1,sdf1,sde1,sdc1,sdb1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ │ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
│ └VG Carroll 20.01t 0 free {9fcFdE-g501-tLfQ-Lg5k-zCYS-fhw3-vEJI1Y}
│ └dm-12 20.01t [253:12] LV Wonderland btrfs 'Wonderland' {f710ec85-b277-45a4-a526-e970f74e2781}
│ └Mounted as /dev/mapper/Carroll-Wonderland @ /mnt/Wonderland
├scsi 0:0:1:0 ATA WDC WD20EARS-00J {WD-WCAWZ1869671}
│└sdb 1.82t [8:16] Partitioned (gpt)
│ └sdb1 1.82t [8:17] MD raid6 (2/13) (w/ sdo1,sdm1,sdl1,sdk1,sdj1,sdi1,sdh1,sdg1,sdf1,sde1,sdc1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:2:0 ATA WDC WD20EFRX-68E {WD-WCC4M6VTHJRR}
│└sdc 1.82t [8:32] Partitioned (gpt)
│ └sdc1 1.82t [8:33] MD raid6 (5/13) (w/ sdo1,sdm1,sdl1,sdk1,sdj1,sdi1,sdh1,sdg1,sdf1,sde1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:3:0 ATA HGST HDN726060AL {NCGGBM7S}
│└sdd 5.46t [8:48] Empty/Unknown
├scsi 0:0:4:0 ATA WDC WD20EFRX-68E {WD-WCC4M6YJN0SD}
│└sde 1.82t [8:64] Partitioned (gpt)
│ └sde1 1.82t [8:65] MD raid6 (0/13) (w/ sdo1,sdm1,sdl1,sdk1,sdj1,sdi1,sdh1,sdg1,sdf1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:5:0 ATA WDC WD20EFRX-68E {WD-WCC4M6YJNEP7}
│└sdf 1.82t [8:80] Partitioned (gpt)
│ └sdf1 1.82t [8:81] MD raid6 (3/13) (w/ sdo1,sdm1,sdl1,sdk1,sdj1,sdi1,sdh1,sdg1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:6:0 ATA WDC WD6002FFWX-6 {K1GLBHMD}
│└sdg 5.46t [8:96] Partitioned (gpt)
│ └sdg1 5.46t [8:97] MD raid6 (12/13) (w/ sdo1,sdm1,sdl1,sdk1,sdj1,sdi1,sdh1,sdf1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:7:0 ATA WDC WD20EZRX-00D {WD-WMC4N2764539}
│└sdh 1.82t [8:112] Partitioned (gpt)
│ └sdh1 1.82t [8:113] MD raid6 (6/13) (w/ sdo1,sdm1,sdl1,sdk1,sdj1,sdi1,sdg1,sdf1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:8:0 ATA WDC WD20EZRX-00D {WD-WCC300583284}
│└sdi 1.82t [8:128] Partitioned (gpt)
│ └sdi1 1.82t [8:129] MD raid6 (8/13) (w/ sdo1,sdm1,sdl1,sdk1,sdj1,sdh1,sdg1,sdf1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:9:0 ATA WDC WD20EARS-00M {WD-WCAZA5414083}
│└sdj 1.82t [8:144] Partitioned (gpt)
│ └sdj1 1.82t [8:145] MD raid6 (9/13) (w/ sdo1,sdm1,sdl1,sdk1,sdi1,sdh1,sdg1,sdf1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:10:0 ATA HGST HDN726060AL {NCGSUEPS}
│└sdk 5.46t [8:160] Partitioned (gpt)
│ └sdk1 5.46t [8:161] MD raid6 (11/13) (w/ sdo1,sdm1,sdl1,sdj1,sdi1,sdh1,sdg1,sdf1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:11:0 ATA WDC WD20EARS-00J {WD-WCAYY0201907}
│└sdl 1.82t [8:176] Partitioned (gpt)
│ └sdl1 1.82t [8:177] MD raid6 (1/13) (w/ sdo1,sdm1,sdk1,sdj1,sdi1,sdh1,sdg1,sdf1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:12:0 ATA HGST HDN726060AL {NCGD6PVS}
│└sdm 5.46t [8:192] Partitioned (gpt)
│ └sdm1 5.46t [8:193] MD raid6 (4/13) (w/ sdo1,sdl1,sdk1,sdj1,sdi1,sdh1,sdg1,sdf1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:x:x:x [Empty]
├scsi 0:0:14:0 ATA HGST HDN726060AL {NAHUM8DY}
│└sdo 5.46t [8:224] Partitioned (gpt)
│ └sdo1 5.46t [8:225] MD raid6 (10/13) (w/ sdm1,sdl1,sdk1,sdj1,sdi1,sdh1,sdg1,sdf1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
└scsi 0:x:x:x [Empty]
Liwei
(Also, sorry about the html crud in the previous email, turns out my antivirus just gained the ability to inject stuff into emails composed from Gmail within a browser. Very sketchy behaviour.)
(Resend because I did mess up the plain-text setting, sorry that you're receiving this twice!)
PCI [mpt3sas] 02:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS3224 PCI-Express Fusion-MPT SAS-3 (rev 01)
├scsi 0:0:0:0 ATA WDC WD2001FASS-0 {WD-WMAY01416344}
│└sda 1.82t [8:0] Partitioned (gpt)
│ └sda1 1.82t [8:1] MD raid6 (7/13) (w/ sdo1,sdm1,sdl1,sdk1,sdj1,sdi1,sdh1,sdg1,sdf1,sde1,sdc1,sdb1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ │ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
│ └VG Carroll 20.01t 0 free {9fcFdE-g501-tLfQ-Lg5k-zCYS-fhw3-vEJI1Y}
│ └dm-12 20.01t [253:12] LV Wonderland btrfs 'Wonderland' {f710ec85-b277-45a4-a526-e970f74e2781}
│ └Mounted as /dev/mapper/Carroll-Wonderland @ /mnt/Wonderland
├scsi 0:0:1:0 ATA WDC WD20EARS-00J {WD-WCAWZ1869671}
│└sdb 1.82t [8:16] Partitioned (gpt)
│ └sdb1 1.82t [8:17] MD raid6 (2/13) (w/ sdo1,sdm1,sdl1,sdk1,sdj1,sdi1,sdh1,sdg1,sdf1,sde1,sdc1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:2:0 ATA WDC WD20EFRX-68E {WD-WCC4M6VTHJRR}
│└sdc 1.82t [8:32] Partitioned (gpt)
│ └sdc1 1.82t [8:33] MD raid6 (5/13) (w/ sdo1,sdm1,sdl1,sdk1,sdj1,sdi1,sdh1,sdg1,sdf1,sde1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:3:0 ATA HGST HDN726060AL {NCGGBM7S}
│└sdd 5.46t [8:48] Empty/Unknown
├scsi 0:0:4:0 ATA WDC WD20EFRX-68E {WD-WCC4M6YJN0SD}
│└sde 1.82t [8:64] Partitioned (gpt)
│ └sde1 1.82t [8:65] MD raid6 (0/13) (w/ sdo1,sdm1,sdl1,sdk1,sdj1,sdi1,sdh1,sdg1,sdf1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:5:0 ATA WDC WD20EFRX-68E {WD-WCC4M6YJNEP7}
│└sdf 1.82t [8:80] Partitioned (gpt)
│ └sdf1 1.82t [8:81] MD raid6 (3/13) (w/ sdo1,sdm1,sdl1,sdk1,sdj1,sdi1,sdh1,sdg1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:6:0 ATA WDC WD6002FFWX-6 {K1GLBHMD}
│└sdg 5.46t [8:96] Partitioned (gpt)
│ └sdg1 5.46t [8:97] MD raid6 (12/13) (w/ sdo1,sdm1,sdl1,sdk1,sdj1,sdi1,sdh1,sdf1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:7:0 ATA WDC WD20EZRX-00D {WD-WMC4N2764539}
│└sdh 1.82t [8:112] Partitioned (gpt)
│ └sdh1 1.82t [8:113] MD raid6 (6/13) (w/ sdo1,sdm1,sdl1,sdk1,sdj1,sdi1,sdg1,sdf1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:8:0 ATA WDC WD20EZRX-00D {WD-WCC300583284}
│└sdi 1.82t [8:128] Partitioned (gpt)
│ └sdi1 1.82t [8:129] MD raid6 (8/13) (w/ sdo1,sdm1,sdl1,sdk1,sdj1,sdh1,sdg1,sdf1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:9:0 ATA WDC WD20EARS-00M {WD-WCAZA5414083}
│└sdj 1.82t [8:144] Partitioned (gpt)
│ └sdj1 1.82t [8:145] MD raid6 (9/13) (w/ sdo1,sdm1,sdl1,sdk1,sdi1,sdh1,sdg1,sdf1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:10:0 ATA HGST HDN726060AL {NCGSUEPS}
│└sdk 5.46t [8:160] Partitioned (gpt)
│ └sdk1 5.46t [8:161] MD raid6 (11/13) (w/ sdo1,sdm1,sdl1,sdj1,sdi1,sdh1,sdg1,sdf1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:11:0 ATA WDC WD20EARS-00J {WD-WCAYY0201907}
│└sdl 1.82t [8:176] Partitioned (gpt)
│ └sdl1 1.82t [8:177] MD raid6 (1/13) (w/ sdo1,sdm1,sdk1,sdj1,sdi1,sdh1,sdg1,sdf1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:0:12:0 ATA HGST HDN726060AL {NCGD6PVS}
│└sdm 5.46t [8:192] Partitioned (gpt)
│ └sdm1 5.46t [8:193] MD raid6 (4/13) (w/ sdo1,sdl1,sdk1,sdj1,sdi1,sdh1,sdg1,sdf1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
├scsi 0:x:x:x [Empty]
├scsi 0:0:14:0 ATA HGST HDN726060AL {NAHUM8DY}
│└sdo 5.46t [8:224] Partitioned (gpt)
│ └sdo1 5.46t [8:225] MD raid6 (10/13) (w/ sdm1,sdl1,sdk1,sdj1,sdi1,sdh1,sdg1,sdf1,sde1,sdc1,sdb1,sda1) in_sync 'NLEStore:Alice' {394b9087-268a-a957-f7f1-d4c4ccefb191}
│ └md126 20.01t [9:126] MD v1.2 raid6 (13) clean, 512k Chunk {394b9087:268aa957:f7f1d4c4:ccefb191}
│ PV LVM2_member 20.01t used, 0 free {8PhZ2b-wiZ1-ok9L-ClLw-OQUy-GrfE-eTo0Hw}
└scsi 0:x:x:x [Empty]
