Re: pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Thanks so far for all the help. I'm still digesting it -- will try to respond one by one later.

One of the other posts prompted me to supply this additional info:

[root@loki ~]# blkid
/dev/sda1: PARTUUID="fd645cd2-d5c2-4bff-be9b-19cba21b31c1"
/dev/sda2: UUID="a90395cd-6297-a553-d29f-32cd04f3054c" UUID_SUB="3f3d4518-967a-d3f5-99bd-d5195c7d079b" LABEL="localhost-live:raid1_boot" TYPE="linux_raid_member" PARTUUID="713603aa-7e4f-4d08-b9a2-6f9c6a692357"
/dev/sda3: UUID="1ce0d10e-d281-ceb8-a53a-4dd4b00d77b1" UUID_SUB="8317645e-e13b-28c5-2154-84a7a5505b5d" LABEL="localhost-live:raid1_os-main" TYPE="linux_raid_member" PARTUUID="4a843100-e448-427c-b575-bfc2a1bc6205"
/dev/sdb1: PARTUUID="9f9a91c0-aaf4-4083-a00c-676187a990ee"
/dev/sdb2: UUID="a90395cd-6297-a553-d29f-32cd04f3054c" UUID_SUB="c2d52945-bf91-d15c-36a8-dca5ac76b9ec" LABEL="localhost-live:raid1_boot" TYPE="linux_raid_member" PARTUUID="f004e563-bab2-475d-a2ae-6e1b8d2d53d7"
/dev/sdb3: UUID="1ce0d10e-d281-ceb8-a53a-4dd4b00d77b1" UUID_SUB="dab22a3f-5528-d6e6-9da9-ce74a32cf0a6" LABEL="localhost-live:raid1_os-main" TYPE="linux_raid_member" PARTUUID="0c85fe79-4d0c-441a-87ff-75852f164a86"
/dev/sdc1: UUID="434db08c-6fdf-4494-19a2-1b166e662faf" UUID_SUB="050a5fc5-b39e-6e60-98f3-ffad71e38d71" LABEL="localhost-live:raid_big" TYPE="linux_raid_member" PARTUUID="3a97d627-78f6-4c5f-8a7d-6d005165e3c9"
/dev/sdd1: UUID="434db08c-6fdf-4494-19a2-1b166e662faf" UUID_SUB="ebb43ce0-bd68-4ff7-c856-2ac68774a702" LABEL="localhost-live:raid_big" TYPE="linux_raid_member" PARTUUID="e13a3833-4fdc-48d8-8dac-6993d3a749c7"
/dev/md127: UUID="2k3Khh-ET0b-e1p6-Q36V-Epj0-qwui-CeV1YH" TYPE="LVM2_member"
/dev/md126: UUID="84ed704c-b983-4ce6-8ddd-42b40fa4a407" BLOCK_SIZE="4096" TYPE="ext4"
/dev/mapper/lv_os--main-root: UUID="c08c6b29-7999-4677-b046-594a6685af75" BLOCK_SIZE="4096" TYPE="ext4"
/dev/mapper/lv_os--main-swap: UUID="96d78117-f933-4931-a3ac-033251041bf2" TYPE="swap"
/dev/md125: UUID="6p0Cks-EHQ9-i95n-fJIk-upXg-Qket-ZcOobc" TYPE="LVM2_member"
/dev/mapper/lv_big-00: LABEL="home" UUID="e64f9b00-a4ec-4466-a001-09c96f5aff0a" BLOCK_SIZE="4096" TYPE="ext4"
/dev/mapper/lv_big-var: UUID="f42563bc-bb66-4837-b357-72358d404e32" BLOCK_SIZE="4096" TYPE="ext4"

[root@loki ~]# lsblk
NAME                   MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                      8:0    0 465.8G  0 disk  
├─sda1                   8:1    0     2M  0 part  
├─sda2                   8:2    0     1G  0 part  
│ └─md126                9:126  0  1022M  0 raid1 /boot
└─sda3                   8:3    0 464.8G  0 part  
  └─md127                9:127  0 464.6G  0 raid1 
    ├─lv_os--main-root 253:0    0   370G  0 lvm   /
    └─lv_os--main-swap 253:1    0     8G  0 lvm   [SWAP]
sdb                      8:16   0 465.8G  0 disk  
├─sdb1                   8:17   0     2M  0 part  
├─sdb2                   8:18   0     1G  0 part  
│ └─md126                9:126  0  1022M  0 raid1 /boot
└─sdb3                   8:19   0 464.8G  0 part  
  └─md127                9:127  0 464.6G  0 raid1 
    ├─lv_os--main-root 253:0    0   370G  0 lvm   /
    └─lv_os--main-swap 253:1    0     8G  0 lvm   [SWAP]
sdc                      8:32   0   3.7T  0 disk  
└─sdc1                   8:33   0   3.7T  0 part  
  └─md125                9:125  0   3.7T  0 raid1 
    ├─lv_big-00        253:2    0   2.3T  0 lvm   /home
    └─lv_big-var       253:3    0   400G  0 lvm   /var
sdd                      8:48   0   3.7T  0 disk  
└─sdd1                   8:49   0   3.7T  0 part  
  └─md125                9:125  0   3.7T  0 raid1 
    ├─lv_big-00        253:2    0   2.3T  0 lvm   /home
    └─lv_big-var       253:3    0   400G  0 lvm   /var
sr0                     11:0    1  1024M  0 rom

--
Tom

On Tue 10/6/20 11:07 -0500 linux-raid@xxxxxxxx wrote:
>I setup a crash&burn fedora 32 workstation using two 500GB SSD /dev/sd{a,b}; and two 4TB disks
>/dev/sd{c,d}.
>
>I'm effectively a raid and lvm newbie, since I only do this type of thing once every 5 years or so.
>
>Through the blivot GUI, using trial and error I got a bootable system installed with
>raid1 (and lvm) as hinted:
>
>* GPT disk label wrote to both SSD disks /dev/sd{a,b}.
>
>* on /dev/sd{a,b}1 created 2MiB "bios boot" partition. I assume only one is used? I could try clobbering /dev/sda1 ...
>
>* on /dev/sd{a,b}2 created 1G "localhost-live:raid1_boot"  ext4 /boot filesystem under raid1.  
>
>* on /dev/sd{a,b}3 created a 464.8G "localhost-live:raid1_os-main" using the remaining disk space, as LVM ( 464.63g VG named lv_os-main )
>
>    * made 8 GiB swap partition over raid1 as /dev/mapper/lv_os--main-swap ( on the SSD )
>
>    * made 370 GiB /  ext4 partition over raid1 as /dev/mapper/lv_os--main-root
>
>* created a 3.65 TiB raid1 /dev/md125 array on /dev/sd{d,c} with the name ???? using the remaining disk space, as LVM ( 3.64t VG named lv_big )
>
>    * /dev/mapper/lv_big-var on /var  type ext4
>    * /dev/mapper/lv_big-00  on /home type ext4
>
>I was able to install and boot fedora 32 OK.
>
>----
>
>Is it possible to get the 2MiB "bios boot" partition into raid1?
>This seems to be the most vulnerable part of my setup.
>
>Let me know additional commands you need to document my setup.
>I do not know the depth of my ignorance -- any general advice is welcome.
>I can redo all of this setup per your suggestions.  
>
>--
>thanks,
>Tom
>
>--------
>[root@loki ~]# mdadm --version
>mdadm - v4.1 - 2018-10-01
>[root@loki ~]# uname -r
>5.8.9-200.fc32.x86_64
>[root@loki ~]# echo /dev/sd[a-d][0-9]
>/dev/sda1 /dev/sda2 /dev/sda3 /dev/sdb1 /dev/sdb2 /dev/sdb3 /dev/sdc1 /dev/sdd1
>[root@loki ~]# mount -t ext4
>/dev/mapper/lv_os--main-root on / type ext4 (rw,relatime,seclabel)
>/dev/md126 on /boot type ext4 (rw,relatime,seclabel)
>/dev/mapper/lv_big-var on /var type ext4 (rw,relatime,seclabel)
>/dev/mapper/lv_big-00 on /home type ext4 (rw,relatime,seclabel)
>[root@loki ~]# ls /dev/md
>raid1_boot  raid1_os-main  raid_big
>[root@loki ~]# fdisk -l    # shows: mbr or gpt label,  'bios boot' partition, raid devices, and lvm devices
>Disk /dev/sda: 465.78 GiB, 500107862016 bytes, 976773168 sectors
>Disk model: Samsung SSD 850
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>Disklabel type: gpt
>Disk identifier: 7CD40013-ACFB-433A-B6A1-B5EF63934004
>
>Device       Start       End   Sectors   Size Type
>/dev/sda1     2048      6143      4096     2M BIOS boot
>/dev/sda2     6144   2103295   2097152     1G Linux RAID
>/dev/sda3  2103296 976773119 974669824 464.8G Linux RAID
>
>
>Disk /dev/sdb: 465.78 GiB, 500107862016 bytes, 976773168 sectors
>Disk model: Samsung SSD 850
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>Disklabel type: gpt
>Disk identifier: 68F51B2D-C077-46D2-96A3-116BAF835998
>
>Device       Start       End   Sectors   Size Type
>/dev/sdb1     2048      6143      4096     2M BIOS boot
>/dev/sdb2     6144   2103295   2097152     1G Linux RAID
>/dev/sdb3  2103296 976773119 974669824 464.8G Linux RAID
>
>
>Disk /dev/sdc: 3.65 TiB, 4000787030016 bytes, 7814037168 sectors
>Disk model: ST4000VN008-2DR1
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 4096 bytes
>I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>Disklabel type: gpt
>Disk identifier: D263D0C0-2C37-4205-B505-BC3B6DD1DFDA
>
>Device     Start        End    Sectors  Size Type
>/dev/sdc1   2048 7814035491 7814033444  3.7T Linux RAID
>
>
>Disk /dev/sdd: 3.65 TiB, 4000787030016 bytes, 7814037168 sectors
>Disk model: ST4000VN008-2DR1
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 4096 bytes
>I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>Disklabel type: gpt
>Disk identifier: 84D9D98B-0C27-4090-B639-AB48A0FA9B1E
>
>Device     Start        End    Sectors  Size Type
>/dev/sdd1   2048 7814035491 7814033444  3.7T Linux RAID
>
>
>Disk /dev/md127: 464.65 GiB, 498895683584 bytes, 974405632 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>
>
>Disk /dev/md126: 1022 MiB, 1071644672 bytes, 2093056 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>
>
>Disk /dev/mapper/lv_os--main-root: 370 GiB, 397284474880 bytes, 775946240 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>
>
>Disk /dev/mapper/lv_os--main-swap: 8 GiB, 8589934592 bytes, 16777216 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>
>
>
>
>Disk /dev/md125: 3.65 TiB, 4000649838592 bytes, 7813769216 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 4096 bytes
>I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>
>
>Disk /dev/mapper/lv_big-00: 2.31 TiB, 2528880099328 bytes, 4939218944 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 4096 bytes
>I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>
>
>Disk /dev/mapper/lv_big-var: 400 GiB, 429496729600 bytes, 838860800 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 4096 bytes
>I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>[root@loki ~]# mdadm --examine /dev/sd[a-d][0-9]  ###
>mdadm: No md superblock detected on /dev/sda1.
>/dev/sda2:
>          Magic : a92b4efc
>        Version : 1.2
>    Feature Map : 0x1
>     Array UUID : a90395cd:6297a553:d29f32cd:04f3054c
>           Name : localhost-live:raid1_boot
>  Creation Time : Sat Sep 19 07:54:55 2020
>     Raid Level : raid1
>   Raid Devices : 2
>
> Avail Dev Size : 2093056 (1022.00 MiB 1071.64 MB)
>     Array Size : 1046528 (1022.00 MiB 1071.64 MB)
>    Data Offset : 4096 sectors
>   Super Offset : 8 sectors
>   Unused Space : before=4016 sectors, after=0 sectors
>          State : clean
>    Device UUID : 3f3d4518:967ad3f5:99bdd519:5c7d079b
>
>Internal Bitmap : 8 sectors from superblock
>    Update Time : Fri Sep 25 07:57:18 2020
>  Bad Block Log : 512 entries available at offset 16 sectors
>       Checksum : b6fe8069 - correct
>         Events : 83
>
>
>   Device Role : Active device 0
>   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>/dev/sda3:
>          Magic : a92b4efc
>        Version : 1.2
>    Feature Map : 0x1
>     Array UUID : 1ce0d10e:d281ceb8:a53a4dd4:b00d77b1
>           Name : localhost-live:raid1_os-main
>  Creation Time : Sat Sep 19 07:54:43 2020
>     Raid Level : raid1
>   Raid Devices : 2
>
> Avail Dev Size : 974405632 (464.63 GiB 498.90 GB)
>     Array Size : 487202816 (464.63 GiB 498.90 GB)
>    Data Offset : 264192 sectors
>   Super Offset : 8 sectors
>   Unused Space : before=264112 sectors, after=0 sectors
>          State : clean
>    Device UUID : 8317645e:e13b28c5:215484a7:a5505b5d
>
>Internal Bitmap : 8 sectors from superblock
>    Update Time : Sat Sep 26 05:40:38 2020
>  Bad Block Log : 512 entries available at offset 16 sectors
>       Checksum : cfc13cbc - correct
>         Events : 894
>
>
>   Device Role : Active device 0
>   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>mdadm: No md superblock detected on /dev/sdb1.
>/dev/sdb2:
>          Magic : a92b4efc
>        Version : 1.2
>    Feature Map : 0x1
>     Array UUID : a90395cd:6297a553:d29f32cd:04f3054c
>           Name : localhost-live:raid1_boot
>  Creation Time : Sat Sep 19 07:54:55 2020
>     Raid Level : raid1
>   Raid Devices : 2
>
> Avail Dev Size : 2093056 (1022.00 MiB 1071.64 MB)
>     Array Size : 1046528 (1022.00 MiB 1071.64 MB)
>    Data Offset : 4096 sectors
>   Super Offset : 8 sectors
>   Unused Space : before=4016 sectors, after=0 sectors
>          State : clean
>    Device UUID : c2d52945:bf91d15c:36a8dca5:ac76b9ec
>
>Internal Bitmap : 8 sectors from superblock
>    Update Time : Fri Sep 25 07:57:18 2020
>  Bad Block Log : 512 entries available at offset 16 sectors
>       Checksum : 289a1404 - correct
>         Events : 83
>
>
>   Device Role : Active device 1
>   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>/dev/sdb3:
>          Magic : a92b4efc
>        Version : 1.2
>    Feature Map : 0x1
>     Array UUID : 1ce0d10e:d281ceb8:a53a4dd4:b00d77b1
>           Name : localhost-live:raid1_os-main
>  Creation Time : Sat Sep 19 07:54:43 2020
>     Raid Level : raid1
>   Raid Devices : 2
>
> Avail Dev Size : 974405632 (464.63 GiB 498.90 GB)
>     Array Size : 487202816 (464.63 GiB 498.90 GB)
>    Data Offset : 264192 sectors
>   Super Offset : 8 sectors
>   Unused Space : before=264112 sectors, after=0 sectors
>          State : clean
>    Device UUID : dab22a3f:5528d6e6:9da9ce74:a32cf0a6
>
>Internal Bitmap : 8 sectors from superblock
>    Update Time : Sat Sep 26 05:40:38 2020
>  Bad Block Log : 512 entries available at offset 16 sectors
>       Checksum : e914f602 - correct
>         Events : 894
>
>
>   Device Role : Active device 1
>   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>/dev/sdc1:
>          Magic : a92b4efc
>        Version : 1.2
>    Feature Map : 0x1
>     Array UUID : 434db08c:6fdf4494:19a21b16:6e662faf
>           Name : localhost-live:raid_big
>  Creation Time : Sat Sep 19 07:54:00 2020
>     Raid Level : raid1
>   Raid Devices : 2
>
> Avail Dev Size : 7813769252 (3725.90 GiB 4000.65 GB)
>     Array Size : 3906884608 (3725.90 GiB 4000.65 GB)
>  Used Dev Size : 7813769216 (3725.90 GiB 4000.65 GB)
>    Data Offset : 264192 sectors
>   Super Offset : 8 sectors
>   Unused Space : before=264112 sectors, after=36 sectors
>          State : clean
>    Device UUID : 050a5fc5:b39e6e60:98f3ffad:71e38d71
>
>Internal Bitmap : 8 sectors from superblock
>    Update Time : Sat Sep 26 06:09:58 2020
>  Bad Block Log : 512 entries available at offset 24 sectors
>       Checksum : 476865a4 - correct
>         Events : 43615
>
>
>   Device Role : Active device 0
>   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>/dev/sdd1:
>          Magic : a92b4efc
>        Version : 1.2
>    Feature Map : 0x1
>     Array UUID : 434db08c:6fdf4494:19a21b16:6e662faf
>           Name : localhost-live:raid_big
>  Creation Time : Sat Sep 19 07:54:00 2020
>     Raid Level : raid1
>   Raid Devices : 2
>
> Avail Dev Size : 7813769252 (3725.90 GiB 4000.65 GB)
>     Array Size : 3906884608 (3725.90 GiB 4000.65 GB)
>  Used Dev Size : 7813769216 (3725.90 GiB 4000.65 GB)
>    Data Offset : 264192 sectors
>   Super Offset : 8 sectors
>   Unused Space : before=264112 sectors, after=36 sectors
>          State : clean
>    Device UUID : ebb43ce0:bd684ff7:c8562ac6:8774a702
>
>Internal Bitmap : 8 sectors from superblock
>    Update Time : Sat Sep 26 06:09:58 2020
>  Bad Block Log : 512 entries available at offset 24 sectors
>       Checksum : a26acedb - correct
>         Events : 43615
>
>
>   Device Role : Active device 1
>   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>[root@loki ~]# pvs
>  PV         VG         Fmt  Attr PSize    PFree
>  /dev/md125 lv_big     lvm2 a--    <3.64t 970.69g
>  /dev/md127 lv_os-main lvm2 a--  <464.63g <86.63g
>
>
>
>
>
>
>



[Index of Archives]     [Linux RAID Wiki]     [ATA RAID]     [Linux SCSI Target Infrastructure]     [Linux Block]     [Linux IDE]     [Linux SCSI]     [Linux Hams]     [Device Mapper]     [Device Mapper Cryptographics]     [Kernel]     [Linux Admin]     [Linux Net]     [GFS]     [RPM]     [git]     [Yosemite Forum]


  Powered by Linux