> -----Original Message----- > From: nixbugz [mailto:nixbugz@xxxxxxxxxxx] > Sent: Wednesday, April 2, 2014 3:35 AM > To: Jiang, Dave > Cc: linux-raid@xxxxxxxxxxxxxxx; Paszkiewicz, Artur > Subject: Re: Can't grow IMSM RAID5 > > -----Original Message----- > From: Jiang, Dave > Sent: Tuesday, April 1, 2014 7:05 PM > To: nixbugz@xxxxxxxxxxx > Cc: linux-raid@xxxxxxxxxxxxxxx ; Paszkiewicz, Artur > Subject: Re: Can't grow IMSM RAID5 > > On Tue, 2014-04-01 at 18:34 +0100, nixbugz wrote: > > -----Original Message----- > > From: Jiang, Dave > > Sent: Tuesday, April 1, 2014 5:08 PM > > To: nixbugz@xxxxxxxxxxx > > Cc: linux-raid@xxxxxxxxxxxxxxx ; Paszkiewicz, Artur > > Subject: Re: Can't grow IMSM RAID5 > > > > On Tue, 2014-04-01 at 16:42 +0100, nixbugz wrote: > > > Hello > > > > > > I’m stuck trying to add a 4th disc to an IMSM RAID5 container: > > > > > > # mdadm -a /dev/md127 /dev/sdb > > > mdadm: added /dev/sdb > > > > > > # mdadm --grow --raid-devices=4 > > > /dev/md127 --backup-file=/mnt/spare/raid-backup-file > > > mdadm: Cannot read superblock for /dev/md127 > > Is your OS installed on this array? What distribution is this? Is it systemd based? It appears that there is an issue with mdmon in such case. It can't access a new disk to write metadata. You can check this by doing mdadm -E /dev/sdb. If it doesn't show any metadata on this disk but the disk is in the container, then it is this problem. As a workaround, please try this: 1. Remove the new disk from the container # mdadm -r /dev/md127 /dev/sdb 2. Takeover mdmon for this array # mdmon --takeover /dev/md127 3. Add the disk again and grow the array: # mdadm -a /dev/md127 /dev/sdb # export MDADM_EXPERIMENTAL=1 # mdadm --grow --raid-devices=4 /dev/md127 --backup-file=/mnt/spare/raid-backup-file > > I think you need to grow the RAID volume and not the container? So it > > would be /dev/md0 or /dev/md1 instead of /dev/md127? Here's URL to the > > Linux IMSM user's manual that hopefully may be of use: > > http://www.intel.com/support/chipsets/rste/sb/CS-033622.htm > > > > > > > > # cat /proc/mdstat > > > Personalities : [raid6] [raid5] [raid4] > > > md0 : active raid5 sdd[2] sda[1] sdc[0] > > > 1887436800 blocks super external:/md127/0 level 5, 128k chunk, > > > algorithm 0 [3/3] [UUU] > > > > > > md1 : active (auto-read-only) raid5 sdc[2] sdd[1] sda[0] > > > 66077696 blocks super external:/md127/1 level 5, 128k chunk, > > > algorithm > > > 0 [3/3] [UUU] > > > > > > md127 : inactive sdb[3](S) sda[2](S) sdc[1](S) sdd[0](S) > > > 12612 blocks super external:imsm > > > > > > unused devices: <none> > > > > > > # mdadm –V > > > mdadm - v3.3 - 3rd September 2013 > > > > > > > > > I don’t know if this is related but mdmon has trouble finding the ports: > > > > > > # mdadm --detail-platform -v > > > mdadm: checking metadata 0.90 > > > mdadm: 0.90 metadata is platform independent > > > mdadm: checking metadata 1.x > > > mdadm: 1.x metadata is platform independent > > > mdadm: checking metadata ddf > > > mdadm: ddf metadata is platform independent > > > mdadm: checking metadata imsm > > > mdmon: found Intel(R) SATA RAID controller at 0000:00:1f.2. > > > Platform : Intel(R) Matrix Storage Manager > > > Version : 11.6.0.1702 > > > RAID Levels : raid0 raid1 raid10 raid5 > > > Chunk Sizes : 4k 8k 16k 32k 64k 128k > > > 2TB volumes : supported > > > 2TB disks : supported > > > Max Disks : 6 > > > Max Volumes : 2 per array, 4 per controller > > > I/O Controller : /sys/devices/pci0000:00/0000:00:1f.2 (SATA) > > > mdmon: failed to determine port number for > > > /sys/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0 > > > mdmon: failed to enumerate ports on SATA controller at 0000:00:1f.2. > > > mdadm: checking metadata mbr > > > mdadm: mbr metadata is platform independent > > > mdadm: checking metadata gpt > > > mdadm: gpt metadata is platform independent > > > # > > > > Artur, any ideas why mdmon isn't happy in this instance? > > This is not related to the issue with grow. I think this is caused be a change is sysfs which mdadm is not aware of (the ata1 dir before host0). It won't cause any problems, but it should be fixed. I will look into this. > > Thanks for your quick response. I wasn't sure whether to grow container > > or > > volume but this from man mdadm decided it: > > > > Using GROW on containers is currently supported only for Intel's > > IMSM > > container format. The number of devices in a container can > > be > > increased - which affects all arrays in the container ... > > > > It gives the same message anyway when applied to the volume md0. > > > > This extract from the Intel doc you linked to says the same (note that md0 > > is the container here :-) > > > > The example below adds a single disk to the RAID container and then > > grows the > > volume(s). Because IMSM volumes inside a container must span the same > > number of disks, all volumes are expanded. > > ... > > mdadm –a /dev/md0 /dev/sde > > mdadm –G /dev/md0 –n 4 --backup-file=/tmp/backup > > > > Might it be something to do with what's already on the new drive? > > This is an Intel DQ77MK motherboard. > > Ooops my bad. It's been a while since I played with the grow feature. > Even if there's something on the drive, it doesn't make sense to me why > it complains not able to read the superblock on /dev/md127. And adding > the drive to the container should've updated the super block on that > drive I would think.... Although I wonder if the mdmon issue is a > clue.... I assume all the drives are attached to the Intel AHCI > controller right? Maybe Artur will have a better idea as he is much more > familiar with the actual code. > N���r�y��زX��vؖ)�{nlj��{���{y�ʇ��j �f����z��w����j:v��wjض�� ��zZ�����j�� > > > > > > The 4 drives are on the first 4 SATA ports - the Intel RST ports. > As for mdmon, my C is very rusty but it looks like this routine in > super-intel.c is expecting dirs called "host[0-9]*": > > static int ahci_get_port_count(const char *hba_path, int *port_count) > { > struct dirent *ent; > DIR *dir; > int host_base = -1; > > *port_count = 0; > if ((dir = opendir(hba_path)) == NULL) > return -1; > > for (ent = readdir(dir); ent; ent = readdir(dir)) { > int host; > > if (sscanf(ent->d_name, "host%d", &host) != 1) > continue; > > > which don't exist in the file-system at that point > > $ ls /sys/bus/pci/drivers/ahci/0000:00:1f.2 > ata1 d3cold_allowed modalias resource1 > ata2 device msi_bus resource2 > ata3 dma_mask_bits msi_irqs resource3 > ata4 driver numa_node resource4 > ata5 enable power resource5 > ata6 firmware_node remove subsystem > broken_parity_status iommu_group rescan subsystem_device > class irq reset subsystem_vendor > config local_cpulist resource uevent > consistent_dma_mask_bits local_cpus resource0 vendor > $ > > but do in the ata* dirs > > $ ls /sys/bus/pci/drivers/ahci/0000:00:1f.2/ata1 > ata_port host0 link1 power uevent > $ > > One more thing, is it normal to see this message repeated on shutdown? > mdadm: Cannot get exclusive access to /dev/md0:Perhaps a running process, > mounted filesystem or active volume group? > ��.n��������+%������w��{.n�����{����w��ܨ}���Ơz�j:+v�����w����ޙ��&�)ߡ�a����z�ޗ���ݢj��w�f
