> -----Original Message----- > From: dan.j.williams@xxxxxxxxx [mailto:dan.j.williams@xxxxxxxxx] On > Behalf Of Dan Williams > Sent: Wednesday, March 31, 2010 12:14 AM > To: Kwolek, Adam > Cc: Neil Brown; linux-raid@xxxxxxxxxxxxxxx; Ciechanowski, Ed > Subject: Re: Problem with changing array_size > > On Tue, Mar 30, 2010 at 6:53 AM, Kwolek, Adam <adam.kwolek@xxxxxxxxx> > wrote: > > I found that setting new raid volume size using array_size entry in > sysfs causes that inode references in vfs can be lost. > > > > Conditions: > > SLES 11 (32bit) machine with > > kernel: 2.6.33 > > mdadm : 3.1.2 > > 5 SATA hdds in system: > > SLES 11 on /dev/sda > > 4 test disks /dev/hdb - /dev/hde > > > > Reproduction: > > 1. Configure: > > # create native raid5 volume, with specified size (relatively > small) > > mdadm -C /dev/md/raid5vol_0 -amd -l 5 --chunk 64 --size 104857 > -n 3 /dev/sdb /dev/sdc /dev/sde -R > > > > # Create file system > > mkfs /dev/md/raid5vol_0 > > > > # mount it and copy some files there > > mount /dev/md/raid5vol_0 /mnt/vol > > cp * /mnt/vol > > > > 2. Try to change volume size. This should affect volume, but not vfs > > echo "157285" >/sys/block/md127/md/array_size > > Of course this affects the filesystem. You have reduced the size of > the volume from 209,664KB to 157,285KB. Perhaps you are confusing the > --size parameter to mdadm (which is per member) and the md/array_size > attribute which is the total volume size?? > > > > In the few tries (maybe at first shot) you would probably find that > when you are trying to execute ls command on mounted volume message in > user space appears: > > ls: cannot access 'copied file name': Input/output error > > In kernel space messages can be found: > > In response to size change: > > Mar 30 16:27:03 linux kernel: md127: detected capacity change > from 214695936 to 161059840 > > Mar 30 16:27:03 linux kernel: VFS: busy inodes on changed > media or resized disk md127 > > Yes, the filesystem is horribly confused at this point because 50MB of > space has disappeared. > > -- > Dan OK, you are right. This was not the best example - my mistake, but I've got another ;), I hope better. To real array size increase, grow has to be used (everything for native meta case as previously). 1. Configure (part 1 - the same as before): #create volume mdadm -C /dev/md/raid5vol_0 -amd -l 5 --chunk 64 --size 104857 -n 3 /dev/sdb /dev/sdc /dev/sde -R #create filesystem mkfs /dev/md/raid5vol_0 # mount it and copy some files there mount /dev/md/raid5vol_0 /mnt/vol cp * /mnt/vol 2. Configure (part 2 - additions) #add spare to be ready for OLCE mdadm --add /dev/md/raid5vol_0 /dev/sdd #set the array_size to value reported by mdstat, #to switch to external array size management (in md.c mddev->external_size variable is set) # if both size set are removed - result is the same echo "209664" >/sys/block/md127/md/array_size #start reshape to add spare to array (OLCE on native meta) mdadm --grow /dev/md/raid5vol_0 --raid-devices 4 --backup-file=/backup_file994.bak 3. Size changes #after reshape vfs is ok, but now ... #set size to value that is set when md increases array size itself echo "314496" >/sys/block/md127/md/array_size mdstats output before and after reshape are: Before reshape: Personalities : [raid6] [raid5] [raid4] md127 : active raid5 sdd[4](S) sde[3] sdc[1] sdb[0] 209664 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/3] [UUU] unused devices: <none> After reshape: Personalities : [raid6] [raid5] [raid4] md127 : active raid5 sdd[4] sde[3] sdc[1] sdb[0] 314496 blocks super 1.2 level 5, 64k chunk, algorithm 2 [4/4] [UUUU] unused devices: <none> The result is the same as in previous example - file system is lost. The operations with array_size in example are only for demonstration what operation is the problem cause. Without those 2 operation (mddev->external_size is not set in md), vfs is corrupted also just after reshape (in md during setting size in raid5_finish_reshape()). With both array_size set operations just after reshape vfs is ok, increasing size kills vfs. BR Adam -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
