Hi Neil
I think this is a bug.
Essentially if I create an auto=part md device then I get md_d0p? partitions.
If I stop the array and just re-assemble, I don't.
It looks like the same (?) problem as Mike (see below - Mike do you have a
patch?) but I'm on 2.6.20.7 with mdadm v2.5.6
FWIW I upgraded from 2.6.16 where it worked (but used in-kernel detection which
isn't working in 2.6.20 for some reason but I don't mind).
Here's a simple sequence of commands:
teak:~# mdadm --stop /dev/md_d0
mdadm: stopped /dev/md_d0
teak:~# mdadm --create /dev/md_d0 -l5 -n5 --bitmap=internal -e1.2 --auto=part
--name media --force /dev/sde1 /dev/sdc1 /dev/sdd1 missing /dev/sdf1
mdadm: /dev/sde1 appears to be part of a raid array:
level=raid5 devices=5 ctime=Mon Apr 23 15:02:13 2007
mdadm: /dev/sdc1 appears to be part of a raid array:
level=raid5 devices=5 ctime=Mon Apr 23 15:02:13 2007
mdadm: /dev/sdd1 appears to be part of a raid array:
level=raid5 devices=5 ctime=Mon Apr 23 15:02:13 2007
mdadm: /dev/sdf1 appears to be part of a raid array:
level=raid5 devices=5 ctime=Mon Apr 23 15:02:13 2007
Continue creating array? y
mdadm: array /dev/md_d0 started.
teak:~# grep md /proc/partitions
254 0 1250241792 md_d0
254 1 1250144138 md_d0p1
254 2 97652 md_d0p2
teak:~# mdadm --stop /dev/md_d0
mdadm: stopped /dev/md_d0
teak:~# mdadm --assemble /dev/md_d0 --auto=part /dev/sde1 /dev/sdc1 /dev/sdd1
/dev/sdf1
mdadm: /dev/md_d0 has been started with 4 drives (out of 5).
teak:~# grep md /proc/partitions
254 0 1250241792 md_d0
If I then run cfdisk it finds the partition table. I write this and get:
teak:~# cfdisk /dev/md_d0
Disk has been changed.
WARNING: If you have created or modified any
DOS 6.x partitions, please see the cfdisk manual
page for additional information.
teak:~# grep md /proc/partitions
254 0 1250241792 md_d0
254 1 1250144138 md_d0p1
254 2 97652 md_d0p2
and the syslog:
Apr 23 15:13:13 localhost kernel: md: md_d0 stopped.
Apr 23 15:13:13 localhost kernel: md: unbind<sde1>
Apr 23 15:13:13 localhost kernel: md: export_rdev(sde1)
Apr 23 15:13:13 localhost kernel: md: unbind<sdf1>
Apr 23 15:13:13 localhost kernel: md: export_rdev(sdf1)
Apr 23 15:13:13 localhost kernel: md: unbind<sdd1>
Apr 23 15:13:13 localhost kernel: md: export_rdev(sdd1)
Apr 23 15:13:13 localhost kernel: md: unbind<sdc1>
Apr 23 15:13:13 localhost kernel: md: export_rdev(sdc1)
Apr 23 15:13:13 localhost mdadm: DeviceDisappeared event detected on md device
/dev/md_d0
Apr 23 15:13:36 localhost kernel: md: bind<sde1>
Apr 23 15:13:36 localhost kernel: md: bind<sdc1>
Apr 23 15:13:36 localhost kernel: md: bind<sdd1>
Apr 23 15:13:36 localhost kernel: md: bind<sdf1>
Apr 23 15:13:36 localhost kernel: raid5: device sdf1 operational as raid disk 4
Apr 23 15:13:36 localhost kernel: raid5: device sdd1 operational as raid disk 2
Apr 23 15:13:36 localhost kernel: raid5: device sdc1 operational as raid disk 1
Apr 23 15:13:36 localhost kernel: raid5: device sde1 operational as raid disk 0
Apr 23 15:13:36 localhost kernel: raid5: allocated 5236kB for md_d0
Apr 23 15:13:36 localhost kernel: raid5: raid level 5 set md_d0 active with 4
out of 5 devices, algorithm 2
Apr 23 15:13:36 localhost kernel: RAID5 conf printout:
Apr 23 15:13:36 localhost kernel: --- rd:5 wd:4
Apr 23 15:13:36 localhost kernel: disk 0, o:1, dev:sde1
Apr 23 15:13:36 localhost kernel: disk 1, o:1, dev:sdc1
Apr 23 15:13:36 localhost kernel: disk 2, o:1, dev:sdd1
Apr 23 15:13:36 localhost kernel: disk 4, o:1, dev:sdf1
Apr 23 15:13:36 localhost kernel: md_d0: bitmap initialized from disk: read 1/1
pages, set 19078 bits, status: 0
Apr 23 15:13:36 localhost kernel: created bitmap (10 pages) for device md_d0
Apr 23 15:13:36 localhost kernel: md_d0: p1 p2
Apr 23 15:13:54 localhost kernel: md: md_d0 stopped.
Apr 23 15:13:54 localhost kernel: md: unbind<sdf1>
Apr 23 15:13:54 localhost kernel: md: export_rdev(sdf1)
Apr 23 15:13:54 localhost kernel: md: unbind<sdd1>
Apr 23 15:13:54 localhost kernel: md: export_rdev(sdd1)
Apr 23 15:13:54 localhost kernel: md: unbind<sdc1>
Apr 23 15:13:54 localhost kernel: md: export_rdev(sdc1)
Apr 23 15:13:54 localhost kernel: md: unbind<sde1>
Apr 23 15:13:54 localhost kernel: md: export_rdev(sde1)
Apr 23 15:13:54 localhost mdadm: DeviceDisappeared event detected on md device
/dev/md_d0
Apr 23 15:14:04 localhost kernel: md: md_d0 stopped.
Apr 23 15:14:04 localhost kernel: md: bind<sdc1>
Apr 23 15:14:04 localhost kernel: md: bind<sdd1>
Apr 23 15:14:04 localhost kernel: md: bind<sdf1>
Apr 23 15:14:04 localhost kernel: md: bind<sde1>
Apr 23 15:14:04 localhost kernel: raid5: device sde1 operational as raid disk 0
Apr 23 15:14:04 localhost kernel: raid5: device sdf1 operational as raid disk 4
Apr 23 15:14:04 localhost kernel: raid5: device sdd1 operational as raid disk 2
Apr 23 15:14:04 localhost kernel: raid5: device sdc1 operational as raid disk 1
Apr 23 15:14:04 localhost kernel: raid5: allocated 5236kB for md_d0
Apr 23 15:14:04 localhost kernel: raid5: raid level 5 set md_d0 active with 4
out of 5 devices, algorithm 2
Apr 23 15:14:04 localhost kernel: RAID5 conf printout:
Apr 23 15:14:04 localhost kernel: --- rd:5 wd:4
Apr 23 15:14:04 localhost kernel: disk 0, o:1, dev:sde1
Apr 23 15:14:04 localhost kernel: disk 1, o:1, dev:sdc1
Apr 23 15:14:04 localhost kernel: disk 2, o:1, dev:sdd1
Apr 23 15:14:04 localhost kernel: disk 4, o:1, dev:sdf1
Apr 23 15:14:04 localhost kernel: md_d0: bitmap initialized from disk: read 1/1
pages, set 0 bits, status: 0
Apr 23 15:14:04 localhost kernel: created bitmap (10 pages) for device md_d0
Apr 23 15:14:04 localhost kernel: md_d0: unknown partition table
after cfdisk write:
Apr 23 15:33:00 localhost kernel: md_d0: p1 p2
Back in dec 2006,Mike Accetta wrote:
> In setting up a partitioned array as the boot disk and using a nash
> initrd to find the root file system by volume label, I see a delay in
> the appearance of the /dev/md_d0p partitions in /proc/partitions. When
> the mdadm --assemble command completes, only /dev/md_d0 is visible.
> Since the raid partitions are not visible after the assemble, the volume
> label search will not consult them in looking for the root volume and
> the boot gets aborted. When I run a similar assemble command while up
> multi-user in a friendlier debug environment I see the same effect and
> observe that pretty much any access of /dev/md_d0 has the side effect of
> then making the /dev/md_d0p partitions visible in /proc/partitions.
>
> I tried a few experiments changing the --assemble code in mdadm. If I
> open() and close() /dev/md_d0 after assembly *before* closing the file
> descriptor which the assemble step used to assemble the array, there is
> no effect. Even doing a BLKRRPART ioctl call on the assembly fd or the
> newly opened fd have no effect. The kernel prints "unknown partition"
> diagnostics on the console. However, if the assembly fd is first
> close()'d, a simple open() of /dev/md_d0 and immediate close() of that
> fd has the side effect of making the /dev/md_d0p partitions visible and
> one sees the console disk partitioning confirmation from the kernel as
> well.
>
> Adding the open()/close() after assembly within mdadm solves my problem,
> but I thought I'd raise the issue on the list as it seems there is a bug
> somewhere. I see in the kernel md driver that the RUN_ARRAY ioctl()
> calls do_md_run() which calls md_probe() which calls add_disk() and I
> gather that this would normally have the side effect of making the
> partitions visible. However, my experiments at user level seem to imply
> that the array isn't completely usable until the assembly file
> descriptor is closed, even on return from the ioctl(), and hence the
> kernel add_disk() isn't having the desired partitioning side effect at
> the point it is being invoked.
>
> This is all with kernel 2.6.18 and mdadm 2.3.1
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
