On Wed, 24 Jul 2013, Eric Eastman wrote: > I still have much to learn about how ceph is built. > > The ceph-deploy list command is now working with my system using a cciss boot > disk. Excellent. Thanks for testing! > Tomorrow I will bring up a HP system that has multiple cciss disks installed > and see if I can use some of them as OSD devices > > On this system, I have Linux multipath turned on, as some of the disks have > more then 1 path to each disk, but ceph-deploy/ceph-disk does not seem to > handle using dm devices. > As I have 2 paths to each physical device, there are issues trying to access > them using /dev/sdX, as there are 2 /dev/sdX devices per physical disk. The > ceph software > does not seem to handle this case, so on my running OSD servers, I have > limited my paths to only 1 per physical device. This prevents any path > redundancy. Here is what the > dm devices look like: As you can see it does not currently handle multipath in a seamless way. You can always set the multipath up yourself, however, partition as you see fit, and then specify the device names yourself, e.g. ceph-deploy osd create HOST:/dev/dm/foo:/dev/dm/bar I believe the gap currently with this arrangement is that the volumes won't be automatically mounted, in which case you'd need to add /etc/fstab entries do to that yourself. sage > > root@ceph11:~# ls -l /dev/dm-* > brw-rw---- 1 root disk 252, 0 Jul 23 23:54 /dev/dm-0 > brw-rw---- 1 root disk 252, 1 Jul 23 23:54 /dev/dm-1 > brw-rw---- 1 root disk 252, 10 Jul 23 23:54 /dev/dm-10 > brw-rw---- 1 root disk 252, 11 Jul 23 23:54 /dev/dm-11 > brw-rw---- 1 root disk 252, 12 Jul 23 23:54 /dev/dm-12 > brw-rw---- 1 root disk 252, 13 Jul 23 23:54 /dev/dm-13 > brw-rw---- 1 root disk 252, 14 Jul 23 23:54 /dev/dm-14 > brw-rw---- 1 root disk 252, 15 Jul 23 23:54 /dev/dm-15 > brw-rw---- 1 root disk 252, 16 Jul 23 23:54 /dev/dm-16 > brw-rw---- 1 root disk 252, 17 Jul 23 23:54 /dev/dm-17 > brw-rw---- 1 root disk 252, 18 Jul 23 23:54 /dev/dm-18 > brw-rw---- 1 root disk 252, 19 Jul 23 23:54 /dev/dm-19 > brw-rw---- 1 root disk 252, 2 Jul 23 23:54 /dev/dm-2 > brw-rw---- 1 root disk 252, 3 Jul 23 23:54 /dev/dm-3 > brw-rw---- 1 root disk 252, 4 Jul 23 23:54 /dev/dm-4 > brw-rw---- 1 root disk 252, 5 Jul 23 23:54 /dev/dm-5 > brw-rw---- 1 root disk 252, 6 Jul 23 23:54 /dev/dm-6 > brw-rw---- 1 root disk 252, 7 Jul 23 23:54 /dev/dm-7 > brw-rw---- 1 root disk 252, 8 Jul 23 23:54 /dev/dm-8 > brw-rw---- 1 root disk 252, 9 Jul 23 23:54 /dev/dm-9 > root@ceph11:~# ls -l /dev/disk/by-id/dm-* > lrwxrwxrwx 1 root root 11 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x2000010006 -> ../../dm-11 > lrwxrwxrwx 1 root root 11 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x2000010007 -> ../../dm-10 > lrwxrwxrwx 1 root root 10 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x2000010008 -> ../../dm-9 > lrwxrwxrwx 1 root root 10 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x2000010009 -> ../../dm-7 > lrwxrwxrwx 1 root root 11 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x200001000a -> ../../dm-14 > lrwxrwxrwx 1 root root 11 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x200001000b -> ../../dm-13 > lrwxrwxrwx 1 root root 10 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x200001000c -> ../../dm-8 > lrwxrwxrwx 1 root root 11 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x200001000d -> ../../dm-15 > lrwxrwxrwx 1 root root 11 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x200001000e -> ../../dm-16 > lrwxrwxrwx 1 root root 11 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x200001000f -> ../../dm-12 > lrwxrwxrwx 1 root root 11 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x2000010010 -> ../../dm-18 > lrwxrwxrwx 1 root root 11 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x2000010011 -> ../../dm-19 > lrwxrwxrwx 1 root root 11 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x2000010012 -> ../../dm-17 > lrwxrwxrwx 1 root root 10 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x2000010013 -> ../../dm-4 > lrwxrwxrwx 1 root root 10 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x2000010014 -> ../../dm-6 > lrwxrwxrwx 1 root root 10 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x2000010015 -> ../../dm-1 > lrwxrwxrwx 1 root root 10 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x2000010016 -> ../../dm-5 > lrwxrwxrwx 1 root root 10 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x2000010017 -> ../../dm-2 > lrwxrwxrwx 1 root root 10 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x2000010018 -> ../../dm-3 > lrwxrwxrwx 1 root root 10 Jul 23 23:54 > /dev/disk/by-id/dm-name-1IET\x20\x20\x20\x20\x2000010019 -> ../../dm-0 > > Thank you for all the help. > > Eric > > > On Wed, 24 Jul 2013, Eric Eastman wrote: > > > Still looks the same. I even tried doing a purge to make sure tried > a purge > > > to make sure everything was clean > > > > Sorry, I just pushed the branch... I forgot to mention it will take > 10-15 > > minutes for it to build and update at gitbuilder.ceph.com. Keep an > eye on > > > > http://ceph.com/gitbuilder.cgi > > > > and wait for the branch name to turn yellow for your distro. > > > > sage > > > > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |