On 23/04/15 15:18, Robert LeBlanc wrote: > > Sorry, reading too fast. That key isn't from a previous attempt, > correct? But I doubt that is the problem as you would receive an > access denied message in the logs. > > Try running Ceph-disk zap and recreate the OSD. Also remove the Auth > key and the osd (ceph osd rm <id>) then do a ceph-disk prepare. I > don't think the first stay up should be trying to create file systems, > that should have been done with prepare. > Hi Robert, If I ceph-disk prepare a raw drive, e.g. /dev/sdb, I get two partitions: one for data and one for journal. This then activates fine. If I ceph-disk prepare a partition and then activate, it works fine. However activate always fails if I prepare with the journal on an SSD. This is repeatable across all three machines in my test cluster. There is something wrong with ceph-disk (or ceph-osd in the background) that simply does not work with these SSDs. I have found a work around for this bug however. If I prepare a partition and then manually point the journal at the SSD partition the OSD then starts correctly. Example process if someone else hits this issue: $ CLUSTERUUID=8c9ff7b5-904a-4f9a-8c9e-d2f8b05b55d2 $ OSDUUID=`uuidgen` $ JOURUUID=`uuidgen` $ ceph-disk zap /dev/sdb $ sgdisk --new=1:0:0 --change-name=1:'ceph data' --partition-guid=1:$OSDUUID \ --typecode=1:4FBD7E29-9D25-41B8-AFD0-062C0CEFF05D --mbrtogpt -- /dev/sdb $ ceph-disk prepare --cluster ceph --cluster-uuid $CLUSTERUUID \ --osd-uuid $OSDUUID /dev/sdb1 $ ceph-disk activate /dev/sdb1 # tells me it started osd.3 $ /etc/init.d/ceph stop osd.3 $ rm /var/lib/ceph/osd/ceph-3/journal $ sgdisk --new=1:0:+10G --change-name=1:'ceph journal' \ --partition-guid=1:$JOURUUID \ --typecode=1:45B0969E-9B03-4F30-B4C6-B4B80CEFF106 --mbrtogpt -- /dev/sdc $ ln -s /dev/disk/by-partuuid/$JOURUUID /var/lib/ceph/osd/ceph-3/journal $ echo $JOURUUID > /var/lib/ceph/osd/ceph-3/journal_uuid $ ceph-osd --mkjournal -i 3 $ /etc/init.d/ceph start osd.3 OSD starts fine. After updating ceph.conf I can reboot and the OSD gets started correctly. I now have a working cluster with three OSDs using SSD journals. Cheers, Dan _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |