Hi Sage,
Our clusters are slightly different but no the monitors start just fine.
On our test and rgw clusters we run monitors co-located with our OSDs.
The monitors start just fine. My understanding is that when booting
the hosts detect a disk hot-plug event in udev via the
/lib/udev/rules.d/95-ceph-osd.rules file. Running the
'/usr/sbin/ceph-disk-activate /dev/vdb1' command for example in our test
cluster does the right thing
# /usr/sbin/ceph-disk-activate /dev/vdb1
=== osd.2 ===
create-or-move updated item name 'osd.2' weight 0.02 at location
{host=ceph02,root=default} to crush map
Starting Ceph osd.2 on ceph02...
starting osd.2 at :/0 osd_data /var/lib/ceph/osd/ceph-2
/var/lib/ceph/osd/ceph-2/journal
Our best guess so far is that this line is not matching the underlying
disk that is getting hotplugged (95-ceph-osd.rules). Is
ID_PART_ENTRY_TYPE just the partition UUID or are we not understanding
identifier correctly?
ENV{ID_PART_ENTRY_TYPE}=="4fbd7e29-9d25-41b8-afd0-062c0ceff05d",
Thanks,
derek
On 1/27/14, 11:29 AM, Sage Weil wrote:
> Hi Derek,
>
> Would like to get to the bottom of your problem. Is it that the monitors
> don't start after a reboot? Is there an error in
> /var/log/ceph/ceph-mon.`hostname`.log?
>
> sage
>
> On Mon, 27 Jan 2014, Derek Yarnell wrote:
>
>> Hi,
>>
>> Would I take this to understand that this may be a known issue with udev
>> on RHEL then? We will for now add them to the fstab.
>>
>> Thanks,
>> derek
>>
>> On 1/25/14, 9:23 PM, Michael J. Kidd wrote:
>>> While clearly not optimal for long term flexibility, I've found that
>>> adding my OSD's to fstab allows the OSDs to mount during boot, and they
>>> start automatically when they're already mounted during boot.
>>>
>>> Hope this helps until a permanent fix is available.
>>>
>>> Michael J. Kidd
>>> Sr. Storage Consultant
>>> Inktank Professional Services
>>>
>>>
>>> On Fri, Jan 24, 2014 at 9:08 PM, Derek Yarnell <derek@xxxxxxxxxxxxxx
>>> <mailto:derek@xxxxxxxxxxxxxx>> wrote:
>>>
>>> So we have a test cluster, and two production clusters all running on
>>> RHEL6.5. Two are running Emperor and one of them running Dumpling. On
>>> all of them our OSDs do not start at boot it seems via the udev rules.
>>> The OSDs were created with ceph-deploy and are all GPT. The OSDs are
>>> visable with `ceph-disk list` and running `/usr/sbin/ceph-disk-activate
>>> {device}` mounts and adds them. Running a `partprobe {device}` does not
>>> seem to trigger the udev rule at all.
>>>
>>> I had found this issue[1] but we are definitely running code that was
>>> released after this ticket was closed. Has there been anyone else that
>>> has problems with udev on RHEL mounting their OSDs?
>>>
>>> [1] - http://tracker.ceph.com/issues/5194
>>>
>>> Thanks,
>>> derek
>>>
>>> --
>>> Derek T. Yarnell
>>> University of Maryland
>>> Institute for Advanced Computer Studies
>>> _______________________________________________
>>> ceph-users mailing list
>>> ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx>
>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>
>>>
>>
>> --
>> Derek T. Yarnell
>> University of Maryland
>> Institute for Advanced Computer Studies
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@xxxxxxxxxxxxxx
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>>
>
--
Derek T. Yarnell
University of Maryland
Institute for Advanced Computer Studies
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
