I ran into this same issue, and found that a reboot ended up setting the
ownership correctly. If you look at /lib/udev/rules.d/95-ceph-osd.rules
you'll see the magic that makes it happen:
# JOURNAL_UUID
ACTION=="add", SUBSYSTEM=="block", \
ENV{DEVTYPE}=="partition", \
ENV{ID_PART_ENTRY_TYPE}=="45b0969e-9b03-4f30-b4c6-b4b80ceff106", \
OWNER:="ceph", GROUP:="ceph", MODE:="660", \
RUN+="/usr/sbin/ceph-disk --log-stdout -v trigger /dev/$name"
ACTION=="change", SUBSYSTEM=="block", \
ENV{ID_PART_ENTRY_TYPE}=="45b0969e-9b03-4f30-b4c6-b4b80ceff106", \
OWNER="ceph", GROUP="ceph", MODE="660"
Bryan
On 1/4/16, 2:39 PM, "ceph-users on behalf of Stuart Longland"
<ceph-users-bounces@xxxxxxxxxxxxxx on behalf of stuartl@xxxxxxxxxx> wrote:
>Hi all,
>
>I just did an update of a storage cluster here, or rather, I've done one
>node out of three updating to Infernalis from Hammer.
>
>I shut down the daemons, as per the guide, then did a recursive chown of
>the /var/lib/ceph directory, then struck the following when re-starting:
>
>> 2016-01-05 07:32:09.114197 7f5b41d0f940 0 ceph version 9.2.0
>>(bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-osd, pid 2899
>> 2016-01-05 07:32:09.123740 7f5b41d0f940 0
>>filestore(/var/lib/ceph/osd/ceph-0) backend xfs (magic 0x58465342)
>> 2016-01-05 07:32:09.124047 7f5b41d0f940 0
>>genericfilestorebackend(/var/lib/ceph/osd/ceph-0) detect_features:
>>FIEMAP ioctl is disabl
>> ed via 'filestore fiemap' config option
>> 2016-01-05 07:32:09.124053 7f5b41d0f940 0
>>genericfilestorebackend(/var/lib/ceph/osd/ceph-0) detect_features:
>>SEEK_DATA/SEEK_HOLE is
>> disabled via 'filestore seek data hole' config option
>> 2016-01-05 07:32:09.124066 7f5b41d0f940 0
>>genericfilestorebackend(/var/lib/ceph/osd/ceph-0) detect_features:
>>splice is supported
>> 2016-01-05 07:32:09.156182 7f5b41d0f940 0
>>genericfilestorebackend(/var/lib/ceph/osd/ceph-0) detect_features:
>>syncfs(2) syscall full
>> y supported (by glibc and kernel)
>> 2016-01-05 07:32:09.156301 7f5b41d0f940 0
>>xfsfilestorebackend(/var/lib/ceph/osd/ceph-0) detect_features: extsize
>>is supported and y
>> our kernel >= 3.5
>> 2016-01-05 07:32:09.232801 7f5b41d0f940 0
>>filestore(/var/lib/ceph/osd/ceph-0) mount: enabling WRITEAHEAD journal
>>mode: checkpoint i
>> s not enabled
>> 2016-01-05 07:32:09.253440 7f5b41d0f940 -1
>>filestore(/var/lib/ceph/osd/ceph-0) mount failed to open journal
>>/dev/sdc5: (13) Permissi
>> on denied
>> 2016-01-05 07:32:09.263646 7f5b41d0f940 -1 osd.0 0 OSD:init: unable to
>>mount object store
>> 2016-01-05 07:32:09.263656 7f5b41d0f940 -1 ESC[0;31m ** ERROR: osd init
>>failed: (13) Permission deniedESC[0m
>
>Things did not co-operate until I chown'ed /dev/sdc5 (and /dev/sdc6) to
>ceph:ceph. (-R in /var/lib/ceph was not sufficient). Even adding ceph
>to the 'disk' group (who owns /dev/sdc5) oddly enough, was not sufficient.
>
>I have that node running, and will do the others, but I am concerned
>about what happens after a reboot. Is it necessary to configure udev to
>chown /dev/sdc[56] at boot or is there some way to fix ceph's permissions?
>--
> _ ___ Stuart Longland - Systems Engineer
>\ /|_) | T: +61 7 3535 9619
> \/ | \ | 38b Douglas Street F: +61 7 3535 9699
> SYSTEMS Milton QLD 4064 http://www.vrt.com.au
>_______________________________________________
>ceph-users mailing list
>ceph-users@xxxxxxxxxxxxxx
>http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
________________________________
This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
