You should check that the owner of your ceph partitions (both journal and data) is 'ceph', otherwise the ceph user won't mount it.
You can simply do : chown ceph:disk /dev/sdc3
If this solve your issue you should set the GPT GUID [1] of the partitions with a tool like sgdisk to make this persistent across reboot.
I think only your journal is affected as ceph-disk does not prepare the partition (WARNING:ceph-disk:Journal /dev/sdc3 was not prepared with ceph-disk. Symlinking directly)
On Thu, Dec 3, 2015 at 3:03 AM, Goncalo Borges <goncalo.borges@xxxxxxxxxxxxx> wrote:
Dear Cephers...
1./ I am currently deploying infernalis 9.2.0 in centos7.
2./ I am not using ceph-deploy (I prefer not to use it) and I prefer to use the short form osd installation procedure described here:
http://docs.ceph.com/docs/infernalis/install/manual-deployment/#short-form3./ In hammer this instructions worked fine but in infernalis 9.2.0, which the implementation of the ceph user running the ceph daemons, those instructions do not seem to work properly anymore.
4./ Here is the behaviour:
4.1/ ceph-disk prepare runs fine:
# ceph-disk prepare --cluster ceph --cluster-uuid a9431bc6-3ee1-4b0a-8d21-0ad883a4d2ed --fs-type xfs /dev/sdh /dev/sdc34.2/ ceph-disk activate fails with permission problems. It seems some temporary directory us made by root which is then not accessible to ceph user.
WARNING:ceph-disk:OSD will not be hot-swappable if journal is not the same device as the osd data
WARNING:ceph-disk:Journal /dev/sdc3 was not prepared with ceph-disk. Symlinking directly.
Creating new GPT entries.
The operation has completed successfully.
meta-data="" isize=2048 agcount=4, agsize=183107519 blks
= sectsz=512 attr=2, projid32bit=1
= crc=0 finobt=0
data = bsize=4096 blocks=732430075, imaxpct=5
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=0
log =internal log bsize=4096 blocks=357631, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
The operation has completed successfully.
# ceph-disk activate /dev/sdh14.3 If I run the the ceph activate command as a ceph user, it complains it need root permissions to mount
got monmap epoch 1
2015-12-03 01:56:41.406742 7fb050fa9900 -1 filestore(/var/lib/ceph/tmp/mnt.dsi7zI) mkjournal error creating journal on /var/lib/ceph/tmp/mnt.dsi7zI/journal: (13) Permission denied
2015-12-03 01:56:41.406777 7fb050fa9900 -1 OSD::mkfs: ObjectStore::mkfs failed with error -13
2015-12-03 01:56:41.406843 7fb050fa9900 -1 ** ERROR: error creating empty object store in /var/lib/ceph/tmp/mnt.dsi7zI: (13) Permission denied
ERROR:ceph-disk:Failed to activate
Traceback (most recent call last):
File "/usr/sbin/ceph-disk", line 3576, in <module>
main(sys.argv[1:])
File "/usr/sbin/ceph-disk", line 3532, in main
main_catch(args.func, args)
File "/usr/sbin/ceph-disk", line 3554, in main_catch
func(args)
File "/usr/sbin/ceph-disk", line 2424, in main_activate
dmcrypt_key_dir=args.dmcrypt_key_dir,
File "/usr/sbin/ceph-disk", line 2197, in mount_activate
(osd_id, cluster) = activate(path, activate_key_template, init)
File "/usr/sbin/ceph-disk", line 2360, in activate
keyring=keyring,
File "/usr/sbin/ceph-disk", line 1950, in mkfs
'--setgroup', get_ceph_user(),
File "/usr/sbin/ceph-disk", line 349, in command_check_call
return subprocess.check_call(arguments)
File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/ceph-osd', '--cluster', 'ceph', '--mkfs', '--mkkey', '-i', '4', '--monmap', '/var/lib/ceph/tmp/mnt.dsi7zI/activate.monmap', '--osd-data', '/var/lib/ceph/tmp/mnt.dsi7zI', '--osd-journal', '/var/lib/ceph/tmp/mnt.dsi7zI/journal', '--osd-uuid', '995e436f-dc6f-42ca-bbbf-3ef9f0bb4952', '--keyring', '/var/lib/ceph/tmp/mnt.dsi7zI/keyring', '--setuser', 'ceph', '--setgroup', 'ceph']' returned non-zero exit status 1
5./ So there is an incoherence on this set of instructions.
# su ceph -s /bin/bash --session-command="ceph-disk activate /dev/sdh1"
Problem opening /dev/sdh for reading! Error is 13.
You must run this program as root or use sudo!
mount: only root can use "--options" option
ceph-disk: Mounting filesystem failed: Command '['/usr/bin/mount', '-t', 'xfs', '-o', 'noatime,inode64', '--', '/dev/sdh1', '/var/lib/ceph/tmp/mnt.0bUl5q']' returned non-zero exit status 1
Is there a way to solve / patch that solves this?
Cheers
Goncalo
-- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale School of Physics A28 | University of Sydney, NSW 2006 T: +61 2 93511937
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
-----------------------------------------------------------------------------------------
Adrien GILLARD
+33 (0)6 29 06 16 31
gillard.adrien@xxxxxxxxx
Adrien GILLARD
+33 (0)6 29 06 16 31
gillard.adrien@xxxxxxxxx
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
