Dear Cephers...
1./ I am currently deploying infernalis 9.2.0 in centos7.
2./ I am not using ceph-deploy (I prefer not to use it) and I prefer
to use the short form osd installation procedure described here:
http://docs.ceph.com/docs/infernalis/install/manual-deployment/#short-form
3./ In hammer this instructions worked fine but in infernalis 9.2.0,
which the implementation of the ceph user running the ceph daemons,
those instructions do not seem to work properly anymore.
4./ Here is the behaviour:
4.1/ ceph-disk prepare runs fine:
# ceph-disk prepare --cluster ceph --cluster-uuid
a9431bc6-3ee1-4b0a-8d21-0ad883a4d2ed --fs-type xfs /dev/sdh
/dev/sdc3
WARNING:ceph-disk:OSD will not be hot-swappable if journal is not
the same device as the osd data
WARNING:ceph-disk:Journal /dev/sdc3 was not prepared with
ceph-disk. Symlinking directly.
Creating new GPT entries.
The operation has completed successfully.
meta-data="" isize=2048 agcount=4,
agsize=183107519 blks
= sectsz=512 attr=2,
projid32bit=1
= crc=0 finobt=0
data = bsize=4096 blocks=732430075,
imaxpct=5
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=0
log =internal log bsize=4096 blocks=357631,
version=2
= sectsz=512 sunit=0 blks,
lazy-count=1
realtime =none extsz=4096 blocks=0,
rtextents=0
The operation has completed successfully.
4.2/ ceph-disk activate fails with permission problems. It seems
some temporary directory us made by root which is then not
accessible to ceph user.
# ceph-disk activate /dev/sdh1
got monmap epoch 1
2015-12-03 01:56:41.406742 7fb050fa9900 -1
filestore(/var/lib/ceph/tmp/mnt.dsi7zI) mkjournal error creating
journal on /var/lib/ceph/tmp/mnt.dsi7zI/journal: (13) Permission
denied
2015-12-03 01:56:41.406777 7fb050fa9900 -1 OSD::mkfs:
ObjectStore::mkfs failed with error -13
2015-12-03 01:56:41.406843 7fb050fa9900 -1 ** ERROR: error
creating empty object store in /var/lib/ceph/tmp/mnt.dsi7zI: (13)
Permission denied
ERROR:ceph-disk:Failed to activate
Traceback (most recent call last):
File "/usr/sbin/ceph-disk", line 3576, in <module>
main(sys.argv[1:])
File "/usr/sbin/ceph-disk", line 3532, in main
main_catch(args.func, args)
File "/usr/sbin/ceph-disk", line 3554, in main_catch
func(args)
File "/usr/sbin/ceph-disk", line 2424, in main_activate
dmcrypt_key_dir=args.dmcrypt_key_dir,
File "/usr/sbin/ceph-disk", line 2197, in mount_activate
(osd_id, cluster) = activate(path, activate_key_template,
init)
File "/usr/sbin/ceph-disk", line 2360, in activate
keyring=keyring,
File "/usr/sbin/ceph-disk", line 1950, in mkfs
'--setgroup', get_ceph_user(),
File "/usr/sbin/ceph-disk", line 349, in command_check_call
return subprocess.check_call(arguments)
File "/usr/lib64/python2.7/subprocess.py", line 542, in
check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/ceph-osd',
'--cluster', 'ceph', '--mkfs', '--mkkey', '-i', '4', '--monmap',
'/var/lib/ceph/tmp/mnt.dsi7zI/activate.monmap', '--osd-data',
'/var/lib/ceph/tmp/mnt.dsi7zI', '--osd-journal',
'/var/lib/ceph/tmp/mnt.dsi7zI/journal', '--osd-uuid',
'995e436f-dc6f-42ca-bbbf-3ef9f0bb4952', '--keyring',
'/var/lib/ceph/tmp/mnt.dsi7zI/keyring', '--setuser', 'ceph',
'--setgroup', 'ceph']' returned non-zero exit status 1
4.3 If I run the the ceph activate command as a ceph user, it
complains it need root permissions to mount
# su ceph -s /bin/bash --session-command="ceph-disk activate
/dev/sdh1"
Problem opening /dev/sdh for reading! Error is 13.
You must run this program as root or use sudo!
mount: only root can use "--options" option
ceph-disk: Mounting filesystem failed: Command '['/usr/bin/mount',
'-t', 'xfs', '-o', 'noatime,inode64', '--', '/dev/sdh1',
'/var/lib/ceph/tmp/mnt.0bUl5q']' returned non-zero exit status 1
5./ So there is an incoherence on this set of instructions.
Is there a way to solve / patch that solves this?
Cheers
Goncalo
--
Goncalo Borges
Research Computing
ARC Centre of Excellence for Particle Physics at the Terascale
School of Physics A28 | University of Sydney, NSW 2006
T: +61 2 93511937
|
