I fixed the issue and opened a ticket on the ceph-deploy bug tracker
http://tracker.ceph.com/issues/13833
tl;dr:
change permission of the ssd journal partition with
chown ceph:ceph /dev/sdd1
On 19.11.2015 11:38, David Riedl wrote:
Hi everyone.
I updated one of my hammer osd nodes to infernalis today.
After many problems with the upgrading process of the running OSDs, I
decided to wipe them and start anew.
I reinstalled all packages and deleted all partitions on the OSDs and
the SSD journal drive.
I zapped the disks with ceph-deploy and also prepared them with
ceph-deploy.
Selinux state is enabled (disabling it didn't help though).
After executing "ceph-deploy osd activate ceph01:/dev/sda1:/dev/sdd1"
I get the following error message from ceph-deploy:
[ceph01][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph
--cluster ceph --name client.bootstrap-osd --keyring
/var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o
/var/lib/ceph/tmp/mnt.pmHRuu/activate.monmap
[ceph01][WARNIN] 2015-11-19 11:22:53.974765 7f1a06852700 0 --
:/3225863658 >> 10.20.60.10:6789/0 pipe(0x7f19f8062590 sd=4 :0 s=1
pgs=0 cs=0 l=1 c=0x7f19f805c1b0).fault
[ceph01][WARNIN] got monmap epoch 16
[ceph01][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-osd
--cluster ceph --mkfs --mkkey -i 0 --monmap
/var/lib/ceph/tmp/mnt.pmHRuu/activate.monmap --osd-data
/var/lib/ceph/tmp/mnt.pmHRuu --osd-journal
/var/lib/ceph/tmp/mnt.pmHRuu/journal --osd-uuid
de162e24-16b6-4796-b6b9-774fdb8ec234 --keyring
/var/lib/ceph/tmp/mnt.pmHRuu/keyring --setuser ceph --setgroup ceph
[ceph01][WARNIN] 2015-11-19 11:22:57.237096 7fb458bb7900 -1
filestore(/var/lib/ceph/tmp/mnt.pmHRuu) mkjournal error creating
journal on /var/lib/ceph/tmp/mnt.pmHRuu/journal: (13) Permission denied
[ceph01][WARNIN] 2015-11-19 11:22:57.237118 7fb458bb7900 -1 OSD::mkfs:
ObjectStore::mkfs failed with error -13
[ceph01][WARNIN] 2015-11-19 11:22:57.237157 7fb458bb7900 -1 ** ERROR:
error creating empty object store in /var/lib/ceph/tmp/mnt.pmHRuu:
(13) Permission denied
[ceph01][WARNIN] ERROR:ceph-disk:Failed to activate
[ceph01][WARNIN] DEBUG:ceph-disk:Unmounting /var/lib/ceph/tmp/mnt.pmHRuu
[ceph01][WARNIN] INFO:ceph-disk:Running command: /bin/umount --
/var/lib/ceph/tmp/mnt.pmHRuu
[ceph01][WARNIN] Traceback (most recent call last):
[ceph01][WARNIN] File "/usr/sbin/ceph-disk", line 3576, in <module>
[ceph01][WARNIN] main(sys.argv[1:])
[ceph01][WARNIN] File "/usr/sbin/ceph-disk", line 3530, in main
[ceph01][WARNIN] args.func(args)
[ceph01][WARNIN] File "/usr/sbin/ceph-disk", line 2424, in
main_activate
[ceph01][WARNIN] dmcrypt_key_dir=args.dmcrypt_key_dir,
[ceph01][WARNIN] File "/usr/sbin/ceph-disk", line 2197, in
mount_activate
[ceph01][WARNIN] (osd_id, cluster) = activate(path,
activate_key_template, init)
[ceph01][WARNIN] File "/usr/sbin/ceph-disk", line 2360, in activate
[ceph01][WARNIN] keyring=keyring,
[ceph01][WARNIN] File "/usr/sbin/ceph-disk", line 1950, in mkfs
[ceph01][WARNIN] '--setgroup', get_ceph_user(),
[ceph01][WARNIN] File "/usr/sbin/ceph-disk", line 349, in
command_check_call
[ceph01][WARNIN] return subprocess.check_call(arguments)
[ceph01][WARNIN] File "/usr/lib64/python2.7/subprocess.py", line
542, in check_call
[ceph01][WARNIN] raise CalledProcessError(retcode, cmd)
[ceph01][WARNIN] subprocess.CalledProcessError: Command
'['/usr/bin/ceph-osd', '--cluster', 'ceph', '--mkfs', '--mkkey', '-i',
'0', '--monmap', '/var/lib/ceph/tmp/mnt.pmHRuu/activate.monmap',
'--osd-data', '/var/lib/ceph/tmp/mnt.pmHRuu', '--osd-journal',
'/var/lib/ceph/tmp/mnt.pmHRuu/journal', '--osd-uuid',
'de162e24-16b6-4796-b6b9-774fdb8ec234', '--keyring',
'/var/lib/ceph/tmp/mnt.pmHRuu/keyring', '--setuser', 'ceph',
'--setgroup', 'ceph']' returned non-zero exit status 1
[ceph01][ERROR ] RuntimeError: command returned non-zero exit status: 1
[ceph_deploy][ERROR ] RuntimeError: Failed to execute command:
ceph-disk -v activate --mark-init systemd --mount /dev/sda1
The output of ls -lahn in /var/lib/ceph/ is
drwxr-x---. 9 167 167 4,0K 19. Nov 10:32 .
drwxr-xr-x. 28 0 0 4,0K 19. Nov 11:14 ..
drwxr-x---. 2 167 167 6 10. Nov 13:06 bootstrap-mds
drwxr-x---. 2 167 167 25 19. Nov 10:48 bootstrap-osd
drwxr-x---. 2 167 167 6 10. Nov 13:06 bootstrap-rgw
drwxr-x---. 2 167 167 6 10. Nov 13:06 mds
drwxr-x---. 2 167 167 6 10. Nov 13:06 mon
drwxr-x---. 2 167 167 6 10. Nov 13:06 osd
drwxr-x---. 2 167 167 65 19. Nov 11:22 tmp
I hope someone can help me, I am really lost right now.
--
Mit freundlichen Grüßen
David Riedl
WINGcon GmbH Wireless New Generation - Consulting & Solutions
Phone: +49 (0) 7543 9661 - 26
E-Mail: david.riedl@xxxxxxxxxxx
Web: http://www.wingcon.com
Sitz der Gesellschaft: Langenargen
Registergericht: ULM, HRB 632019
USt-Id.: DE232931635, WEEE-Id.: DE74015979
Geschäftsführer: Norbert Schäfer, Fritz R. Paul
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
