Thanks for the fix!
Two questions though:
Is that the right place for the udev rule? I have CentOS 7. The folder exists, but all the other udev rules are in /usr/lib/udev/rules.d/.
Can I just create a new file named "89-ceph-journal.rules" in the /usr/lib/udev/rules.d/ folder?
Regards
David
On 19.11.2015 14:02, Mykola Dvornik wrote:
cat /etc/udev/rules.d/89-ceph-journal.rules
KERNEL=="sdd?" SUBSYSTEM=="block" OWNER="ceph" GROUP="disk" MODE="0660"
On 19 November 2015 at 13:54, Mykola <mykola.dvornik@xxxxxxxxx> wrote:
I am afraid one would need an udev rule to make it persistent.
Sent from Outlook Mail for Windows 10 phone
From: David Riedl
Sent: Thursday, November 19, 2015 1:42 PM
To: ceph-users@xxxxxxxx
Subject: Re: Can't activate osd in infernalis
I fixed the issue and opened a ticket on the ceph-deploy bug tracker
http://tracker.ceph.com/issues/13833
tl;dr:
change permission of the ssd journal partition with
chown ceph:ceph /dev/sdd1
On 19.11.2015 11:38, David Riedl wrote:
> Hi everyone.
> I updated one of my hammer osd nodes to infernalis today.
> After many problems with the upgrading process of the running OSDs, I
> decided to wipe them and start anew.
> I reinstalled all packages and deleted all partitions on the OSDs and
> the SSD journal drive.
> I zapped the disks with ceph-deploy and also prepared them with
> ceph-deploy.
> Selinux state is enabled (disabling it didn't help though).
>
> After executing "ceph-deploy osd activate ceph01:/dev/sda1:/dev/sdd1"
> I get the following error message from ceph-deploy:
>
>
> [ceph01][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph
> --cluster ceph --name client.bootstrap-osd --keyring
> /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o
> /var/lib/ceph/tmp/mnt.pmHRuu/activate.monmap
> [ceph01][WARNIN] 2015-11-19 11:22:53.974765 7f1a06852700 0 --
> :/3225863658 >> 10.20.60.10:6789/0 pipe(0x7f19f8062590 sd=4 :0 s=1
> pgs=0 cs=0 l=1 c=0x7f19f805c1b0).fault
> [ceph01][WARNIN] got monmap epoch 16
> [ceph01][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-osd
> --cluster ceph --mkfs --mkkey -i 0 --monmap
> /var/lib/ceph/tmp/mnt.pmHRuu/activate.monmap --osd-data
> /var/lib/ceph/tmp/mnt.pmHRuu --osd-journal
> /var/lib/ceph/tmp/mnt.pmHRuu/journal --osd-uuid
> de162e24-16b6-4796-b6b9-774fdb8ec234 --keyring
> /var/lib/ceph/tmp/mnt.pmHRuu/keyring --setuser ceph --setgroup ceph
> [ceph01][WARNIN] 2015-11-19 11:22:57.237096 7fb458bb7900 -1
> filestore(/var/lib/ceph/tmp/mnt.pmHRuu) mkjournal error creating
> journal on /var/lib/ceph/tmp/mnt.pmHRuu/journal: (13) Permission denied
> [ceph01][WARNIN] 2015-11-19 11:22:57.237118 7fb458bb7900 -1 OSD::mkfs:
> ObjectStore::mkfs failed with error -13
> [ceph01][WARNIN] 2015-11-19 11:22:57.237157 7fb458bb7900 -1 ** ERROR:
> error creating empty object store in /var/lib/ceph/tmp/mnt.pmHRuu:
> (13) Permission denied
> [ceph01][WARNIN] ERROR:ceph-disk:Failed to activate
> [ceph01][WARNIN] DEBUG:ceph-disk:Unmounting /var/lib/ceph/tmp/mnt.pmHRuu
> [ceph01][WARNIN] INFO:ceph-disk:Running command: /bin/umount --
> /var/lib/ceph/tmp/mnt.pmHRuu
> [ceph01][WARNIN] Traceback (most recent call last):
> [ceph01][WARNIN] File "/usr/sbin/ceph-disk", line 3576, in <module>
> [ceph01][WARNIN] main(sys.argv[1:])
> [ceph01][WARNIN] File "/usr/sbin/ceph-disk", line 3530, in main
> [ceph01][WARNIN] args.func(args)
> [ceph01][WARNIN] File "/usr/sbin/ceph-disk", line 2424, in
> main_activate
> [ceph01][WARNIN] dmcrypt_key_dir=args.dmcrypt_key_dir,
> [ceph01][WARNIN] File "/usr/sbin/ceph-disk", line 2197, in
> mount_activate
> [ceph01][WARNIN] (osd_id, cluster) = activate(path,
> activate_key_template, init)
> [ceph01][WARNIN] File "/usr/sbin/ceph-disk", line 2360, in activate
> [ceph01][WARNIN] keyring=keyring,
> [ceph01][WARNIN] File "/usr/sbin/ceph-disk", line 1950, in mkfs
> [ceph01][WARNIN] '--setgroup', get_ceph_user(),
> [ceph01][WARNIN] File "/usr/sbin/ceph-disk", line 349, in
> command_check_call
> [ceph01][WARNIN] return subprocess.check_call(arguments)
> [ceph01][WARNIN] File "/usr/lib64/python2.7/subprocess.py", line
> 542, in check_call
> [ceph01][WARNIN] raise CalledProcessError(retcode, cmd)
> [ceph01][WARNIN] subprocess.CalledProcessError: Command
> '['/usr/bin/ceph-osd', '--cluster', 'ceph', '--mkfs', '--mkkey', '-i',
> '0', '--monmap', '/var/lib/ceph/tmp/mnt.pmHRuu/activate.monmap',
> '--osd-data', '/var/lib/ceph/tmp/mnt.pmHRuu', '--osd-journal',
> '/var/lib/ceph/tmp/mnt.pmHRuu/journal', '--osd-uuid',
> 'de162e24-16b6-4796-b6b9-774fdb8ec234', '--keyring',
> '/var/lib/ceph/tmp/mnt.pmHRuu/keyring', '--setuser', 'ceph',
> '--setgroup', 'ceph']' returned non-zero exit status 1
> [ceph01][ERROR ] RuntimeError: command returned non-zero exit status: 1
> [ceph_deploy][ERROR ] RuntimeError: Failed to execute command:
> ceph-disk -v activate --mark-init systemd --mount /dev/sda1
>
> The output of ls -lahn in /var/lib/ceph/ is
>
> drwxr-x---. 9 167 167 4,0K 19. Nov 10:32 .
> drwxr-xr-x. 28 0 0 4,0K 19. Nov 11:14 ..
> drwxr-x---. 2 167 167 6 10. Nov 13:06 bootstrap-mds
> drwxr-x---. 2 167 167 25 19. Nov 10:48 bootstrap-osd
> drwxr-x---. 2 167 167 6 10. Nov 13:06 bootstrap-rgw
> drwxr-x---. 2 167 167 6 10. Nov 13:06 mds
> drwxr-x---. 2 167 167 6 10. Nov 13:06 mon
> drwxr-x---. 2 167 167 6 10. Nov 13:06 osd
> drwxr-x---. 2 167 167 65 19. Nov 11:22 tmp
>
>
> I hope someone can help me, I am really lost right now.
>
--
Mit freundlichen Grüßen
David Riedl
WINGcon GmbH Wireless New Generation - Consulting & Solutions
Phone: +49 (0) 7543 9661 - 26
E-Mail: david.riedl@xxxxxxxxxxx
Sitz der Gesellschaft: Langenargen
Registergericht: ULM, HRB 632019
USt-Id.: DE232931635, WEEE-Id.: DE74015979
Geschäftsführer: Norbert Schäfer, Fritz R. Paul
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
Mykola
-- Mit freundlichen Grüßen David Riedl WINGcon GmbH Wireless New Generation - Consulting & Solutions Phone: +49 (0) 7543 9661 - 26 E-Mail: david.riedl@xxxxxxxxxxx Web: http://www.wingcon.com Sitz der Gesellschaft: Langenargen Registergericht: ULM, HRB 632019 USt-Id.: DE232931635, WEEE-Id.: DE74015979 Geschäftsführer: Norbert Schäfer, Fritz R. Paul
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
