Re: mkjournal error creating journal ... : (13) Permission denied

Gunwoo Gim <wind8702@xxxxxxxxx> · Tue, 14 Mar 2017 07:53:29 +0000

I'd love to get helped out; it'd be much appreciated.
Best Wishes,
Nicholas.

On Tue, Mar 14, 2017 at 4:51 PM Gunwoo Gim <wind8702@xxxxxxxxx> wrote:
 Hello, I'm trying to deploy a ceph filestore cluster with LVM using ceph-ansible playbook. I've been fixing a couple of code blocks in ceph-ansible and ceph-disk/main.py and made some progress but now I'm stuck again; 'ceph-disk activate osd' fails.
 Please let me just show you the error message and the output of 'ls':

~ # ceph-disk -v activate /dev/mapper/vg--hdd1-lv--hdd1p1
main_activate: path = /dev/mapper/vg--hdd1-lv--hdd1p1
get_dm_uuid: get_dm_uuid /dev/mapper/vg--hdd1-lv--hdd1p1 uuid path is /sys/dev/block/252:12/dm/uuid
get_dm_uuid: get_dm_uuid /dev/mapper/vg--hdd1-lv--hdd1p1 uuid is part1-LVM-ETn7wXOmnesc9MNpleTYzP29jjOkp19J12ELrQez43LFPfdFc1dItn8EFF299401

command: Running command: /sbin/blkid -p -s TYPE -o value -- /dev/mapper/vg--hdd1-lv--hdd1p1
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs
mount: Mounting /dev/mapper/vg--hdd1-lv--hdd1p1 on /var/lib/ceph/tmp/mnt.cJDc7I with options noatime,largeio,inode64,swalloc
command_check_call: Running command: /bin/mount -t xfs -o noatime,largeio,inode64,swalloc -- /dev/mapper/vg--hdd1-lv--hdd1p1 /var/lib/ceph/tmp/mnt.cJDc7I
activate: Cluster uuid is 0bc0ea6d-ed8a-4ef0-9e82-ba6454a7214e
command: Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid
activate: Cluster name is ceph
activate: OSD uuid is 5097be3f-349e-480d-8b0d-d68c13ae2f72
activate: OSD id is 1
activate: Initializing OSD...
command_check_call: Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/tmp/mnt.cJDc7I/activate.monmap
got monmap epoch 2
command: Running command: /usr/bin/timeout 300 ceph-osd --cluster ceph --mkfs --mkkey -i 1 --monmap /var/lib/ceph/tmp/mnt.cJDc7I/activate.monmap --osd-data /var/lib/ceph/tmp/mnt.cJDc7I --osd-journal /var/lib/ceph/tmp/mnt.cJDc7I/journal --osd-uuid 5097be3f-349e-480d-8b0d-d68c13ae2f72 --keyring /var/lib/ceph/tmp/mnt.cJDc7I/keyring --setuser ceph --setgroup ceph
mount_activate: Failed to activate
unmount: Unmounting /var/lib/ceph/tmp/mnt.cJDc7I
command_check_call: Running command: /bin/umount -- /var/lib/ceph/tmp/mnt.cJDc7I
Traceback (most recent call last):
  File "/usr/sbin/ceph-disk", line 9, in <module>
    load_entry_point('ceph-disk==1.0.0', 'console_scripts', 'ceph-disk')()
  File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 5251, in run
    main(sys.argv[1:])
  File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 5202, in main
    args.func(args)
  File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 3553, in main_activate
    reactivate=args.reactivate,
  File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 3310, in mount_activate
    (osd_id, cluster) = activate(path, activate_key_template, init)
  File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 3486, in activate
    keyring=keyring,
  File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 2948, in mkfs
    '--setgroup', get_ceph_group(),
  File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 2895, in ceph_osd_mkfs
    raise Error('%s failed : %s' % (str(arguments), error))
ceph_disk.main.Error: Error: ['ceph-osd', '--cluster', 'ceph', '--mkfs', '--mkkey', '-i', u'1', '--monmap', '/var/lib/ceph/tmp/mnt.cJDc7I/activate.monmap', '--osd-data', '/var/lib/ceph/tmp/mnt.cJDc7I', '--osd-journal', '/var/lib/ceph/tmp/mnt.cJDc7I/journal', '--osd-uuid', u'5097be3f-349e-480d-8b0d-d68c13ae2f72', '--keyring', '/var/lib/ceph/tmp/mnt.cJDc7I/keyring', '--setuser', 'ceph', '--setgroup', 'ceph'] failed : 2017-03-14 16:01:10.051537 7fdc9a025a40 -1 filestore(/var/lib/ceph/tmp/mnt.cJDc7I) mkjournal error creating journal on /var/lib/ceph/tmp/mnt.cJDc7I/journal: (13) Permission denied
2017-03-14 16:01:10.051565 7fdc9a025a40 -1 OSD::mkfs: ObjectStore::mkfs failed with error -13
2017-03-14 16:01:10.051624 7fdc9a025a40 -1  ** ERROR: error creating empty object store in /var/lib/ceph/tmp/mnt.cJDc7I: (13) Permission denied

~ # ls -al /var/lib/ceph/tmp
total 8
drwxr-xr-x  2 ceph ceph 4096 Mar 14 16:01 .
drwxr-xr-x 11 ceph ceph 4096 Mar 14 11:12 ..
-rwxr-xr-x  1 root root    0 Mar 14 11:12 ceph-disk.activate.lock
-rwxr-xr-x  1 root root    0 Mar 14 11:44 ceph-disk.prepare.lock

~ # ls -l /dev/mapper/vg-*-lv-*p*
lrwxrwxrwx 1 root root 8 Mar 14 13:46 /dev/mapper/vg--hdd1-lv--hdd1p1 -> ../dm-12
lrwxrwxrwx 1 root root 8 Mar 14 13:46 /dev/mapper/vg--hdd2-lv--hdd2p1 -> ../dm-14
lrwxrwxrwx 1 root root 8 Mar 14 13:46 /dev/mapper/vg--hdd3-lv--hdd3p1 -> ../dm-16
lrwxrwxrwx 1 root root 8 Mar 14 13:46 /dev/mapper/vg--hdd4-lv--hdd4p1 -> ../dm-18
lrwxrwxrwx 1 root root 8 Mar 14 13:46 /dev/mapper/vg--hdd5-lv--hdd5p1 -> ../dm-20
lrwxrwxrwx 1 root root 8 Mar 14 13:46 /dev/mapper/vg--hdd6-lv--hdd6p1 -> ../dm-22
lrwxrwxrwx 1 root root 8 Mar 14 13:46 /dev/mapper/vg--hdd7-lv--hdd7p1 -> ../dm-24
lrwxrwxrwx 1 root root 8 Mar 14 13:46 /dev/mapper/vg--hdd8-lv--hdd8p1 -> ../dm-26
lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--hdd9-lv--hdd9p1 -> ../dm-28
lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd1-lv--ssd1p1 -> ../dm-11
lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd1-lv--ssd1p2 -> ../dm-15
lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd1-lv--ssd1p3 -> ../dm-19
lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd1-lv--ssd1p4 -> ../dm-23
lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd1-lv--ssd1p5 -> ../dm-27
lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd2-lv--ssd2p1 -> ../dm-13
lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd2-lv--ssd2p2 -> ../dm-17
lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd2-lv--ssd2p3 -> ../dm-21
lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd2-lv--ssd2p4 -> ../dm-25

~ # ls -l /dev/dm-*
brw-rw---- 1 root disk 252,  0 Mar 14 13:46 /dev/dm-0
brw-rw---- 1 root disk 252,  1 Mar 14 13:46 /dev/dm-1
brw-rw---- 1 root disk 252, 10 Mar 14 13:47 /dev/dm-10
brw-rw---- 1 ceph ceph 252, 11 Mar 14 13:47 /dev/dm-11
brw-rw---- 1 ceph ceph 252, 12 Mar 14 13:46 /dev/dm-12
brw-rw---- 1 ceph ceph 252, 13 Mar 14 13:47 /dev/dm-13
brw-rw---- 1 ceph ceph 252, 14 Mar 14 13:46 /dev/dm-14
brw-rw---- 1 ceph ceph 252, 15 Mar 14 13:47 /dev/dm-15
brw-rw---- 1 ceph ceph 252, 16 Mar 14 13:46 /dev/dm-16
brw-rw---- 1 ceph ceph 252, 17 Mar 14 13:47 /dev/dm-17
brw-rw---- 1 ceph ceph 252, 18 Mar 14 13:46 /dev/dm-18
brw-rw---- 1 ceph ceph 252, 19 Mar 14 13:47 /dev/dm-19
brw-rw---- 1 root disk 252,  2 Mar 14 13:46 /dev/dm-2
brw-rw---- 1 ceph ceph 252, 20 Mar 14 13:46 /dev/dm-20
brw-rw---- 1 ceph ceph 252, 21 Mar 14 13:47 /dev/dm-21
brw-rw---- 1 ceph ceph 252, 22 Mar 14 13:46 /dev/dm-22
brw-rw---- 1 ceph ceph 252, 23 Mar 14 13:47 /dev/dm-23
brw-rw---- 1 ceph ceph 252, 24 Mar 14 13:46 /dev/dm-24
brw-rw---- 1 ceph ceph 252, 25 Mar 14 13:47 /dev/dm-25
brw-rw---- 1 ceph ceph 252, 26 Mar 14 13:46 /dev/dm-26
brw-rw---- 1 ceph ceph 252, 27 Mar 14 13:47 /dev/dm-27
brw-rw---- 1 ceph ceph 252, 28 Mar 14 13:47 /dev/dm-28
brw-rw---- 1 root disk 252,  3 Mar 14 13:46 /dev/dm-3
brw-rw---- 1 root disk 252,  4 Mar 14 13:46 /dev/dm-4
brw-rw---- 1 root disk 252,  5 Mar 14 13:46 /dev/dm-5
brw-rw---- 1 root disk 252,  6 Mar 14 13:47 /dev/dm-6
brw-rw---- 1 root disk 252,  7 Mar 14 13:46 /dev/dm-7
brw-rw---- 1 root disk 252,  8 Mar 14 13:46 /dev/dm-8
brw-rw---- 1 root disk 252,  9 Mar 14 13:47 /dev/dm-9

 Best Regards,
 Nicholas.
-- 
You can find my PGP public key here: https://google.com/+DewrKim/about
-- 
You can find my PGP public key here: https://google.com/+DewrKim/about
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com