Re: rbd mapping failes

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 

mount cephfs failes to (I added 3 MDS)
anybody any ideas how to debug this further?

I used ceph-deploy to create the cluster,
the xfs filesystem on the OSD's is okay, I can copy remove and open files on that partition
so I asume it's something inside of ceph???

TIA

Bernhard

P.S.: Version is
ceph version 0.67.2-16-gd41cf86 (d41cf866ee028ef7b821a5c37b991e85cbf3637f) on uptodate raring

----------
root@nuke36[/1]:/etc/ceph # ceph -s
2013-08-30 15:03:18.454701 7f3b7cd18700  1 -- :/0 messenger.start
2013-08-30 15:03:18.455460 7f3b7cd18700  1 -- :/1003684 --> 192.168.242.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- ?+0 0x7f3b7800e8f0 con 0x7f3b7800e4e0
2013-08-30 15:03:18.456412 7f3b7c517700  1 -- 192.168.242.36:0/1003684 learned my addr 192.168.242.36:0/1003684
2013-08-30 15:03:18.458069 7f3b76ffd700  1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 1 ==== mon_map v1 ==== 776+0+0 (3609201999 0 0) 0x7f3b6c000c30 con 0x7f3b7800e4e0
2013-08-30 15:03:18.458308 7f3b76ffd700  1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 2 ==== auth_reply(proto 2 0 Success) v1 ==== 33+0+0 (345113272 0 0) 0x7f3b6c0008f0 con 0x7f3b7800e4e0
2013-08-30 15:03:18.458612 7f3b76ffd700  1 -- 192.168.242.36:0/1003684 --> 192.168.242.92:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- ?+0 0x7f3b60001af0 con 0x7f3b7800e4e0
2013-08-30 15:03:18.459532 7f3b76ffd700  1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 3 ==== auth_reply(proto 2 0 Success) v1 ==== 206+0+0 (1084599267 0 0) 0x7f3b6c0008f0 con 0x7f3b7800e4e0
2013-08-30 15:03:18.459816 7f3b76ffd700  1 -- 192.168.242.36:0/1003684 --> 192.168.242.92:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- ?+0 0x7f3b600020d0 con 0x7f3b7800e4e0
2013-08-30 15:03:18.460739 7f3b76ffd700  1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 4 ==== auth_reply(proto 2 0 Success) v1 ==== 393+0+0 (496062897 0 0) 0x7f3b6c0008f0 con 0x7f3b7800e4e0
2013-08-30 15:03:18.460844 7f3b76ffd700  1 -- 192.168.242.36:0/1003684 --> 192.168.242.92:6789/0 -- mon_subscribe({monmap=0+}) v2 -- ?+0 0x7f3b7800ed80 con 0x7f3b7800e4e0
2013-08-30 15:03:18.461118 7f3b7cd18700  1 -- 192.168.242.36:0/1003684 --> 192.168.242.92:6789/0 -- mon_subscribe({monmap=2+,osdmap=0}) v2 -- ?+0 0x7f3b780079f0 con 0x7f3b7800e4e0
2013-08-30 15:03:18.461138 7f3b7cd18700  1 -- 192.168.242.36:0/1003684 --> 192.168.242.92:6789/0 -- mon_subscribe({monmap=2+,osdmap=0}) v2 -- ?+0 0x7f3b7800fa10 con 0x7f3b7800e4e0
2013-08-30 15:03:18.461813 7f3b76ffd700  1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 5 ==== mon_map v1 ==== 776+0+0 (3609201999 0 0) 0x7f3b6c0008f0 con 0x7f3b7800e4e0
2013-08-30 15:03:18.462016 7f3b76ffd700  1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 6 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3156621930 0 0) 0x7f3b6c001340 con 0x7f3b7800e4e0
2013-08-30 15:03:18.463931 7f3b7cd18700  1 -- 192.168.242.36:0/1003684 --> 192.168.242.92:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- ?+0 0x7f3b7800b0f0 con 0x7f3b7800e4e0
2013-08-30 15:03:18.463966 7f3b76ffd700  1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 7 ==== osd_map(34..34 src has 1..34) v3 ==== 2483+0+0 (453205619 0 0) 0x7f3b6c0008c0 con 0x7f3b7800e4e0
2013-08-30 15:03:18.464694 7f3b76ffd700  1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 8 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3156621930 0 0) 0x7f3b6c0010e0 con 0x7f3b7800e4e0
2013-08-30 15:03:18.464749 7f3b76ffd700  1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 9 ==== osd_map(34..34 src has 1..34) v3 ==== 2483+0+0 (453205619 0 0) 0x7f3b6c002720 con 0x7f3b7800e4e0
2013-08-30 15:03:18.464765 7f3b76ffd700  1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 10 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3156621930 0 0) 0x7f3b6c002b20 con 0x7f3b7800e4e0
2013-08-30 15:03:18.468276 7f3b76ffd700  1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 11 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0  v0) v1 ==== 72+0+24040 (1092875540 0 2922658865) 0x7f3b6c002720 con 0x7f3b7800e4e0
2013-08-30 15:03:18.510756 7f3b7cd18700  1 -- 192.168.242.36:0/1003684 --> 192.168.242.92:6789/0 -- mon_command({"prefix": "status"} v 0) v1 -- ?+0 0x7f3b7800b0d0 con 0x7f3b7800e4e0
2013-08-30 15:03:18.512490 7f3b76ffd700  1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 12 ==== mon_command_ack([{"prefix": "status"}]=0  v0) v1 ==== 54+0+497 (1155462804 0 3461792647) 0x7f3b6c001080 con 0x7f3b7800e4e0
  cluster f57cdca3-7222-4095-853b-03727461f725
   health HEALTH_OK
   monmap e1: 5 mons at {atom01=192.168.242.31:6789/0,atom02=192.168.242.32:6789/0,nuke36=192.168.242.36:6789/0,ping=192.168.242.92:6789/0,pong=192.168.242.93:6789/0}, election epoch 42, quorum 0,1,2,3,4 atom01,atom02,nuke36,ping,pong
   osdmap e34: 2 osds: 2 up, 2 in
    pgmap v367: 1192 pgs: 1192 active+clean; 9788 bytes data, 94460 KB used, 3722 GB / 3722 GB avail
   mdsmap e17: 1/1/1 up {0=atom02=up:active}, 2 up:standby

2013-08-30 15:03:18.516793 7f3b7cd18700  1 -- 192.168.242.36:0/1003684 mark_down 0x7f3b7800e4e0 -- 0x7f3b7800e280
2013-08-30 15:03:18.517204 7f3b7cd18700  1 -- 192.168.242.36:0/1003684 mark_down_all
2013-08-30 15:03:18.517921 7f3b7cd18700  1 -- 192.168.242.36:0/1003684 shutdown complete.
root@nuke36[/1]:/etc/ceph # mount -t ceph 192.168.242.36:6789:/ /mnt/ceph/
mount error 5 = Input/output error
----------



Am 30.08.2013 11:48:35, schrieb Bernhard Glomm:
Hi all,

due to a problem with ceph-deploy I currently use

deb http://gitbuilder.ceph.com/ceph-deb-raring-x86_64-basic/ref/wip-4924/ raring main
(ceph version 0.67.2-16-gd41cf86 (d41cf866ee028ef7b821a5c37b991e85cbf3637f))

Now the initialization of the cluster works like a charm,
ceph health is okay,
just the mapping of the created rbd is failing.

---------------------
root@ping[/1]:~ # ceph osd pool delete kvm-pool kvm-pool --yes-i-really-really-mean-it
pool 'kvm-pool' deleted
root@ping[/1]:~ # ceph osd lspools

0 data,1 metadata,2 rbd,
root@ping[/1]:~ #
root@ping[/1]:~ # ceph osd pool create kvm-pool 1000
pool 'kvm-pool' created
root@ping[/1]:~ # ceph osd lspools
0 data,1 metadata,2 rbd,4 kvm-pool,
root@ping[/1]:~ # ceph osd pool set kvm-pool min_size 2
set pool 4 min_size to 2
root@ping[/1]:~ # ceph osd dump | grep 'rep size'
pool 0 'data' rep size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 64 pgp_num 64 last_change 1 owner 0 crash_replay_interval 45
pool 1 'metadata' rep size 2 min_size 1 crush_ruleset 1 object_hash rjenkins pg_num 64 pgp_num 64 last_change 1 owner 0
pool 2 'rbd' rep size 2 min_size 1 crush_ruleset 2 object_hash rjenkins pg_num 64 pgp_num 64 last_change 1 owner 0
pool 4 'kvm-pool' rep size 2 min_size 2 crush_ruleset 0 object_hash rjenkins pg_num 1000 pgp_num 1000 last_change 33 owner 0
root@ping[/1]:~ # rbd create atom03.cimg --size 4000 --pool kvm-pool
root@ping[/1]:~ # rbd create atom04.cimg --size 4000 --pool kvm-pool
root@ping[/1]:~ # rbd ls kvm-pool
atom03.cimg
atom04.cimg
root@ping[/1]:~ # rbd --image atom03.cimg --pool kvm-pool info
rbd image 'atom03.cimg':
        size 4000 MB in 1000 objects
        order 22 (4096 KB objects)
        block_name_prefix: rb.0.114d.2ae8944a
        format: 1
root@ping[/1]:~ # rbd --image atom04.cimg --pool kvm-pool info
rbd image 'atom04.cimg':
        size 4000 MB in 1000 objects
        order 22 (4096 KB objects)
        block_name_prefix: rb.0.127d.74b0dc51
        format: 1
root@ping[/1]:~ # rbd map atom03.cimg --pool kvm-pool --id admin
rbd: '/sbin/udevadm settle' failed! (256)
root@ping[/1]:~ # rbd map --pool kvm-pool --image atom03.cimg --id admin --keyring /etc/ceph/ceph.client.admin.keyring
^Crbd: '/sbin/udevadm settle' failed! (2)
root@ping[/1]:~ # rbd map kvm-pool/atom03.cimg --id admin --keyring /etc/ceph/ceph.client.admin.keyring
rbd: '/sbin/udevadm settle' failed! (256)
---------------------

Do I miss something?
But I think this set of commands worked perfectly with cuttlefish?

TIA

Bernhard

--

Bernhard Glomm
IT Administration
Phone: +49 (30) 86880 134 Fax: +49 (30) 86880 100 Skype: bernhard.glomm.ecologic Ecologic Institut gemeinnützige GmbH | Pfalzburger Str. 43/44 | 10717 Berlin | Germany
GF: R. Andreas Kraemer | AG: Charlottenburg HRB 57947 | USt/VAT-IdNr.: DE811963464
Ecologic™ is a Trade Mark (TM) of Ecologic Institut gemeinnützige GmbH
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



--
*Ecologic Institute* Bernhard Glomm
IT Administration
Phone: +49 (30) 86880 134
Fax: +49 (30) 86880 100
Skype: bernhard.glomm.ecologic
Website: | Video: | Newsletter: | Facebook: | Linkedin: | Twitter: | YouTube: | Google+:
Ecologic Institut gemeinnützige GmbH | Pfalzburger Str. 43/44 | 10717 Berlin | Germany
GF: R. Andreas Kraemer | AG: Charlottenburg HRB 57947 | USt/VAT-IdNr.: DE811963464
Ecologic™ is a Trade Mark (TM) of Ecologic Institut gemeinnützige GmbH 
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]

  Powered by Linux