oot at u12ceph02:~# rbd map foo --pool rbd --name client.admin -m u12ceph01 -k /etc/ceph/ceph.client.admin.keyring rbd: add failed: (5) Input/output error dmesg shows these right away after the IO error: [ 461.010895] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 461.010917] libceph: mon0 10.190.10.13:6789 socket error on read [ 471.041574] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 471.041597] libceph: mon0 10.190.10.13:6789 socket error on read [ 481.057452] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 481.057471] libceph: mon0 10.190.10.13:6789 socket error on read [ 491.073662] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 491.073681] libceph: mon0 10.190.10.13:6789 socket error on read [ 501.089744] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 501.089768] libceph: mon0 10.190.10.13:6789 socket error on read [ 511.105703] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 511.105723] libceph: mon0 10.190.10.13:6789 socket error on read [ 677.686613] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 677.686631] libceph: mon0 10.190.10.13:6789 socket error on read [ 687.713358] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 687.713381] libceph: mon0 10.190.10.13:6789 socket error on read [ 697.729262] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 697.729283] libceph: mon0 10.190.10.13:6789 socket error on read [ 707.745522] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 707.745539] libceph: mon0 10.190.10.13:6789 socket error on read [ 717.761325] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 717.761361] libceph: mon0 10.190.10.13:6789 socket error on read [ 727.777513] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 727.777547] libceph: mon0 10.190.10.13:6789 socket error on read [ 837.056165] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 837.056217] libceph: mon0 10.190.10.13:6789 socket error on read [ 847.073477] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 847.073513] libceph: mon0 10.190.10.13:6789 socket error on read [ 857.089465] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 857.089501] libceph: mon0 10.190.10.13:6789 socket error on read [ 867.105422] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 867.105444] libceph: mon0 10.190.10.13:6789 socket error on read [ 877.121579] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 877.121609] libceph: mon0 10.190.10.13:6789 socket error on read [ 887.137564] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042aca < server's 2104a042aca, missing 21000000000 [ 887.137586] libceph: mon0 10.190.10.13:6789 socket error on read [ 1047.221414] libceph: mon1 10.190.10.16:6789 feature set mismatch, my 4a042a42 < server's 2104a042a42, missing 21000000000 [ 1047.221445] libceph: mon1 10.190.10.16:6789 socket error on read [ 1057.249725] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042a42 < server's 2104a042a42, missing 21000000000 [ 1057.249747] libceph: mon0 10.190.10.13:6789 socket error on read [ 1067.266076] libceph: mon2 10.190.10.15:6789 feature set mismatch, my 4a042a42 < server's 2104a042a42, missing 21000000000 [ 1067.266100] libceph: mon2 10.190.10.15:6789 socket error on read [ 1077.280856] libceph: mon1 10.190.10.16:6789 feature set mismatch, my 4a042a42 < server's 2104a042a42, missing 21000000000 [ 1077.280882] libceph: mon1 10.190.10.16:6789 socket error on read [ 1087.296814] libceph: mon1 10.190.10.16:6789 feature set mismatch, my 4a042a42 < server's 2104a042a42, missing 21000000000 [ 1087.296836] libceph: mon1 10.190.10.16:6789 socket error on read [ 1097.313491] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042a42 < server's 2104a042a42, missing 21000000000 [ 1097.313511] libceph: mon0 10.190.10.13:6789 socket error on read [ 1242.421497] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042a42 < server's 2104a042a42, missing 21000000000 [ 1242.421518] libceph: mon0 10.190.10.13:6789 socket error on read [ 1252.449843] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042a42 < server's 2104a042a42, missing 21000000000 [ 1252.449870] libceph: mon0 10.190.10.13:6789 socket error on read [ 1262.465479] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042a42 < server's 2104a042a42, missing 21000000000 [ 1262.465505] libceph: mon0 10.190.10.13:6789 socket error on read [ 1272.482000] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042a42 < server's 2104a042a42, missing 21000000000 [ 1272.482025] libceph: mon0 10.190.10.13:6789 socket error on read [ 1282.497443] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042a42 < server's 2104a042a42, missing 21000000000 [ 1282.497468] libceph: mon0 10.190.10.13:6789 socket error on read [ 1292.513645] libceph: mon0 10.190.10.13:6789 feature set mismatch, my 4a042a42 < server's 2104a042a42, missing 21000000000 [ 1292.513670] libceph: mon0 10.190.10.13:6789 socket error on read On Jul 31, 2014, at 1:13 PM, Ilya Dryomov <ilya.dryomov at inktank.com> wrote: > On Fri, Aug 1, 2014 at 12:08 AM, Larry Liu <larryliugml at gmail.com> wrote: >> I'm on ubuntu 12.04 kernel 3.2.0-53-generic. Just deployed ceph 0.80.5. Creating pools & images work fine, but getting this error when try to map a rbi image or mount cephfs: >> rbd: add failed: (5) Input/output error >> >> I did try flipping crush tunables instructed at http://ceph.com/docs/master/rados/operations/crush-map/#tunables, but still no luck. Anyone had the same problem or fixes? > > Is there anything in dmesg? What are the exact commands you use to > create rbd image and map it? > > Thanks, > > Ilya -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140801/eb74e75b/attachment.pgp>