On Tue, Jul 5, 2016 at 1:34 PM, Patrick Donnelly <pdonnell@xxxxxxxxxx> wrote: > Hi Goncalo, > > I believe this segfault may be the one fixed here: > > https://github.com/ceph/ceph/pull/10027 Ah, nice one Patrick. Goncalo, the patch is fairly simple, just the addition of a lock on two lines to resolve the race. Could you try recompiling with those changes and let us know how it goes? Cheers, Brad > > (Sorry for brief top-post. Im on mobile.) > > On Jul 4, 2016 9:16 PM, "Goncalo Borges" <goncalo.borges@xxxxxxxxxxxxx> > wrote: >> >> Dear All... >> >> We have recently migrated all our ceph infrastructure from 9.2.0 to >> 10.2.2. >> >> We are currently using ceph-fuse to mount cephfs in a number of clients. >> >> ceph-fuse 10.2.2 client is segfaulting in some situations. One of the >> scenarios where ceph-fuse segfaults is when a user submits a parallel (mpi) >> application requesting 4 hosts with 4 cores each (16 instances in total) . >> According to the user, each instance has its own dedicated inputs and >> outputs. >> >> Please note that if we go back to ceph-fuse 9.2.0 client everything works >> fine. >> >> The ceph-fuse 10.2.2 client segfault is the following (we were able to >> capture it mounting ceph-fuse in debug mode): >>> >>> 2016-07-04 21:21:00.074087 7f6aed92be40 0 ceph version 10.2.2 >>> (45107e21c568dd033c2f0a3107dec8f0b0e58374), process ceph-fuse, pid 7346 >>> ceph-fuse[7346]: starting ceph client >>> 2016-07-04 21:21:00.107816 7f6aed92be40 -1 init, newargv = 0x7f6af8c12320 >>> newargc=11 >>> ceph-fuse[7346]: starting fuse >>> *** Caught signal (Segmentation fault) ** >>> in thread 7f69d7fff700 thread_name:ceph-fuse >>> ceph version 10.2.2 (45107e21c568dd033c2f0a3107dec8f0b0e58374) >>> 1: (()+0x297ef2) [0x7f6aedbecef2] >>> 2: (()+0x3b88c0f7e0) [0x7f6aec64b7e0] >>> 3: (Client::get_root_ino()+0x10) [0x7f6aedaf0330] >>> 4: (CephFuse::Handle::make_fake_ino(inodeno_t, snapid_t)+0x175) >>> [0x7f6aedaee035] >>> 5: (()+0x199891) [0x7f6aedaee891] >>> 6: (()+0x15b76) [0x7f6aed50db76] >>> 7: (()+0x12aa9) [0x7f6aed50aaa9] >>> 8: (()+0x3b88c07aa1) [0x7f6aec643aa1] >>> 9: (clone()+0x6d) [0x7f6aeb8d193d] >>> 2016-07-05 10:09:14.045131 7f69d7fff700 -1 *** Caught signal >>> (Segmentation fault) ** >>> in thread 7f69d7fff700 thread_name:ceph-fuse >>> >>> ceph version 10.2.2 (45107e21c568dd033c2f0a3107dec8f0b0e58374) >>> 1: (()+0x297ef2) [0x7f6aedbecef2] >>> 2: (()+0x3b88c0f7e0) [0x7f6aec64b7e0] >>> 3: (Client::get_root_ino()+0x10) [0x7f6aedaf0330] >>> 4: (CephFuse::Handle::make_fake_ino(inodeno_t, snapid_t)+0x175) >>> [0x7f6aedaee035] >>> 5: (()+0x199891) [0x7f6aedaee891] >>> 6: (()+0x15b76) [0x7f6aed50db76] >>> 7: (()+0x12aa9) [0x7f6aed50aaa9] >>> 8: (()+0x3b88c07aa1) [0x7f6aec643aa1] >>> 9: (clone()+0x6d) [0x7f6aeb8d193d] >>> NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed >>> to interpret this. >>> >>> >> The full dump is quite long. Here are the very last bits of it. Let me >> know if you need the full dump. >>> >>> --- begin dump of recent events --- >>> -9999> 2016-07-05 10:09:13.956502 7f6a5ffff700 3 client.464559 >>> _getxattr(1000037c789, "security.capability", 0) = -61 >>> -9998> 2016-07-05 10:09:13.956507 7f6aa96fa700 3 client.464559 ll_write >>> 0x7f6a08028be0 1000037c78c 20094~34 >>> -9997> 2016-07-05 10:09:13.956527 7f6aa96fa700 3 client.464559 ll_write >>> 0x7f6a08028be0 20094~34 = 34 >>> -9996> 2016-07-05 10:09:13.956535 7f69d7fff700 3 client.464559 ll_write >>> 0x7f6a100145f0 1000037c78d 28526~34 >>> -9995> 2016-07-05 10:09:13.956553 7f69d7fff700 3 client.464559 ll_write >>> 0x7f6a100145f0 28526~34 = 34 >>> -9994> 2016-07-05 10:09:13.956561 7f6ac0dfa700 3 client.464559 >>> ll_forget 1000037c78c 1 >>> -9993> 2016-07-05 10:09:13.956569 7f6a5ffff700 3 client.464559 >>> ll_forget 1000037c789 1 >>> -9992> 2016-07-05 10:09:13.956577 7f6a5ebfd700 3 client.464559 ll_write >>> 0x7f6a94006350 1000037c789 22010~216 >>> -9991> 2016-07-05 10:09:13.956594 7f6a5ebfd700 3 client.464559 ll_write >>> 0x7f6a94006350 22010~216 = 216 >>> -9990> 2016-07-05 10:09:13.956603 7f6aa8cf9700 3 client.464559 >>> ll_getxattr 1000037c78c.head security.capability size 0 >>> -9989> 2016-07-05 10:09:13.956609 7f6aa8cf9700 3 client.464559 >>> _getxattr(1000037c78c, "security.capability", 0) = -61 >>> >>> <snip> >>> >>> -160> 2016-07-05 10:09:14.043687 7f69d7fff700 3 client.464559 >>> _getxattr(1000037c78a, "security.capability", 0) = -61 >>> -159> 2016-07-05 10:09:14.043694 7f6ac0dfa700 3 client.464559 ll_write >>> 0x7f6a08042560 1000037c78b 11900~34 >>> -158> 2016-07-05 10:09:14.043712 7f6ac0dfa700 3 client.464559 ll_write >>> 0x7f6a08042560 11900~34 = 34 >>> -157> 2016-07-05 10:09:14.043722 7f6ac17fb700 3 client.464559 >>> ll_getattr 100001e9c80.head >>> -156> 2016-07-05 10:09:14.043727 7f6ac17fb700 3 client.464559 >>> ll_getattr 100001e9c80.head = 0 >>> -155> 2016-07-05 10:09:14.043734 7f69d7fff700 3 client.464559 >>> ll_forget 1000037c78a 1 >>> -154> 2016-07-05 10:09:14.043738 7f6a5ebfd700 3 client.464559 ll_write >>> 0x7f6a140d5930 1000037c78a 18292~34 >>> -153> 2016-07-05 10:09:14.043759 7f6a5ebfd700 3 client.464559 ll_write >>> 0x7f6a140d5930 18292~34 = 34 >>> -152> 2016-07-05 10:09:14.043767 7f6ac17fb700 3 client.464559 >>> ll_forget 100001e9c80 1 >>> -151> 2016-07-05 10:09:14.043784 7f6aa8cf9700 3 client.464559 ll_flush >>> 0x7f6a00049fe0 100001e9c80 >>> -150> 2016-07-05 10:09:14.043794 7f6aa8cf9700 3 client.464559 >>> ll_getxattr 1000037c78a.head security.capability size 0 >>> -149> 2016-07-05 10:09:14.043799 7f6aa8cf9700 3 client.464559 >>> _getxattr(1000037c78a, "security.capability", 0) = -61 >>> -148> 2016-07-05 10:09:14.043804 7f6aab4fd700 3 client.464559 >>> ll_release (fh)0x7f6a00049fe0 100001e9c80 >>> -147> 2016-07-05 10:09:14.043809 7f6aab4fd700 5 client.464559 >>> _release_fh 0x7f6a00049fe0 mode 1 on 100001e9c80.head(faked_ino=0 ref=3 >>> ll_ref=6 cap_refs={1024=0,2048=0} open={1=1} mode=100755 size=1208688/0 >>> mtime=2016-05-25 12:20:23.667957 caps=pAsLsXsFscr(0=pAsLsXsFscr) >>> objectset[100001e9c80 ts 0/0 objects 1 dirty_or_tx 0] parents=0x7f6acc045ec0 >>> 0x7f6acc04a600) >>> -146> 2016-07-05 10:09:14.043827 7f6aa8cf9700 3 client.464559 >>> ll_forget 1000037c78a 1 >>> -145> 2016-07-05 10:09:14.043838 7f6a5ffff700 3 client.464559 ll_write >>> 0x7f6a140d5930 1000037c78a 18326~34 >>> -144> 2016-07-05 10:09:14.043862 7f6a5ffff700 3 client.464559 ll_write >>> 0x7f6a140d5930 18326~34 = 34 >>> -143> 2016-07-05 10:09:14.043872 7f6a5f5fe700 3 client.464559 >>> ll_getxattr 1000037c78b.head security.capability size 0 >>> -142> 2016-07-05 10:09:14.043879 7f6a5f5fe700 3 client.464559 >>> _getxattr(1000037c78b, "security.capability", 0) = -61 >>> -141> 2016-07-05 10:09:14.043889 7f6a5f5fe700 3 client.464559 >>> ll_forget 1000037c78b 1 >>> -140> 2016-07-05 10:09:14.043894 7f6ac21fc700 3 client.464559 >>> ll_getxattr 1000037c78a.head security.capability size 0 >>> -139> 2016-07-05 10:09:14.043902 7f6ac21fc700 3 client.464559 >>> _getxattr(1000037c78a, "security.capability", 0) = -61 >>> -138> 2016-07-05 10:09:14.043909 7f6a5f5fe700 3 client.464559 ll_write >>> 0x7f6a08042560 1000037c78b 11934~34 >>> -137> 2016-07-05 10:09:14.043931 7f6a5f5fe700 3 client.464559 ll_write >>> 0x7f6a08042560 11934~34 = 34 >>> -136> 2016-07-05 10:09:14.043954 7f6ac0dfa700 3 client.464559 >>> ll_getxattr 1000037c78b.head security.capability size 0 >>> -135> 2016-07-05 10:09:14.043960 7f6ac0dfa700 3 client.464559 >>> _getxattr(1000037c78b, "security.capability", 0) = -61 >>> -134> 2016-07-05 10:09:14.043965 7f6aa96fa700 3 client.464559 ll_write >>> 0x7f6a140d5930 1000037c78a 18360~34 >>> -133> 2016-07-05 10:09:14.043983 7f6aa96fa700 3 client.464559 ll_write >>> 0x7f6a140d5930 18360~34 = 34 >>> -132> 2016-07-05 10:09:14.044005 7f6a5ebfd700 3 client.464559 >>> ll_getxattr 1000037c78a.head security.capability size 0 >>> -131> 2016-07-05 10:09:14.044010 7f6a5ebfd700 3 client.464559 >>> _getxattr(1000037c78a, "security.capability", 0) = -61 >>> -130> 2016-07-05 10:09:14.044015 7f6a5ebfd700 3 client.464559 >>> ll_forget 1000037c78a 1 >>> -129> 2016-07-05 10:09:14.044021 7f6a5ebfd700 3 client.464559 ll_write >>> 0x7f6a140d5930 1000037c78a 18394~34 >>> -128> 2016-07-05 10:09:14.044036 7f6a5ebfd700 3 client.464559 ll_write >>> 0x7f6a140d5930 18394~34 = 34 >>> -127> 2016-07-05 10:09:14.044043 7f6ac0dfa700 3 client.464559 >>> ll_forget 1000037c78b 1 >>> -126> 2016-07-05 10:09:14.044053 7f69d7fff700 3 client.464559 ll_write >>> 0x7f6a08042560 1000037c78b 11968~34 >>> -125> 2016-07-05 10:09:14.044067 7f69d7fff700 3 client.464559 ll_write >>> 0x7f6a08042560 11968~34 = 34 >>> -124> 2016-07-05 10:09:14.044091 7f6aab4fd700 3 client.464559 >>> ll_getxattr 1000037c78b.head security.capability size 0 >>> -123> 2016-07-05 10:09:14.044095 7f6aab4fd700 3 client.464559 >>> _getxattr(1000037c78b, "security.capability", 0) = -61 >>> -122> 2016-07-05 10:09:14.044100 7f6aab4fd700 3 client.464559 >>> ll_forget 1000037c78b 1 >>> -121> 2016-07-05 10:09:14.044104 7f6aab4fd700 3 client.464559 ll_write >>> 0x7f6a08042560 1000037c78b 12002~34 >>> -120> 2016-07-05 10:09:14.044117 7f6aab4fd700 3 client.464559 ll_write >>> 0x7f6a08042560 12002~34 = 34 >>> -119> 2016-07-05 10:09:14.044124 7f6ac17fb700 3 client.464559 >>> ll_getattr 1000009b5cb.head >>> -118> 2016-07-05 10:09:14.044129 7f6ac17fb700 3 client.464559 >>> ll_getattr 1000009b5cb.head = 0 >>> -117> 2016-07-05 10:09:14.044140 7f6ac0dfa700 3 client.464559 >>> ll_getxattr 1000037c78a.head security.capability size 0 >>> -116> 2016-07-05 10:09:14.044148 7f6ac0dfa700 3 client.464559 >>> _getxattr(1000037c78a, "security.capability", 0) = -61 >>> -115> 2016-07-05 10:09:14.044155 7f6ac17fb700 3 client.464559 >>> ll_forget 1000009b5cb 1 >>> -114> 2016-07-05 10:09:14.044163 7f6ac17fb700 3 client.464559 ll_write >>> 0x7f6a140d5930 1000037c78a 18428~34 >>> -113> 2016-07-05 10:09:14.044179 7f6ac17fb700 3 client.464559 ll_write >>> 0x7f6a140d5930 18428~34 = 34 >>> -112> 2016-07-05 10:09:14.044186 7f6a5ffff700 3 client.464559 >>> ll_lookup 0x7f6acc019b10 data >>> -111> 2016-07-05 10:09:14.044194 7f6a5ffff700 3 client.464559 >>> ll_lookup 0x7f6acc019b10 data -> 0 (1000009b9b4) >>> -110> 2016-07-05 10:09:14.044201 7f6ac0dfa700 3 client.464559 >>> ll_forget 1000037c78a 1 >>> -109> 2016-07-05 10:09:14.044209 7f6aa8cf9700 3 client.464559 >>> ll_getxattr 1000037c78b.head security.capability size 0 >>> -108> 2016-07-05 10:09:14.044215 7f6aa8cf9700 3 client.464559 >>> _getxattr(1000037c78b, "security.capability", 0) = -61 >>> -107> 2016-07-05 10:09:14.044221 7f6aa8cf9700 3 client.464559 >>> ll_forget 1000037c78b 1 >>> -106> 2016-07-05 10:09:14.044227 7f6ac21fc700 3 client.464559 >>> ll_forget 1000037c78a 1 >>> -105> 2016-07-05 10:09:14.044238 7f6a5f5fe700 3 client.464559 >>> ll_getxattr 1000037c78a.head security.capability size 0 >>> -104> 2016-07-05 10:09:14.044243 7f6a5f5fe700 3 client.464559 >>> _getxattr(1000037c78a, "security.capability", 0) = -61 >>> -103> 2016-07-05 10:09:14.044251 7f6a5f5fe700 3 client.464559 >>> ll_forget 1000037c78a 1 >>> -102> 2016-07-05 10:09:14.044258 7f6aa96fa700 3 client.464559 ll_write >>> 0x7f6a08042560 1000037c78b 12036~34 >>> -101> 2016-07-05 10:09:14.044273 7f6aa96fa700 3 client.464559 ll_write >>> 0x7f6a08042560 12036~34 = 34 >>> -100> 2016-07-05 10:09:14.044281 7f6ac0dfa700 3 client.464559 >>> ll_getattr 1000009b9b4.head >>> -99> 2016-07-05 10:09:14.044286 7f6ac0dfa700 3 client.464559 >>> ll_getattr 1000009b9b4.head = 0 >>> -98> 2016-07-05 10:09:14.044294 7f6a5f5fe700 3 client.464559 ll_write >>> 0x7f6a140d5930 1000037c78a 18462~34 >>> -97> 2016-07-05 10:09:14.044310 7f6a5f5fe700 3 client.464559 ll_write >>> 0x7f6a140d5930 18462~34 = 34 >>> -96> 2016-07-05 10:09:14.044320 7f6a5ebfd700 3 client.464559 >>> ll_getxattr 1000037c78b.head security.capability size 0 >>> -95> 2016-07-05 10:09:14.044324 7f6a5ebfd700 3 client.464559 >>> _getxattr(1000037c78b, "security.capability", 0) = -61 >>> -94> 2016-07-05 10:09:14.044331 7f6a5ffff700 3 client.464559 >>> ll_forget 1000009b5cb 1 >>> -93> 2016-07-05 10:09:14.044348 7f6ac17fb700 3 client.464559 ll_write >>> 0x7f6a08042560 1000037c78b 12070~34 >>> -92> 2016-07-05 10:09:14.044371 7f6ac17fb700 3 client.464559 ll_write >>> 0x7f6a08042560 12070~34 = 34 >>> -91> 2016-07-05 10:09:14.044381 7f6ac0dfa700 3 client.464559 >>> ll_forget 1000009b9b4 1 >>> -90> 2016-07-05 10:09:14.044389 7f6a5ebfd700 3 client.464559 >>> ll_forget 1000037c78b 1 >>> -89> 2016-07-05 10:09:14.044397 7f6aab4fd700 3 client.464559 >>> ll_getxattr 1000037c78a.head security.capability size 0 >>> -88> 2016-07-05 10:09:14.044401 7f6aab4fd700 3 client.464559 >>> _getxattr(1000037c78a, "security.capability", 0) = -61 >>> -87> 2016-07-05 10:09:14.044407 7f6aa8cf9700 3 client.464559 >>> ll_getxattr 1000037c78b.head security.capability size 0 >>> -86> 2016-07-05 10:09:14.044413 7f6aa8cf9700 3 client.464559 >>> _getxattr(1000037c78b, "security.capability", 0) = -61 >>> -85> 2016-07-05 10:09:14.044423 7f69d7fff700 3 client.464559 >>> ll_lookup 0x7f6acc026050 ruggerod >>> -84> 2016-07-05 10:09:14.044428 7f69d7fff700 3 client.464559 >>> ll_lookup 0x7f6acc026050 ruggerod -> 0 (10000174776) >>> -83> 2016-07-05 10:09:14.044432 7f6ac21fc700 3 client.464559 ll_write >>> 0x7f6a08042560 1000037c78b 12104~34 >>> -82> 2016-07-05 10:09:14.044454 7f6ac21fc700 3 client.464559 ll_write >>> 0x7f6a08042560 12104~34 = 34 >>> -81> 2016-07-05 10:09:14.044464 7f6aab4fd700 3 client.464559 >>> ll_forget 1000037c78a 1 >>> -80> 2016-07-05 10:09:14.044471 7f6aa8cf9700 3 client.464559 >>> ll_forget 1000037c78b 1 >>> -79> 2016-07-05 10:09:14.044481 7f6aa96fa700 3 client.464559 ll_write >>> 0x7f6a140d5930 1000037c78a 18496~34 >>> -78> 2016-07-05 10:09:14.044497 7f6aa96fa700 3 client.464559 ll_write >>> 0x7f6a140d5930 18496~34 = 34 >>> -77> 2016-07-05 10:09:14.044507 7f6aa8cf9700 3 client.464559 >>> ll_getxattr 1000037c78b.head security.capability size 0 >>> -76> 2016-07-05 10:09:14.044513 7f6aa8cf9700 3 client.464559 >>> _getxattr(1000037c78b, "security.capability", 0) = -61 >>> -75> 2016-07-05 10:09:14.044519 7f6aa8cf9700 3 client.464559 >>> ll_forget 1000037c78b 1 >>> -74> 2016-07-05 10:09:14.044525 7f6a5f5fe700 3 client.464559 >>> ll_getxattr 1000037c78a.head security.capability size 0 >>> -73> 2016-07-05 10:09:14.044532 7f6a5f5fe700 3 client.464559 >>> _getxattr(1000037c78a, "security.capability", 0) = -61 >>> -72> 2016-07-05 10:09:14.044539 7f6aa8cf9700 3 client.464559 ll_write >>> 0x7f6a08042560 1000037c78b 12138~34 >>> -71> 2016-07-05 10:09:14.044558 7f6aa8cf9700 3 client.464559 ll_write >>> 0x7f6a08042560 12138~34 = 34 >>> -70> 2016-07-05 10:09:14.044565 7f6a5f5fe700 3 client.464559 >>> ll_forget 1000037c78a 1 >>> -69> 2016-07-05 10:09:14.044573 7f6a5ffff700 3 client.464559 ll_write >>> 0x7f6a140d5930 1000037c78a 18530~34 >>> -68> 2016-07-05 10:09:14.044594 7f6a5ffff700 3 client.464559 ll_write >>> 0x7f6a140d5930 18530~34 = 34 >>> -67> 2016-07-05 10:09:14.044602 7f6ac17fb700 3 client.464559 >>> ll_getxattr 1000037c78b.head security.capability size 0 >>> -66> 2016-07-05 10:09:14.044607 7f6ac17fb700 3 client.464559 >>> _getxattr(1000037c78b, "security.capability", 0) = -61 >>> -65> 2016-07-05 10:09:14.044612 7f6ac17fb700 3 client.464559 >>> ll_forget 1000037c78b 1 >>> -64> 2016-07-05 10:09:14.044619 7f6ac17fb700 3 client.464559 >>> ll_getxattr 1000037c78a.head security.capability size 0 >>> -63> 2016-07-05 10:09:14.044623 7f6ac17fb700 3 client.464559 >>> _getxattr(1000037c78a, "security.capability", 0) = -61 >>> -62> 2016-07-05 10:09:14.044629 7f6a5ebfd700 3 client.464559 ll_write >>> 0x7f6a08042560 1000037c78b 12172~34 >>> -61> 2016-07-05 10:09:14.044649 7f6a5ebfd700 3 client.464559 ll_write >>> 0x7f6a08042560 12172~34 = 34 >>> -60> 2016-07-05 10:09:14.044657 7f6ac17fb700 3 client.464559 >>> ll_forget 1000037c78a 1 >>> -59> 2016-07-05 10:09:14.044665 7f6ac21fc700 3 client.464559 ll_write >>> 0x7f6a140d5930 1000037c78a 18564~34 >>> -58> 2016-07-05 10:09:14.044684 7f6ac21fc700 3 client.464559 ll_write >>> 0x7f6a140d5930 18564~34 = 34 >>> -57> 2016-07-05 10:09:14.044692 7f6aab4fd700 3 client.464559 >>> ll_getxattr 1000037c78b.head security.capability size 0 >>> -56> 2016-07-05 10:09:14.044697 7f6aab4fd700 3 client.464559 >>> _getxattr(1000037c78b, "security.capability", 0) = -61 >>> -55> 2016-07-05 10:09:14.044701 7f6aab4fd700 3 client.464559 >>> ll_forget 1000037c78b 1 >>> -54> 2016-07-05 10:09:14.044709 7f6aab4fd700 3 client.464559 ll_write >>> 0x7f6a08042560 1000037c78b 12206~34 >>> -53> 2016-07-05 10:09:14.044723 7f6aab4fd700 3 client.464559 ll_write >>> 0x7f6a08042560 12206~34 = 34 >>> -52> 2016-07-05 10:09:14.044732 7f6aa96fa700 3 client.464559 >>> ll_getxattr 1000037c78a.head security.capability size 0 >>> -51> 2016-07-05 10:09:14.044738 7f6aa96fa700 3 client.464559 >>> _getxattr(1000037c78a, "security.capability", 0) = -61 >>> -50> 2016-07-05 10:09:14.044745 7f6aa96fa700 3 client.464559 >>> ll_forget 1000037c78a 1 >>> -49> 2016-07-05 10:09:14.044749 7f6ac0dfa700 3 client.464559 >>> ll_getxattr 1000037c78b.head security.capability size 0 >>> -48> 2016-07-05 10:09:14.044757 7f6ac0dfa700 3 client.464559 >>> _getxattr(1000037c78b, "security.capability", 0) = -61 >>> -47> 2016-07-05 10:09:14.044764 7f6aa96fa700 3 client.464559 ll_write >>> 0x7f6a140d5930 1000037c78a 18598~34 >>> -46> 2016-07-05 10:09:14.044783 7f6aa96fa700 3 client.464559 ll_write >>> 0x7f6a140d5930 18598~34 = 34 >>> -45> 2016-07-05 10:09:14.044790 7f6ac0dfa700 3 client.464559 >>> ll_forget 1000037c78b 1 >>> -44> 2016-07-05 10:09:14.044798 7f6aa8cf9700 3 client.464559 ll_write >>> 0x7f6a08042560 1000037c78b 12240~34 >>> -43> 2016-07-05 10:09:14.044815 7f6aa8cf9700 3 client.464559 ll_write >>> 0x7f6a08042560 12240~34 = 34 >>> -42> 2016-07-05 10:09:14.044822 7f6a5f5fe700 3 client.464559 >>> ll_getxattr 1000037c78a.head security.capability size 0 >>> -41> 2016-07-05 10:09:14.044828 7f6a5f5fe700 3 client.464559 >>> _getxattr(1000037c78a, "security.capability", 0) = -61 >>> -40> 2016-07-05 10:09:14.044832 7f6a5f5fe700 3 client.464559 >>> ll_forget 1000037c78a 1 >>> -39> 2016-07-05 10:09:14.044841 7f6a5f5fe700 3 client.464559 >>> ll_getxattr 1000037c78b.head security.capability size 0 >>> -38> 2016-07-05 10:09:14.044845 7f6a5f5fe700 3 client.464559 >>> _getxattr(1000037c78b, "security.capability", 0) = -61 >>> -37> 2016-07-05 10:09:14.044853 7f6a5ebfd700 3 client.464559 ll_write >>> 0x7f6a140d5930 1000037c78a 18632~34 >>> -36> 2016-07-05 10:09:14.044869 7f6a5ebfd700 3 client.464559 ll_write >>> 0x7f6a140d5930 18632~34 = 34 >>> -35> 2016-07-05 10:09:14.044876 7f6a5f5fe700 3 client.464559 >>> ll_forget 1000037c78b 1 >>> -34> 2016-07-05 10:09:14.044889 7f6ac17fb700 3 client.464559 ll_write >>> 0x7f6a08042560 1000037c78b 12274~34 >>> -33> 2016-07-05 10:09:14.044909 7f6ac17fb700 3 client.464559 ll_write >>> 0x7f6a08042560 12274~34 = 34 >>> -32> 2016-07-05 10:09:14.044919 7f6ac21fc700 3 client.464559 >>> ll_getxattr 1000037c78a.head security.capability size 0 >>> -31> 2016-07-05 10:09:14.044924 7f6ac21fc700 3 client.464559 >>> _getxattr(1000037c78a, "security.capability", 0) = -61 >>> -30> 2016-07-05 10:09:14.044932 7f6ac21fc700 3 client.464559 >>> ll_forget 1000037c78a 1 >>> -29> 2016-07-05 10:09:14.044936 7f6a5ffff700 3 client.464559 >>> ll_getxattr 1000037c78b.head security.capability size 0 >>> -28> 2016-07-05 10:09:14.044942 7f6a5ffff700 3 client.464559 >>> _getxattr(1000037c78b, "security.capability", 0) = -61 >>> -27> 2016-07-05 10:09:14.044948 7f6ac21fc700 3 client.464559 ll_write >>> 0x7f6a140d5930 1000037c78a 18666~34 >>> -26> 2016-07-05 10:09:14.044970 7f6ac21fc700 3 client.464559 ll_write >>> 0x7f6a140d5930 18666~34 = 34 >>> -25> 2016-07-05 10:09:14.044978 7f6a5ffff700 3 client.464559 >>> ll_forget 1000037c78b 1 >>> -24> 2016-07-05 10:09:14.044987 7f6aab4fd700 3 client.464559 ll_write >>> 0x7f6a08042560 1000037c78b 12308~34 >>> -23> 2016-07-05 10:09:14.045003 7f6aab4fd700 3 client.464559 ll_write >>> 0x7f6a08042560 12308~34 = 34 >>> -22> 2016-07-05 10:09:14.045012 7f6aa96fa700 3 client.464559 >>> ll_getxattr 1000037c78a.head security.capability size 0 >>> -21> 2016-07-05 10:09:14.045018 7f6aa96fa700 3 client.464559 >>> _getxattr(1000037c78a, "security.capability", 0) = -61 >>> -20> 2016-07-05 10:09:14.045025 7f6aa96fa700 3 client.464559 >>> ll_forget 1000037c78a 1 >>> -19> 2016-07-05 10:09:14.045028 7f6ac0dfa700 3 client.464559 >>> ll_getxattr 1000037c78b.head security.capability size 0 >>> -18> 2016-07-05 10:09:14.045031 7f6ac0dfa700 3 client.464559 >>> _getxattr(1000037c78b, "security.capability", 0) = -61 >>> -17> 2016-07-05 10:09:14.045037 7f6aa96fa700 3 client.464559 ll_write >>> 0x7f6a140d5930 1000037c78a 18700~34 >>> -16> 2016-07-05 10:09:14.045053 7f6aa96fa700 3 client.464559 ll_write >>> 0x7f6a140d5930 18700~34 = 34 >>> -15> 2016-07-05 10:09:14.045059 7f6ac0dfa700 3 client.464559 >>> ll_forget 1000037c78b 1 >>> -14> 2016-07-05 10:09:14.045065 7f6aa8cf9700 3 client.464559 ll_write >>> 0x7f6a08042560 1000037c78b 12342~34 >>> -13> 2016-07-05 10:09:14.045082 7f6aa8cf9700 3 client.464559 ll_write >>> 0x7f6a08042560 12342~34 = 34 >>> -12> 2016-07-05 10:09:14.045090 7f6a5ebfd700 3 client.464559 >>> ll_getxattr 1000037c78a.head security.capability size 0 >>> -11> 2016-07-05 10:09:14.045095 7f6a5ebfd700 3 client.464559 >>> _getxattr(1000037c78a, "security.capability", 0) = -61 >>> -10> 2016-07-05 10:09:14.045101 7f6a5ebfd700 3 client.464559 >>> ll_forget 1000037c78a 1 >>> -9> 2016-07-05 10:09:14.045112 7f6a5f5fe700 3 client.464559 >>> ll_getxattr 1000037c78b.head security.capability size 0 >>> -8> 2016-07-05 10:09:14.045117 7f6a5f5fe700 3 client.464559 >>> _getxattr(1000037c78b, "security.capability", 0) = -61 >>> -7> 2016-07-05 10:09:14.045122 7f6ac17fb700 3 client.464559 ll_write >>> 0x7f6a140d5930 1000037c78a 18734~34 >>> -6> 2016-07-05 10:09:14.045139 7f6ac17fb700 3 client.464559 ll_write >>> 0x7f6a140d5930 18734~34 = 34 >>> -5> 2016-07-05 10:09:14.045131 7f69d7fff700 -1 *** Caught signal >>> (Segmentation fault) ** >>> in thread 7f69d7fff700 thread_name:ceph-fuse >>> >>> ceph version 10.2.2 (45107e21c568dd033c2f0a3107dec8f0b0e58374) >>> 1: (()+0x297ef2) [0x7f6aedbecef2] >>> 2: (()+0x3b88c0f7e0) [0x7f6aec64b7e0] >>> 3: (Client::get_root_ino()+0x10) [0x7f6aedaf0330] >>> 4: (CephFuse::Handle::make_fake_ino(inodeno_t, snapid_t)+0x175) >>> [0x7f6aedaee035] >>> 5: (()+0x199891) [0x7f6aedaee891] >>> 6: (()+0x15b76) [0x7f6aed50db76] >>> 7: (()+0x12aa9) [0x7f6aed50aaa9] >>> 8: (()+0x3b88c07aa1) [0x7f6aec643aa1] >>> 9: (clone()+0x6d) [0x7f6aeb8d193d] >>> NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed >>> to interpret this. >>> >>> -4> 2016-07-05 10:09:14.045146 7f6a5f5fe700 3 client.464559 >>> ll_forget 1000037c78b 1 >>> -3> 2016-07-05 10:09:14.045152 7f6ac21fc700 3 client.464559 ll_write >>> 0x7f6a08042560 1000037c78b 12376~34 >>> -2> 2016-07-05 10:09:14.045165 7f6ac21fc700 3 client.464559 ll_write >>> 0x7f6a08042560 12376~34 = 34 >>> -1> 2016-07-05 10:09:14.045171 7f6a5ffff700 3 client.464559 >>> ll_getxattr 1000037c78a.head security.capability size 0 >>> 0> 2016-07-05 10:09:14.045175 7f6a5ffff700 3 client.464559 >>> _getxattr(1000037c78a, "security.capability", 0) = -61 >>> --- logging levels --- >>> 0/ 5 none >>> 0/ 1 lockdep >>> 0/ 1 context >>> 1/ 1 crush >>> 1/ 5 mds >>> 1/ 5 mds_balancer >>> 1/ 5 mds_locker >>> 1/ 5 mds_log >>> 1/ 5 mds_log_expire >>> 1/ 5 mds_migrator >>> 0/ 1 buffer >>> 0/ 1 timer >>> 0/ 1 filer >>> 0/ 1 striper >>> 0/ 1 objecter >>> 0/ 5 rados >>> 0/ 5 rbd >>> 0/ 5 rbd_mirror >>> 0/ 5 rbd_replay >>> 0/ 5 journaler >>> 0/ 5 objectcacher >>> 0/ 5 client >>> 0/ 5 osd >>> 0/ 5 optracker >>> 0/ 5 objclass >>> 1/ 3 filestore >>> 1/ 3 journal >>> 0/ 5 ms >>> 1/ 5 mon >>> 0/10 monc >>> 1/ 5 paxos >>> 0/ 5 tp >>> 1/ 5 auth >>> 1/ 5 crypto >>> 1/ 1 finisher >>> 1/ 5 heartbeatmap >>> 1/ 5 perfcounter >>> 1/ 5 rgw >>> 1/10 civetweb >>> 1/ 5 javaclient >>> 1/ 5 asok >>> 1/ 1 throttle >>> 0/ 0 refs >>> 1/ 5 xio >>> 1/ 5 compressor >>> 1/ 5 newstore >>> 1/ 5 bluestore >>> 1/ 5 bluefs >>> 1/ 3 bdev >>> 1/ 5 kstore >>> 4/ 5 rocksdb >>> 4/ 5 leveldb >>> 1/ 5 kinetic >>> 1/ 5 fuse >>> -2/-2 (syslog threshold) >>> 99/99 (stderr threshold) >>> max_recent 10000 >>> max_new 1000 >>> log_file >>> --- end dump of recent events --- >> >> >> Note sure if this is relevant but we successfully compiled ceph-fuse >> ourselfs (both 9.2.0 and 10.2.2) in an sl6 machine with with gcc 4.8.4, >> python 2.7.6, boost 1.53.0-25 and fuse 2.9.7. We do this because of the >> nature of the research performed by our scientists where there is (still) >> the requirement to provide cephfs under sl6. >> >> Any help in understanding why this is happening would be very appreciated. >> >> Cheers >> >> Goncalo >> >> -- >> Goncalo Borges >> Research Computing >> ARC Centre of Excellence for Particle Physics at the Terascale >> School of Physics A28 | University of Sydney, NSW 2006 >> T: +61 2 93511937 >> >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Cheers, Brad _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |