|
Hi All...
Just to confirm that, after applying the patch and recompiling,
we are no longer seeing segfaults.
I just tested with a user application which would kill ceph-fuse
almost instantaneously. Now it is running for quite some time,
reading and updating the files that it should.
I should test with other applications which were also triggering
the ceph-fuse segfault, but for now, it is looking good.
Thanks Patrick for pointing this out.
Is there a particular reason why in 9.2.0 we were not getting
such segfaults? I am asking because the patch was simply to
introduce two lock functions in two specific lines of
src/client/Client.cc which, I imagine, were also not there in
9.2.0 (unless there was a big rewrite of src/client/Client.cc from
9.2.0 to 10.2.2)
Cheers
Goncalo
On 07/05/2016 02:45 PM, Goncalo Borges
wrote:
Hi Brad, Shinobu, Patrick...
Indeed if I run with 'debug client = 20' it seems I get a
very similar log to what Patrick has in the patch. However it
is difficult for me to really say if it is exactly the same
thing.
One thing I could try is simply to apply the fix
in the source code and recompile. Is this something safe to
do?
Cheers
Goncalo
On 07/05/2016 01:34 PM, Patrick
Donnelly wrote:
Hi Goncalo,
I believe this segfault may be the one fixed here:
https://github.com/ceph/ceph/pull/10027
(Sorry for brief top-post. Im on mobile.)
On Jul 4, 2016 9:16 PM, "Goncalo Borges" <goncalo.borges@xxxxxxxxxxxxx>
wrote:
>
> Dear All...
>
> We have recently migrated all our ceph infrastructure
from 9.2.0 to 10.2.2.
>
> We are currently using ceph-fuse to mount cephfs in a
number of clients.
>
> ceph-fuse 10.2.2 client is segfaulting in some
situations. One of the scenarios where ceph-fuse segfaults is
when a user submits a parallel (mpi) application requesting 4
hosts with 4 cores each (16 instances in total) . According to
the user, each instance has its own dedicated inputs and
outputs.
>
> Please note that if we go back to ceph-fuse 9.2.0 client
everything works fine.
>
> The ceph-fuse 10.2.2 client segfault is the following (we
were able to capture it mounting ceph-fuse in debug mode):
>>
>> 2016-07-04 21:21:00.074087 7f6aed92be40 0 ceph
version 10.2.2 (45107e21c568dd033c2f0a3107dec8f0b0e58374),
process ceph-fuse, pid 7346
>> ceph-fuse[7346]: starting ceph client
>> 2016-07-04 21:21:00.107816 7f6aed92be40 -1 init,
newargv = 0x7f6af8c12320 newargc=11
>> ceph-fuse[7346]: starting fuse
>> *** Caught signal (Segmentation fault) **
>> in thread 7f69d7fff700 thread_name:ceph-fuse
>> ceph version 10.2.2
(45107e21c568dd033c2f0a3107dec8f0b0e58374)
>> 1: (()+0x297ef2) [0x7f6aedbecef2]
>> 2: (()+0x3b88c0f7e0) [0x7f6aec64b7e0]
>> 3: (Client::get_root_ino()+0x10) [0x7f6aedaf0330]
>> 4: (CephFuse::Handle::make_fake_ino(inodeno_t,
snapid_t)+0x175) [0x7f6aedaee035]
>> 5: (()+0x199891) [0x7f6aedaee891]
>> 6: (()+0x15b76) [0x7f6aed50db76]
>> 7: (()+0x12aa9) [0x7f6aed50aaa9]
>> 8: (()+0x3b88c07aa1) [0x7f6aec643aa1]
>> 9: (clone()+0x6d) [0x7f6aeb8d193d]
>> 2016-07-05 10:09:14.045131 7f69d7fff700 -1 *** Caught
signal (Segmentation fault) **
>> in thread 7f69d7fff700 thread_name:ceph-fuse
>>
>> ceph version 10.2.2
(45107e21c568dd033c2f0a3107dec8f0b0e58374)
>> 1: (()+0x297ef2) [0x7f6aedbecef2]
>> 2: (()+0x3b88c0f7e0) [0x7f6aec64b7e0]
>> 3: (Client::get_root_ino()+0x10) [0x7f6aedaf0330]
>> 4: (CephFuse::Handle::make_fake_ino(inodeno_t,
snapid_t)+0x175) [0x7f6aedaee035]
>> 5: (()+0x199891) [0x7f6aedaee891]
>> 6: (()+0x15b76) [0x7f6aed50db76]
>> 7: (()+0x12aa9) [0x7f6aed50aaa9]
>> 8: (()+0x3b88c07aa1) [0x7f6aec643aa1]
>> 9: (clone()+0x6d) [0x7f6aeb8d193d]
>> NOTE: a copy of the executable, or `objdump -rdS
<executable>` is needed to interpret this.
>>
>>
> The full dump is quite long. Here are the very last bits
of it. Let me know if you need the full dump.
>>
>> --- begin dump of recent events ---
>> -9999> 2016-07-05 10:09:13.956502 7f6a5ffff700 3
client.464559 _getxattr(1000037c789, "security.capability", 0)
= -61
>> -9998> 2016-07-05 10:09:13.956507 7f6aa96fa700 3
client.464559 ll_write 0x7f6a08028be0 1000037c78c 20094~34
>> -9997> 2016-07-05 10:09:13.956527 7f6aa96fa700 3
client.464559 ll_write 0x7f6a08028be0 20094~34 = 34
>> -9996> 2016-07-05 10:09:13.956535 7f69d7fff700 3
client.464559 ll_write 0x7f6a100145f0 1000037c78d 28526~34
>> -9995> 2016-07-05 10:09:13.956553 7f69d7fff700 3
client.464559 ll_write 0x7f6a100145f0 28526~34 = 34
>> -9994> 2016-07-05 10:09:13.956561 7f6ac0dfa700 3
client.464559 ll_forget 1000037c78c 1
>> -9993> 2016-07-05 10:09:13.956569 7f6a5ffff700 3
client.464559 ll_forget 1000037c789 1
>> -9992> 2016-07-05 10:09:13.956577 7f6a5ebfd700 3
client.464559 ll_write 0x7f6a94006350 1000037c789 22010~216
>> -9991> 2016-07-05 10:09:13.956594 7f6a5ebfd700 3
client.464559 ll_write 0x7f6a94006350 22010~216 = 216
>> -9990> 2016-07-05 10:09:13.956603 7f6aa8cf9700 3
client.464559 ll_getxattr 1000037c78c.head security.capability
size 0
>> -9989> 2016-07-05 10:09:13.956609 7f6aa8cf9700 3
client.464559 _getxattr(1000037c78c, "security.capability", 0)
= -61
>>
>> <snip>
>>
>> -160> 2016-07-05 10:09:14.043687 7f69d7fff700 3
client.464559 _getxattr(1000037c78a, "security.capability", 0)
= -61
>> -159> 2016-07-05 10:09:14.043694 7f6ac0dfa700 3
client.464559 ll_write 0x7f6a08042560 1000037c78b 11900~34
>> -158> 2016-07-05 10:09:14.043712 7f6ac0dfa700 3
client.464559 ll_write 0x7f6a08042560 11900~34 = 34
>> -157> 2016-07-05 10:09:14.043722 7f6ac17fb700 3
client.464559 ll_getattr 100001e9c80.head
>> -156> 2016-07-05 10:09:14.043727 7f6ac17fb700 3
client.464559 ll_getattr 100001e9c80.head = 0
>> -155> 2016-07-05 10:09:14.043734 7f69d7fff700 3
client.464559 ll_forget 1000037c78a 1
>> -154> 2016-07-05 10:09:14.043738 7f6a5ebfd700 3
client.464559 ll_write 0x7f6a140d5930 1000037c78a 18292~34
>> -153> 2016-07-05 10:09:14.043759 7f6a5ebfd700 3
client.464559 ll_write 0x7f6a140d5930 18292~34 = 34
>> -152> 2016-07-05 10:09:14.043767 7f6ac17fb700 3
client.464559 ll_forget 100001e9c80 1
>> -151> 2016-07-05 10:09:14.043784 7f6aa8cf9700 3
client.464559 ll_flush 0x7f6a00049fe0 100001e9c80
>> -150> 2016-07-05 10:09:14.043794 7f6aa8cf9700 3
client.464559 ll_getxattr 1000037c78a.head security.capability
size 0
>> -149> 2016-07-05 10:09:14.043799 7f6aa8cf9700 3
client.464559 _getxattr(1000037c78a, "security.capability", 0)
= -61
>> -148> 2016-07-05 10:09:14.043804 7f6aab4fd700 3
client.464559 ll_release (fh)0x7f6a00049fe0 100001e9c80
>> -147> 2016-07-05 10:09:14.043809 7f6aab4fd700 5
client.464559 _release_fh 0x7f6a00049fe0 mode 1 on
100001e9c80.head(faked_ino=0 ref=3 ll_ref=6
cap_refs={1024=0,2048=0} open={1=1} mode=100755 size=1208688/0
mtime=2016-05-25 12:20:23.667957
caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100001e9c80 ts 0/0
objects 1 dirty_or_tx 0] parents=0x7f6acc045ec0
0x7f6acc04a600)
>> -146> 2016-07-05 10:09:14.043827 7f6aa8cf9700 3
client.464559 ll_forget 1000037c78a 1
>> -145> 2016-07-05 10:09:14.043838 7f6a5ffff700 3
client.464559 ll_write 0x7f6a140d5930 1000037c78a 18326~34
>> -144> 2016-07-05 10:09:14.043862 7f6a5ffff700 3
client.464559 ll_write 0x7f6a140d5930 18326~34 = 34
>> -143> 2016-07-05 10:09:14.043872 7f6a5f5fe700 3
client.464559 ll_getxattr 1000037c78b.head security.capability
size 0
>> -142> 2016-07-05 10:09:14.043879 7f6a5f5fe700 3
client.464559 _getxattr(1000037c78b, "security.capability", 0)
= -61
>> -141> 2016-07-05 10:09:14.043889 7f6a5f5fe700 3
client.464559 ll_forget 1000037c78b 1
>> -140> 2016-07-05 10:09:14.043894 7f6ac21fc700 3
client.464559 ll_getxattr 1000037c78a.head security.capability
size 0
>> -139> 2016-07-05 10:09:14.043902 7f6ac21fc700 3
client.464559 _getxattr(1000037c78a, "security.capability", 0)
= -61
>> -138> 2016-07-05 10:09:14.043909 7f6a5f5fe700 3
client.464559 ll_write 0x7f6a08042560 1000037c78b 11934~34
>> -137> 2016-07-05 10:09:14.043931 7f6a5f5fe700 3
client.464559 ll_write 0x7f6a08042560 11934~34 = 34
>> -136> 2016-07-05 10:09:14.043954 7f6ac0dfa700 3
client.464559 ll_getxattr 1000037c78b.head security.capability
size 0
>> -135> 2016-07-05 10:09:14.043960 7f6ac0dfa700 3
client.464559 _getxattr(1000037c78b, "security.capability", 0)
= -61
>> -134> 2016-07-05 10:09:14.043965 7f6aa96fa700 3
client.464559 ll_write 0x7f6a140d5930 1000037c78a 18360~34
>> -133> 2016-07-05 10:09:14.043983 7f6aa96fa700 3
client.464559 ll_write 0x7f6a140d5930 18360~34 = 34
>> -132> 2016-07-05 10:09:14.044005 7f6a5ebfd700 3
client.464559 ll_getxattr 1000037c78a.head security.capability
size 0
>> -131> 2016-07-05 10:09:14.044010 7f6a5ebfd700 3
client.464559 _getxattr(1000037c78a, "security.capability", 0)
= -61
>> -130> 2016-07-05 10:09:14.044015 7f6a5ebfd700 3
client.464559 ll_forget 1000037c78a 1
>> -129> 2016-07-05 10:09:14.044021 7f6a5ebfd700 3
client.464559 ll_write 0x7f6a140d5930 1000037c78a 18394~34
>> -128> 2016-07-05 10:09:14.044036 7f6a5ebfd700 3
client.464559 ll_write 0x7f6a140d5930 18394~34 = 34
>> -127> 2016-07-05 10:09:14.044043 7f6ac0dfa700 3
client.464559 ll_forget 1000037c78b 1
>> -126> 2016-07-05 10:09:14.044053 7f69d7fff700 3
client.464559 ll_write 0x7f6a08042560 1000037c78b 11968~34
>> -125> 2016-07-05 10:09:14.044067 7f69d7fff700 3
client.464559 ll_write 0x7f6a08042560 11968~34 = 34
>> -124> 2016-07-05 10:09:14.044091 7f6aab4fd700 3
client.464559 ll_getxattr 1000037c78b.head security.capability
size 0
>> -123> 2016-07-05 10:09:14.044095 7f6aab4fd700 3
client.464559 _getxattr(1000037c78b, "security.capability", 0)
= -61
>> -122> 2016-07-05 10:09:14.044100 7f6aab4fd700 3
client.464559 ll_forget 1000037c78b 1
>> -121> 2016-07-05 10:09:14.044104 7f6aab4fd700 3
client.464559 ll_write 0x7f6a08042560 1000037c78b 12002~34
>> -120> 2016-07-05 10:09:14.044117 7f6aab4fd700 3
client.464559 ll_write 0x7f6a08042560 12002~34 = 34
>> -119> 2016-07-05 10:09:14.044124 7f6ac17fb700 3
client.464559 ll_getattr 1000009b5cb.head
>> -118> 2016-07-05 10:09:14.044129 7f6ac17fb700 3
client.464559 ll_getattr 1000009b5cb.head = 0
>> -117> 2016-07-05 10:09:14.044140 7f6ac0dfa700 3
client.464559 ll_getxattr 1000037c78a.head security.capability
size 0
>> -116> 2016-07-05 10:09:14.044148 7f6ac0dfa700 3
client.464559 _getxattr(1000037c78a, "security.capability", 0)
= -61
>> -115> 2016-07-05 10:09:14.044155 7f6ac17fb700 3
client.464559 ll_forget 1000009b5cb 1
>> -114> 2016-07-05 10:09:14.044163 7f6ac17fb700 3
client.464559 ll_write 0x7f6a140d5930 1000037c78a 18428~34
>> -113> 2016-07-05 10:09:14.044179 7f6ac17fb700 3
client.464559 ll_write 0x7f6a140d5930 18428~34 = 34
>> -112> 2016-07-05 10:09:14.044186 7f6a5ffff700 3
client.464559 ll_lookup 0x7f6acc019b10 data
>> -111> 2016-07-05 10:09:14.044194 7f6a5ffff700 3
client.464559 ll_lookup 0x7f6acc019b10 data -> 0
(1000009b9b4)
>> -110> 2016-07-05 10:09:14.044201 7f6ac0dfa700 3
client.464559 ll_forget 1000037c78a 1
>> -109> 2016-07-05 10:09:14.044209 7f6aa8cf9700 3
client.464559 ll_getxattr 1000037c78b.head security.capability
size 0
>> -108> 2016-07-05 10:09:14.044215 7f6aa8cf9700 3
client.464559 _getxattr(1000037c78b, "security.capability", 0)
= -61
>> -107> 2016-07-05 10:09:14.044221 7f6aa8cf9700 3
client.464559 ll_forget 1000037c78b 1
>> -106> 2016-07-05 10:09:14.044227 7f6ac21fc700 3
client.464559 ll_forget 1000037c78a 1
>> -105> 2016-07-05 10:09:14.044238 7f6a5f5fe700 3
client.464559 ll_getxattr 1000037c78a.head security.capability
size 0
>> -104> 2016-07-05 10:09:14.044243 7f6a5f5fe700 3
client.464559 _getxattr(1000037c78a, "security.capability", 0)
= -61
>> -103> 2016-07-05 10:09:14.044251 7f6a5f5fe700 3
client.464559 ll_forget 1000037c78a 1
>> -102> 2016-07-05 10:09:14.044258 7f6aa96fa700 3
client.464559 ll_write 0x7f6a08042560 1000037c78b 12036~34
>> -101> 2016-07-05 10:09:14.044273 7f6aa96fa700 3
client.464559 ll_write 0x7f6a08042560 12036~34 = 34
>> -100> 2016-07-05 10:09:14.044281 7f6ac0dfa700 3
client.464559 ll_getattr 1000009b9b4.head
>> -99> 2016-07-05 10:09:14.044286 7f6ac0dfa700 3
client.464559 ll_getattr 1000009b9b4.head = 0
>> -98> 2016-07-05 10:09:14.044294 7f6a5f5fe700 3
client.464559 ll_write 0x7f6a140d5930 1000037c78a 18462~34
>> -97> 2016-07-05 10:09:14.044310 7f6a5f5fe700 3
client.464559 ll_write 0x7f6a140d5930 18462~34 = 34
>> -96> 2016-07-05 10:09:14.044320 7f6a5ebfd700 3
client.464559 ll_getxattr 1000037c78b.head security.capability
size 0
>> -95> 2016-07-05 10:09:14.044324 7f6a5ebfd700 3
client.464559 _getxattr(1000037c78b, "security.capability", 0)
= -61
>> -94> 2016-07-05 10:09:14.044331 7f6a5ffff700 3
client.464559 ll_forget 1000009b5cb 1
>> -93> 2016-07-05 10:09:14.044348 7f6ac17fb700 3
client.464559 ll_write 0x7f6a08042560 1000037c78b 12070~34
>> -92> 2016-07-05 10:09:14.044371 7f6ac17fb700 3
client.464559 ll_write 0x7f6a08042560 12070~34 = 34
>> -91> 2016-07-05 10:09:14.044381 7f6ac0dfa700 3
client.464559 ll_forget 1000009b9b4 1
>> -90> 2016-07-05 10:09:14.044389 7f6a5ebfd700 3
client.464559 ll_forget 1000037c78b 1
>> -89> 2016-07-05 10:09:14.044397 7f6aab4fd700 3
client.464559 ll_getxattr 1000037c78a.head security.capability
size 0
>> -88> 2016-07-05 10:09:14.044401 7f6aab4fd700 3
client.464559 _getxattr(1000037c78a, "security.capability", 0)
= -61
>> -87> 2016-07-05 10:09:14.044407 7f6aa8cf9700 3
client.464559 ll_getxattr 1000037c78b.head security.capability
size 0
>> -86> 2016-07-05 10:09:14.044413 7f6aa8cf9700 3
client.464559 _getxattr(1000037c78b, "security.capability", 0)
= -61
>> -85> 2016-07-05 10:09:14.044423 7f69d7fff700 3
client.464559 ll_lookup 0x7f6acc026050 ruggerod
>> -84> 2016-07-05 10:09:14.044428 7f69d7fff700 3
client.464559 ll_lookup 0x7f6acc026050 ruggerod -> 0
(10000174776)
>> -83> 2016-07-05 10:09:14.044432 7f6ac21fc700 3
client.464559 ll_write 0x7f6a08042560 1000037c78b 12104~34
>> -82> 2016-07-05 10:09:14.044454 7f6ac21fc700 3
client.464559 ll_write 0x7f6a08042560 12104~34 = 34
>> -81> 2016-07-05 10:09:14.044464 7f6aab4fd700 3
client.464559 ll_forget 1000037c78a 1
>> -80> 2016-07-05 10:09:14.044471 7f6aa8cf9700 3
client.464559 ll_forget 1000037c78b 1
>> -79> 2016-07-05 10:09:14.044481 7f6aa96fa700 3
client.464559 ll_write 0x7f6a140d5930 1000037c78a 18496~34
>> -78> 2016-07-05 10:09:14.044497 7f6aa96fa700 3
client.464559 ll_write 0x7f6a140d5930 18496~34 = 34
>> -77> 2016-07-05 10:09:14.044507 7f6aa8cf9700 3
client.464559 ll_getxattr 1000037c78b.head security.capability
size 0
>> -76> 2016-07-05 10:09:14.044513 7f6aa8cf9700 3
client.464559 _getxattr(1000037c78b, "security.capability", 0)
= -61
>> -75> 2016-07-05 10:09:14.044519 7f6aa8cf9700 3
client.464559 ll_forget 1000037c78b 1
>> -74> 2016-07-05 10:09:14.044525 7f6a5f5fe700 3
client.464559 ll_getxattr 1000037c78a.head security.capability
size 0
>> -73> 2016-07-05 10:09:14.044532 7f6a5f5fe700 3
client.464559 _getxattr(1000037c78a, "security.capability", 0)
= -61
>> -72> 2016-07-05 10:09:14.044539 7f6aa8cf9700 3
client.464559 ll_write 0x7f6a08042560 1000037c78b 12138~34
>> -71> 2016-07-05 10:09:14.044558 7f6aa8cf9700 3
client.464559 ll_write 0x7f6a08042560 12138~34 = 34
>> -70> 2016-07-05 10:09:14.044565 7f6a5f5fe700 3
client.464559 ll_forget 1000037c78a 1
>> -69> 2016-07-05 10:09:14.044573 7f6a5ffff700 3
client.464559 ll_write 0x7f6a140d5930 1000037c78a 18530~34
>> -68> 2016-07-05 10:09:14.044594 7f6a5ffff700 3
client.464559 ll_write 0x7f6a140d5930 18530~34 = 34
>> -67> 2016-07-05 10:09:14.044602 7f6ac17fb700 3
client.464559 ll_getxattr 1000037c78b.head security.capability
size 0
>> -66> 2016-07-05 10:09:14.044607 7f6ac17fb700 3
client.464559 _getxattr(1000037c78b, "security.capability", 0)
= -61
>> -65> 2016-07-05 10:09:14.044612 7f6ac17fb700 3
client.464559 ll_forget 1000037c78b 1
>> -64> 2016-07-05 10:09:14.044619 7f6ac17fb700 3
client.464559 ll_getxattr 1000037c78a.head security.capability
size 0
>> -63> 2016-07-05 10:09:14.044623 7f6ac17fb700 3
client.464559 _getxattr(1000037c78a, "security.capability", 0)
= -61
>> -62> 2016-07-05 10:09:14.044629 7f6a5ebfd700 3
client.464559 ll_write 0x7f6a08042560 1000037c78b 12172~34
>> -61> 2016-07-05 10:09:14.044649 7f6a5ebfd700 3
client.464559 ll_write 0x7f6a08042560 12172~34 = 34
>> -60> 2016-07-05 10:09:14.044657 7f6ac17fb700 3
client.464559 ll_forget 1000037c78a 1
>> -59> 2016-07-05 10:09:14.044665 7f6ac21fc700 3
client.464559 ll_write 0x7f6a140d5930 1000037c78a 18564~34
>> -58> 2016-07-05 10:09:14.044684 7f6ac21fc700 3
client.464559 ll_write 0x7f6a140d5930 18564~34 = 34
>> -57> 2016-07-05 10:09:14.044692 7f6aab4fd700 3
client.464559 ll_getxattr 1000037c78b.head security.capability
size 0
>> -56> 2016-07-05 10:09:14.044697 7f6aab4fd700 3
client.464559 _getxattr(1000037c78b, "security.capability", 0)
= -61
>> -55> 2016-07-05 10:09:14.044701 7f6aab4fd700 3
client.464559 ll_forget 1000037c78b 1
>> -54> 2016-07-05 10:09:14.044709 7f6aab4fd700 3
client.464559 ll_write 0x7f6a08042560 1000037c78b 12206~34
>> -53> 2016-07-05 10:09:14.044723 7f6aab4fd700 3
client.464559 ll_write 0x7f6a08042560 12206~34 = 34
>> -52> 2016-07-05 10:09:14.044732 7f6aa96fa700 3
client.464559 ll_getxattr 1000037c78a.head security.capability
size 0
>> -51> 2016-07-05 10:09:14.044738 7f6aa96fa700 3
client.464559 _getxattr(1000037c78a, "security.capability", 0)
= -61
>> -50> 2016-07-05 10:09:14.044745 7f6aa96fa700 3
client.464559 ll_forget 1000037c78a 1
>> -49> 2016-07-05 10:09:14.044749 7f6ac0dfa700 3
client.464559 ll_getxattr 1000037c78b.head security.capability
size 0
>> -48> 2016-07-05 10:09:14.044757 7f6ac0dfa700 3
client.464559 _getxattr(1000037c78b, "security.capability", 0)
= -61
>> -47> 2016-07-05 10:09:14.044764 7f6aa96fa700 3
client.464559 ll_write 0x7f6a140d5930 1000037c78a 18598~34
>> -46> 2016-07-05 10:09:14.044783 7f6aa96fa700 3
client.464559 ll_write 0x7f6a140d5930 18598~34 = 34
>> -45> 2016-07-05 10:09:14.044790 7f6ac0dfa700 3
client.464559 ll_forget 1000037c78b 1
>> -44> 2016-07-05 10:09:14.044798 7f6aa8cf9700 3
client.464559 ll_write 0x7f6a08042560 1000037c78b 12240~34
>> -43> 2016-07-05 10:09:14.044815 7f6aa8cf9700 3
client.464559 ll_write 0x7f6a08042560 12240~34 = 34
>> -42> 2016-07-05 10:09:14.044822 7f6a5f5fe700 3
client.464559 ll_getxattr 1000037c78a.head security.capability
size 0
>> -41> 2016-07-05 10:09:14.044828 7f6a5f5fe700 3
client.464559 _getxattr(1000037c78a, "security.capability", 0)
= -61
>> -40> 2016-07-05 10:09:14.044832 7f6a5f5fe700 3
client.464559 ll_forget 1000037c78a 1
>> -39> 2016-07-05 10:09:14.044841 7f6a5f5fe700 3
client.464559 ll_getxattr 1000037c78b.head security.capability
size 0
>> -38> 2016-07-05 10:09:14.044845 7f6a5f5fe700 3
client.464559 _getxattr(1000037c78b, "security.capability", 0)
= -61
>> -37> 2016-07-05 10:09:14.044853 7f6a5ebfd700 3
client.464559 ll_write 0x7f6a140d5930 1000037c78a 18632~34
>> -36> 2016-07-05 10:09:14.044869 7f6a5ebfd700 3
client.464559 ll_write 0x7f6a140d5930 18632~34 = 34
>> -35> 2016-07-05 10:09:14.044876 7f6a5f5fe700 3
client.464559 ll_forget 1000037c78b 1
>> -34> 2016-07-05 10:09:14.044889 7f6ac17fb700 3
client.464559 ll_write 0x7f6a08042560 1000037c78b 12274~34
>> -33> 2016-07-05 10:09:14.044909 7f6ac17fb700 3
client.464559 ll_write 0x7f6a08042560 12274~34 = 34
>> -32> 2016-07-05 10:09:14.044919 7f6ac21fc700 3
client.464559 ll_getxattr 1000037c78a.head security.capability
size 0
>> -31> 2016-07-05 10:09:14.044924 7f6ac21fc700 3
client.464559 _getxattr(1000037c78a, "security.capability", 0)
= -61
>> -30> 2016-07-05 10:09:14.044932 7f6ac21fc700 3
client.464559 ll_forget 1000037c78a 1
>> -29> 2016-07-05 10:09:14.044936 7f6a5ffff700 3
client.464559 ll_getxattr 1000037c78b.head security.capability
size 0
>> -28> 2016-07-05 10:09:14.044942 7f6a5ffff700 3
client.464559 _getxattr(1000037c78b, "security.capability", 0)
= -61
>> -27> 2016-07-05 10:09:14.044948 7f6ac21fc700 3
client.464559 ll_write 0x7f6a140d5930 1000037c78a 18666~34
>> -26> 2016-07-05 10:09:14.044970 7f6ac21fc700 3
client.464559 ll_write 0x7f6a140d5930 18666~34 = 34
>> -25> 2016-07-05 10:09:14.044978 7f6a5ffff700 3
client.464559 ll_forget 1000037c78b 1
>> -24> 2016-07-05 10:09:14.044987 7f6aab4fd700 3
client.464559 ll_write 0x7f6a08042560 1000037c78b 12308~34
>> -23> 2016-07-05 10:09:14.045003 7f6aab4fd700 3
client.464559 ll_write 0x7f6a08042560 12308~34 = 34
>> -22> 2016-07-05 10:09:14.045012 7f6aa96fa700 3
client.464559 ll_getxattr 1000037c78a.head security.capability
size 0
>> -21> 2016-07-05 10:09:14.045018 7f6aa96fa700 3
client.464559 _getxattr(1000037c78a, "security.capability", 0)
= -61
>> -20> 2016-07-05 10:09:14.045025 7f6aa96fa700 3
client.464559 ll_forget 1000037c78a 1
>> -19> 2016-07-05 10:09:14.045028 7f6ac0dfa700 3
client.464559 ll_getxattr 1000037c78b.head security.capability
size 0
>> -18> 2016-07-05 10:09:14.045031 7f6ac0dfa700 3
client.464559 _getxattr(1000037c78b, "security.capability", 0)
= -61
>> -17> 2016-07-05 10:09:14.045037 7f6aa96fa700 3
client.464559 ll_write 0x7f6a140d5930 1000037c78a 18700~34
>> -16> 2016-07-05 10:09:14.045053 7f6aa96fa700 3
client.464559 ll_write 0x7f6a140d5930 18700~34 = 34
>> -15> 2016-07-05 10:09:14.045059 7f6ac0dfa700 3
client.464559 ll_forget 1000037c78b 1
>> -14> 2016-07-05 10:09:14.045065 7f6aa8cf9700 3
client.464559 ll_write 0x7f6a08042560 1000037c78b 12342~34
>> -13> 2016-07-05 10:09:14.045082 7f6aa8cf9700 3
client.464559 ll_write 0x7f6a08042560 12342~34 = 34
>> -12> 2016-07-05 10:09:14.045090 7f6a5ebfd700 3
client.464559 ll_getxattr 1000037c78a.head security.capability
size 0
>> -11> 2016-07-05 10:09:14.045095 7f6a5ebfd700 3
client.464559 _getxattr(1000037c78a, "security.capability", 0)
= -61
>> -10> 2016-07-05 10:09:14.045101 7f6a5ebfd700 3
client.464559 ll_forget 1000037c78a 1
>> -9> 2016-07-05 10:09:14.045112 7f6a5f5fe700 3
client.464559 ll_getxattr 1000037c78b.head security.capability
size 0
>> -8> 2016-07-05 10:09:14.045117 7f6a5f5fe700 3
client.464559 _getxattr(1000037c78b, "security.capability", 0)
= -61
>> -7> 2016-07-05 10:09:14.045122 7f6ac17fb700 3
client.464559 ll_write 0x7f6a140d5930 1000037c78a 18734~34
>> -6> 2016-07-05 10:09:14.045139 7f6ac17fb700 3
client.464559 ll_write 0x7f6a140d5930 18734~34 = 34
>> -5> 2016-07-05 10:09:14.045131 7f69d7fff700 -1
*** Caught signal (Segmentation fault) **
>> in thread 7f69d7fff700 thread_name:ceph-fuse
>>
>> ceph version 10.2.2
(45107e21c568dd033c2f0a3107dec8f0b0e58374)
>> 1: (()+0x297ef2) [0x7f6aedbecef2]
>> 2: (()+0x3b88c0f7e0) [0x7f6aec64b7e0]
>> 3: (Client::get_root_ino()+0x10) [0x7f6aedaf0330]
>> 4: (CephFuse::Handle::make_fake_ino(inodeno_t,
snapid_t)+0x175) [0x7f6aedaee035]
>> 5: (()+0x199891) [0x7f6aedaee891]
>> 6: (()+0x15b76) [0x7f6aed50db76]
>> 7: (()+0x12aa9) [0x7f6aed50aaa9]
>> 8: (()+0x3b88c07aa1) [0x7f6aec643aa1]
>> 9: (clone()+0x6d) [0x7f6aeb8d193d]
>> NOTE: a copy of the executable, or `objdump -rdS
<executable>` is needed to interpret this.
>>
>> -4> 2016-07-05 10:09:14.045146 7f6a5f5fe700 3
client.464559 ll_forget 1000037c78b 1
>> -3> 2016-07-05 10:09:14.045152 7f6ac21fc700 3
client.464559 ll_write 0x7f6a08042560 1000037c78b 12376~34
>> -2> 2016-07-05 10:09:14.045165 7f6ac21fc700 3
client.464559 ll_write 0x7f6a08042560 12376~34 = 34
>> -1> 2016-07-05 10:09:14.045171 7f6a5ffff700 3
client.464559 ll_getxattr 1000037c78a.head security.capability
size 0
>> 0> 2016-07-05 10:09:14.045175 7f6a5ffff700 3
client.464559 _getxattr(1000037c78a, "security.capability", 0)
= -61
>> --- logging levels ---
>> 0/ 5 none
>> 0/ 1 lockdep
>> 0/ 1 context
>> 1/ 1 crush
>> 1/ 5 mds
>> 1/ 5 mds_balancer
>> 1/ 5 mds_locker
>> 1/ 5 mds_log
>> 1/ 5 mds_log_expire
>> 1/ 5 mds_migrator
>> 0/ 1 buffer
>> 0/ 1 timer
>> 0/ 1 filer
>> 0/ 1 striper
>> 0/ 1 objecter
>> 0/ 5 rados
>> 0/ 5 rbd
>> 0/ 5 rbd_mirror
>> 0/ 5 rbd_replay
>> 0/ 5 journaler
>> 0/ 5 objectcacher
>> 0/ 5 client
>> 0/ 5 osd
>> 0/ 5 optracker
>> 0/ 5 objclass
>> 1/ 3 filestore
>> 1/ 3 journal
>> 0/ 5 ms
>> 1/ 5 mon
>> 0/10 monc
>> 1/ 5 paxos
>> 0/ 5 tp
>> 1/ 5 auth
>> 1/ 5 crypto
>> 1/ 1 finisher
>> 1/ 5 heartbeatmap
>> 1/ 5 perfcounter
>> 1/ 5 rgw
>> 1/10 civetweb
>> 1/ 5 javaclient
>> 1/ 5 asok
>> 1/ 1 throttle
>> 0/ 0 refs
>> 1/ 5 xio
>> 1/ 5 compressor
>> 1/ 5 newstore
>> 1/ 5 bluestore
>> 1/ 5 bluefs
>> 1/ 3 bdev
>> 1/ 5 kstore
>> 4/ 5 rocksdb
>> 4/ 5 leveldb
>> 1/ 5 kinetic
>> 1/ 5 fuse
>> -2/-2 (syslog threshold)
>> 99/99 (stderr threshold)
>> max_recent 10000
>> max_new 1000
>> log_file
>> --- end dump of recent events ---
>
>
> Note sure if this is relevant but we successfully
compiled ceph-fuse ourselfs (both 9.2.0 and 10.2.2) in an sl6
machine with with gcc 4.8.4, python 2.7.6, boost 1.53.0-25 and
fuse 2.9.7. We do this because of the nature of the research
performed by our scientists where there is (still) the
requirement to provide cephfs under sl6.
>
> Any help in understanding why this is happening would be
very appreciated.
>
> Cheers
>
> Goncalo
>
> --
> Goncalo Borges
> Research Computing
> ARC Centre of Excellence for Particle Physics at the
Terascale
> School of Physics A28 | University of Sydney, NSW 2006
> T: +61 2 93511937
>
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
--
Goncalo Borges
Research Computing
ARC Centre of Excellence for Particle Physics at the Terascale
School of Physics A28 | University of Sydney, NSW 2006
T: +61 2 93511937
--
Goncalo Borges
Research Computing
ARC Centre of Excellence for Particle Physics at the Terascale
School of Physics A28 | University of Sydney, NSW 2006
T: +61 2 93511937
|
