|
Hi, I found the coredump file, but it's a 15Mo file (zipped), I can't post it on this mailling list. Here is some parts of the repport : ProblemType: Crash... ProcStatus:... Yann Le 28/06/2016 09:31, Anoop C S a
écrit :
On Mon, 2016-06-27 at 15:05 +0200, Yann LEMARIE wrote:@Anoop, Where can I find the coredump file ?You will get hints about the crash from entries inside /var/log/messages(for example pid of the process, location of coredump etc).The crash occurs 2 times last 7 days, each time a sunday morning with no reason, no increase of traffic or something like this, the volume was mounted since 15 days. The bricks are used as a CDN like, distributting small images and css files with a nginx https service (with a load balancer and 2 EC2), on a sunday morning there is not a lot of activity ...>From the very minimal back trace that we have from brick logs I would assume that a truncate operation was being handled by trash translator and it crashed.Volume infos:root@nfs05 /var/log/glusterfs # gluster volume info cdn Volume Name: cdn Type: Replicate Volume ID: c53b9bae-5e12-4f13-8217-53d8c96c302c Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: nfs05:/srv/cdn Brick2: nfs06:/srv/cdn Options Reconfigured: performance.readdir-ahead: on features.trash: on features.trash-max-filesize: 20MBI don't know if there is a link with this crash problem, but I have another problem with my 2 servers that make GluserFS's clients disconnected (from another volume) :Jun 24 02:28:04 nfs05 kernel: [2039468.818617] xen_netfront: xennet: skb rides the rocket: 19 slots Jun 24 02:28:11 nfs05 kernel: [2039475.744086] net_ratelimit: 66 callbacks suppressedIt seem to be a network interface problem : https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1317811 Yann Le 27/06/2016 12:59, Anoop C S a écrit :On Mon, 2016-06-27 at 09:47 +0200, Yann LEMARIE wrote:Hi, I'm using GlusterFS since many years and never see this problem, but this is the second time in one week ... I have 3 volumes with 2 bricks and 1 volume crash with no reason,Did you observe the crash while mounting the volume? Or can you be more specific on what were you doing just before you saw the crash? Can you please share the output of `gluster volume info <VOLNAME>`?I just have to stop/start the volume to make it up again. The only logs I can find are in syslog :Jun 26 11:27:44 nfs05 srv-cdn[7879]: pending frames: Jun 26 11:27:44 nfs05 srv-cdn[7879]: frame : type(0) op(10) Jun 26 11:27:44 nfs05 srv-cdn[7879]: patchset: git://git.gluster.com/glusterfs.git Jun 26 11:27:44 nfs05 srv-cdn[7879]: signal received: 11 Jun 26 11:27:44 nfs05 srv-cdn[7879]: time of crash: Jun 26 11:27:44 nfs05 srv-cdn[7879]: 2016-06-26 09:27:44 Jun 26 11:27:44 nfs05 srv-cdn[7879]: configuration details: Jun 26 11:27:44 nfs05 srv-cdn[7879]: argp 1 Jun 26 11:27:44 nfs05 srv-cdn[7879]: backtrace 1 Jun 26 11:27:44 nfs05 srv-cdn[7879]: dlfcn 1 Jun 26 11:27:44 nfs05 srv-cdn[7879]: libpthread 1 Jun 26 11:27:44 nfs05 srv-cdn[7879]: llistxattr 1 Jun 26 11:27:44 nfs05 srv-cdn[7879]: setfsid 1 Jun 26 11:27:44 nfs05 srv-cdn[7879]: spinlock 1 Jun 26 11:27:44 nfs05 srv-cdn[7879]: epoll.h 1 Jun 26 11:27:44 nfs05 srv-cdn[7879]: xattr.h 1 Jun 26 11:27:44 nfs05 srv-cdn[7879]: st_atim.tv_nsec 1 Jun 26 11:27:44 nfs05 srv-cdn[7879]: package-string: glusterfs 3.7.11 Jun 26 11:27:44 nfs05 srv-cdn[7879]: --------- Jun 26 11:27:44 nfs06 srv-cdn[1787]: pending frames: Jun 26 11:27:44 nfs06 srv-cdn[1787]: frame : type(0) op(10) Jun 26 11:27:44 nfs06 srv-cdn[1787]: patchset: git://git.gluster.com/glusterfs.git Jun 26 11:27:44 nfs06 srv-cdn[1787]: signal received: 11 Jun 26 11:27:44 nfs06 srv-cdn[1787]: time of crash: Jun 26 11:27:44 nfs06 srv-cdn[1787]: 2016-06-26 09:27:44 Jun 26 11:27:44 nfs06 srv-cdn[1787]: configuration details: Jun 26 11:27:44 nfs06 srv-cdn[1787]: argp 1 Jun 26 11:27:44 nfs06 srv-cdn[1787]: backtrace 1 Jun 26 11:27:44 nfs06 srv-cdn[1787]: dlfcn 1 Jun 26 11:27:44 nfs06 srv-cdn[1787]: libpthread 1 Jun 26 11:27:44 nfs06 srv-cdn[1787]: llistxattr 1 Jun 26 11:27:44 nfs06 srv-cdn[1787]: setfsid 1 Jun 26 11:27:44 nfs06 srv-cdn[1787]: spinlock 1 Jun 26 11:27:44 nfs06 srv-cdn[1787]: epoll.h 1 Jun 26 11:27:44 nfs06 srv-cdn[1787]: xattr.h 1 Jun 26 11:27:44 nfs06 srv-cdn[1787]: st_atim.tv_nsec 1 Jun 26 11:27:44 nfs06 srv-cdn[1787]: package-string: glusterfs 3.7.11 Jun 26 11:27:44 nfs06 srv-cdn[1787]: ---------Thanks for your help Regards -- Yann Lemarié iRaiser - Support Technique ylemarie@xxxxxxxxxx _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users-- Yann Lemarié iRaiser - Support Technique ylemarie@xxxxxxxxxx _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users --
|
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
