Hi Chen,
Could you please send us following logs-
1 - brick logs - under /var/log/messages/brick/
2 - mount logs
Also some information like what kind of IO was happening (read,write, unlink, rename on different mount) to understand this issue in a better way.
---
Ashish
----- Original Message -----
From: "陈陈" <chenchen@xxxxxxxxxxxxxxxx>
To: gluster-users@xxxxxxxxxxx
Sent: Friday, March 25, 2016 8:59:04 AM
Subject: Need some help on Mismatching xdata / Failed combine iatt / Too many fd
Hi Everyone,
I have a "2 x (4 + 2) = 12 Distributed-Disperse" volume. After upgraded
to 3.7.8 I noticed the volume is frequently out of service. The
glustershd.log is flooded by:
[ec-combine.c:866:ec_combine_check] 0-mainvol-disperse-1: Mismatching
xdata in answers of 'LOOKUP'"
[ec-common.c:116:ec_check_status] 0-mainvol-disperse-1: Operation failed
on some subvolumes (up=3F, mask=3F, remaining=0, good=1E, bad=21)
[ec-common.c:71:ec_heal_report] 0-mainvol-disperse-1: Heal failed
[Invalid argument]
[ec-combine.c:206:ec_iatt_combine] 0-mainvol-disperse-0: Failed to
combine iatt (inode: xxx, links: 1-1, uid: 1000-1000, gid: 1000-1000,
rdev: 0-0, size: xxx-xxx, mode: 100600-100600)
in normal working state, and sometimes 1000+ lines of:
[client-rpc-fops.c:466:client3_3_open_cbk] 0-mainvol-client-7: remote
operation failed. Path: <gfid:xxxx> (xxxx) [Too many open files]
and the brick went offline. "top open" showed "Max open fds: 899195".
Can anyone suggest me what happened, and what should I do? I was trying
to deal with the terrible IOPS problem but things got even worse.
Each Server has 2 x E5-2630v3 (32threads/server), 32GB RAM. Additional
infos are in the attachements. Many thanks.
Sincerely yours,
Chen
--
Chen Chen
Shanghai SmartQuerier Biotechnology Co., Ltd.
Add: Room 410, 781 Cai Lun Road, China (Shanghai) Pilot Free Trade Zone
Shanghai 201203, P. R. China
Mob: +86 15221885893
Email: chenchen@xxxxxxxxxxxxxxxx
Web: www.smartquerier.com
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users
