problem: when self-heal is running with many files, it make cpu high load and gluster client cannot write new file and read file.
as described at: http://www.gluster.org/pipermail/gluster-users/2015-November/024228.html
from following tests, I realize that: when close tcp connections between glusterfs and glusterfsd in gluster server,it can reduce cpu load and not affect gluster client to write and read file.
========================= config begin ==================================
gluster 3.5.1
#gluster volume status
Status of volume: tr5
Gluster process Port Online Pid
------------------------------------------------------------------------------
Brick 192.168.5.252:/home/gluster351/r15 49157 Y 11899
Brick 192.168.5.76:/home/gluster/r15 49152 Y 26692
NFS Server on localhost N/A N N/A
Self-heal Daemon on localhost N/A Y 11918
NFS Server on 192.168.5.76 N/A N N/A
Self-heal Daemon on 192.168.5.76 N/A Y 26705
Task Status of Volume tr5
------------------------------------------------------------------------------
There are no active volume tasks
gluster client: 192.168.16.207:1019
# mount
192.168.5.252:/tr5 on /home/mariadb/data/tr5 type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
========================= config end ==================================
========================= test 1 begin =================================
at 192.168.5.252,run:
iptables -A INPUT -p tcp --dport 49157 -j DROP -s 192.168.5.252
iptables -A INPUT -p tcp --dport 49157 -j DROP -s 192.168.5.76
at 192.168.5.76,run:
iptables -A INPUT -p tcp --dport 49152 -j DROP -s 192.168.5.252
iptables -A INPUT -p tcp --dport 49152 -j DROP -s 192.168.5.76
gluster client can write and read files as normal.
========================= test 1 end ==================================
========================= test 2 begin ==================================
1: at 192.168.5.76, shutdown process: glusterd, glusterfsd, glusterfs.
2: at gluster client, copy 10k files to gluster server.
3: at 192.168.5.252, there are 10k+ link file under directory /home/gluster351/r15/.glusterfs/indices/xattrop.
4: at 192.168.5.76, start gluster service. then self-heal begins.
5: during self-heal, at 192.168.5.252 , glusterfs %CPU 5.7 , glusterfsd %CPU 7.0
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
11918 root 20 0 321m 36m 2816 S 7.0 0.5 1:28.07 glusterfs
11899 root 20 0 824m 42m 3164 S 5.7 0.5 20:43.39 glusterfsd
6: during self-heal, at 192.168.5.76 , glusterfs %CPU 0 , glusterfsd %CPU 6.0
iptables -A INPUT -p tcp --dport 49152 -j DROP -s 192.168.5.252
iptables -A INPUT -p tcp --dport 49152 -j DROP -s 192.168.5.76
after run iptables, at 192.168.5.76 and 192.168.5.252, both glusterfs %CPU 0 , glusterfsd %CPU 0
8: at 192.168.5.252, there are 7000+ link file under directory /home/gluster351/r15/.glusterfs/indices/xattrop.
9: gluster client can write and read files as normal.
========================= test 2 begin ==================================
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
