VM pauses are happening because quorum is not met. i.e. at least two of the brick in the setup are not connected to the client. So it is becoming Read-only filesystem.
PranithOn Wed, May 11, 2016 at 2:53 PM, maokx@xxxxxxxx <maokx@xxxxxxxx> wrote:
Hi all:can glusterfs replicate run in 100M ethernet?gluster:[root@server187 qemu]# rpm -aq|grep gluster
glusterfs-3.6.2-1.el7.x86_64
glusterfs-extra-xlators-3.6.2-1.el7.x86_64
svserver-gluster-4.16.14-0.el7.centos.noarch
glusterfs-api-3.6.2-1.el7.x86_64
glusterfs-fuse-3.6.2-1.el7.x86_64
glusterfs-cli-3.6.2-1.el7.x86_64
glusterfs-server-3.6.2-1.el7.x86_64
glusterfs-rdma-3.6.2-1.el7.x86_64
glusterfs-libs-3.6.2-1.el7.x86_64
glusterfs-devel-3.6.2-1.el7.x86_64
[root@server187 qemu]#1、why vm on running server has paused?Is 100M ethernet lead to?I use replicate 3 glusterfs,one server down,two server still up.In this case,one vm on server running is become pause .detail:[2016-05-05 08:51:22.942682] I [rpc-clnt.c:1761:rpc_clnt_reconfig] 0-pool01-client-2: changing port to 49152 (from 0)
[2016-05-05 08:51:22.947960] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-05 08:51:22.948196] I [client-handshake.c:1413:select_server_supported_programs] 0-pool01-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-05-05 08:51:22.948346] I [client-handshake.c:1200:client_setvolume_cbk] 0-pool01-client-2: Connected to pool01-client-2, attached to remote volume '/data/data01'.
[2016-05-05 08:51:22.948358] I [client-handshake.c:1210:client_setvolume_cbk] 0-pool01-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2016-05-05 08:51:22.948365] I [client-handshake.c:1052:client_post_handshake] 0-pool01-client-2: 3 fds open - Delaying child_up until they are re-opened
[2016-05-05 08:51:22.948612] I [client-handshake.c:674:client_child_up_reopen_done] 0-pool01-client-2: last fd open'd/lock-self-heal'd - notifying CHILD-UP
[2016-05-05 08:51:22.948646] I [MSGID: 108002] [afr-common.c:3631:afr_notify] 0-pool01-replicate-0: Client-quorum is met
[2016-05-05 08:51:22.948677] I [client-handshake.c:188:client_set_lk_version_cbk] 0-pool01-client-2: Server lk version = 1
[2016-05-05 08:51:23.945232] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-05 08:51:26.948173] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-05 08:51:29.397014] I [afr-common.c:1484:afr_local_discovery_cbk] 0-pool01-replicate-0: selecting local read_child pool01-client-2
[2016-05-05 08:51:29.951076] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-05 08:51:32.954015] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-05 08:51:35.956908] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-05 08:51:36.238878] W [fuse-bridge.c:1261:fuse_err_cbk] 0-glusterfs-fuse: 407688: FSYNC() ERR => -1 (Read-only file system)
[2016-05-05 08:51:36.280494] W [fuse-bridge.c:1261:fuse_err_cbk] 0-glusterfs-fuse: 407689: FSYNC() ERR => -1 (Read-only file system)
[2016-05-05 08:51:38.959928] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-05 08:51:41.962890] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-05 08:51:44.965974] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-05 08:51:47.968896] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-05 08:51:50.971802] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-05 08:51:53.974781] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier2、why Response so slowly?Is 100M ethernet lead to?I use replicate 3 glusterfs,one server down,two server still up.In this case,one brick on server running is become readonly because of Client-quorum is not met.But after about three hours,that brick become better automatic.why Response so slowly?Is 100M ethernet lead to?Detail:[2016-05-04 06:49:47.924382] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 06:50:47.262468] W [socket.c:611:__socket_rwv] 0-pool01-client-0: readv on 192.168.101.185:49152 failed (No data available)
[2016-05-04 06:50:47.262530] I [client.c:2215:client_rpc_notify] 0-pool01-client-0: disconnected from pool01-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2016-05-04 06:50:47.262546] W [MSGID: 108001] [afr-common.c:3635:afr_notify] 0-pool01-replicate-0: Client-quorum is not met
[2016-05-04 06:50:54.310210] I [afr-common.c:1484:afr_local_discovery_cbk] 0-pool01-replicate-0: selecting local read_child pool01-client-2
[2016-05-04 06:50:57.932864] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 06:50:57.933201] E [socket.c:2267:socket_connect_finish] 0-pool01-client-0: connection to 192.168.101.185:24007 failed (Connection refused)
[2016-05-04 06:51:00.935714] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 06:51:03.349865] W [fuse-bridge.c:2270:fuse_writev_cbk] 0-glusterfs-fuse: 37106: WRITE => -1 (Read-only file system)
[2016-05-04 06:51:03.851940] W [fuse-bridge.c:2270:fuse_writev_cbk] 0-glusterfs-fuse: 37117: WRITE => -1 (Read-only file system)
[2016-05-04 06:51:03.938817] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 06:51:04.354396] W [fuse-bridge.c:2270:fuse_writev_cbk] 0-glusterfs-fuse: 37142: WRITE => -1 (Read-only file system)...........[2016-05-04 09:43:20.018833] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:43:21.775009] W [fuse-bridge.c:1079:fuse_setattr_cbk] 0-glusterfs-fuse: 51883: SETATTR() /__DIRECT_IO_TEST__ => -1 (Read-only file system)
[2016-05-04 09:43:23.021665] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:43:26.024536] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:43:29.027486] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:43:31.791565] W [fuse-bridge.c:1079:fuse_setattr_cbk] 0-glusterfs-fuse: 51903: SETATTR() /__DIRECT_IO_TEST__ => -1 (Read-only file system)
[2016-05-04 09:43:32.030394] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:43:35.033372] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:43:38.036299] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:43:41.039224] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:43:41.807885] W [fuse-bridge.c:1079:fuse_setattr_cbk] 0-glusterfs-fuse: 51915: SETATTR() /__DIRECT_IO_TEST__ => -1 (Read-only file system)
[2016-05-04 09:43:44.042165] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:43:47.045042] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:43:50.047917] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:43:51.824073] W [fuse-bridge.c:1079:fuse_setattr_cbk] 0-glusterfs-fuse: 51927: SETATTR() /__DIRECT_IO_TEST__ => -1 (Read-only file system)
[2016-05-04 09:43:53.050793] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:43:56.053775] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:43:59.056653] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:01.839749] W [fuse-bridge.c:1079:fuse_setattr_cbk] 0-glusterfs-fuse: 51939: SETATTR() /__DIRECT_IO_TEST__ => -1 (Read-only file system)
[2016-05-04 09:44:02.059552] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:07.062665] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:07.063593] I [rpc-clnt.c:1761:rpc_clnt_reconfig] 0-pool01-client-1: changing port to 49152 (from 0)
[2016-05-04 09:44:07.066088] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:07.066641] I [client-handshake.c:1413:select_server_supported_programs] 0-pool01-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-05-04 09:44:07.067234] I [client-handshake.c:1200:client_setvolume_cbk] 0-pool01-client-1: Connected to pool01-client-1, attached to remote volume '/data/data01'.
[2016-05-04 09:44:07.067247] I [client-handshake.c:1210:client_setvolume_cbk] 0-pool01-client-1: Server and Client lk-version numbers are not same, reopening the fds
[2016-05-04 09:44:07.067253] I [client-handshake.c:1052:client_post_handshake] 0-pool01-client-1: 2 fds open - Delaying child_up until they are re-opened
[2016-05-04 09:44:07.067778] I [client-handshake.c:674:client_child_up_reopen_done] 0-pool01-client-1: last fd open'd/lock-self-heal'd - notifying CHILD-UP
[2016-05-04 09:44:07.067804] I [MSGID: 108002] [afr-common.c:3631:afr_notify] 0-pool01-replicate-0: Client-quorum is met
[2016-05-04 09:44:07.068272] I [client-handshake.c:188:client_set_lk_version_cbk] 0-pool01-client-1: Server lk version = 1
[2016-05-04 09:44:08.065350] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:11.068215] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:11.842992] I [afr-common.c:1484:afr_local_discovery_cbk] 0-pool01-replicate-0: selecting local read_child pool01-client-2
[2016-05-04 09:44:11.865405] W [client-rpc-fops.c:1210:client3_3_removexattr_cbk] 0-pool01-client-2: remote operation failed: No data available
[2016-05-04 09:44:11.865633] W [client-rpc-fops.c:1210:client3_3_removexattr_cbk] 0-pool01-client-1: remote operation failed: No data available
[2016-05-04 09:44:11.866102] W [fuse-bridge.c:1261:fuse_err_cbk] 0-glusterfs-fuse: 51953: REMOVEXATTR() /__DIRECT_IO_TEST__ => -1 (No data available)
[2016-05-04 09:44:14.071042] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:17.073986] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:20.076992] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:23.079942] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:26.082841] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:29.085667] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:32.088532] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:35.091452] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:38.094352] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:41.097332] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:44.100262] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:47.103083] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:50.105942] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:53.108953] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:56.111865] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:44:59.114713] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:45:02.117608] W [common-utils.c:2621:gf_ports_reserved] 0-glusterfs-socket: 54321 is not a valid port identifier
[2016-05-04 09:45:04.595414] W [client-rpc-fops.c:1210:client3_3_removexattr_cbk] 0-pool01-client-2: remote operation failed: No data available
[2016-05-04 09:45:04.595634] W [client-rpc-fops.c:1210:client3_3_removexattr_cbk] 0-pool01-client-1: remote operation failed: No data available
[2016-05-04 09:45:04.595966] W [fuse-bridge.c:1261:fuse_err_cbk] 0-glusterfs-fuse: 52091: REMOVEXATTR() /__DIRECT_IO_TEST__ => -1 (No data available)
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
