Re: Problem with SSL/TLS encryption on Gluster 4.0 & 4.1

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 

Hello Andrei,

I did some research. But I didn't find a bug report for this issue. If you want you can create such a report here:
https://bugzilla.redhat.com/enter_bug.cgi?product=GlusterFS

It is the recommended way in GlusterFS to report a bug, github issues are for new features

Regards
David

2018-07-13 11:44 GMT+02:00 Havriliuc Andrei <andrei@xxxxxxxxxxxxx>:

Hello David,


Yeah. I also disabled SSL/TLS for management and communication for Gluster. The problem is that I will have traffic over public WAN for a geo replication instance and I cannot go forward until this is solved.


Was there a bug filed about this? I've looked at the issues on the Glusterfs github page but I didn't find anything related to this:


https://github.com/gluster/glusterfs/issues


Regards,

Andrei


On 7/12/18 4:03 PM, David Spisla wrote:
Hello Andrei,

I am also using Gluster 4.1 on CentOS and I have the same problem. I tested it with a volume which had no network encryption and one with network encryption. You are not the only one.
It seems to be a bug. At the moment there is no other choice to disable client.ssl and sever.ssl on a volume to have stable I/O.

Regards
David


2018-07-12 10:45 GMT+02:00 Havriliuc Andrei <andrei@xxxxxxxxxxxxx>:
Hello,

I am doing some tests with GlusterFS 4.0 and I can't seem to solve some SSL/TLS issues. I am trying to set up a 2 node replicated gluster volume with SSL/TLS. For this setup, I use 3 KVM VMs (2 storage nodes + 1 client node). For the networking part, I use a dedicated private LAN for the KVM VMs. Each VM is able to ping the other, so there's no problem with the connectivity.

To try to make the procedure I used as clear as possible, I will put all commands in chronological order:



=====================================================

1. First, I update the systems, install ntp and then reboot:

yum update

yum install ntp

systemctl status ntpd
systemctl start ntpd
systemctl enable ntpd
systemctl status ntpd


=====================================================


2. I use a separate partition within the two VM storage nodes. Each of the two nodes see the separate partition as /dev/sdb. After creating a thinly provisioned LV on each of the two nodes, I create an XFS filesystem on them:


pvcreate /dev/sdb1

vgcreate -s 32M vg_glusterfs /dev/sdb1

lvcreate -L 20G --thinpool glusterfs_thin_pool vg_glusterfs

lvcreate -V 15G --thin -n glusterfs_thin_vol1 vg_glusterfs/glusterfs_thin_pool

mkfs.xfs -i size=512 /dev/vg_glusterfs/glusterfs_thin_vol1


=====================================================


3. On each node, I configured the brick data partition and added the following to /etc/fstab:

mkdir /data

echo "/dev/vg_glusterfs/glusterfs_thin_vol1 /data xfs defaults 1 2" >> /etc/fstab

mount -a


=====================================================

4. After mounting the volume, I see the following in df -Th, which is correct:

[root@gluster1 brick1]# df -Th
Filesystem                                   Type      Size  Used Avail Use% Mounted on
/dev/sda1                                    ext4       46G  1.4G 42G   4% /
devtmpfs                                     devtmpfs  3.9G     0 3.9G   0% /dev
tmpfs                                        tmpfs     3.9G     0 3.9G   0% /dev/shm
tmpfs                                        tmpfs     3.9G  8.6M 3.9G   1% /run
tmpfs                                        tmpfs     3.9G     0 3.9G   0% /sys/fs/cgroup
tmpfs                                        tmpfs     783M     0 783M   0% /run/user/0
/dev/mapper/vg_glusterfs-glusterfs_thin_vol1 xfs        15G   34M 15G   1% /data


=====================================================

5. Create specific volume dirs on all storage nodes:


mkdir -pv /data/glusterfs/${HOSTNAME%%.*}/vol01


=====================================================

6. Add entries in /etc/hosts:

vim /etc/hosts

192.168.10.233    gluster1
192.168.10.234    gluster2
192.168.10.237    gluster-client

=====================================================

7. Install gluster from the CentOS SIG:


yum search centos-release-gluster
yum install centos-release-gluster40


yum install glusterfs-server

=====================================================

8. Set up TLS/SSL encryption on all nodes and clients (gluster1, gluster2, gluster-client):

openssl genrsa -out /etc/ssl/glusterfs.key 2048

In gluster1 node:
openssl req -new -x509 -key /etc/ssl/glusterfs.key -subj "/CN=gluster1" -out /etc/ssl/glusterfs.pem
In gluster2 node:
openssl req -new -x509 -key /etc/ssl/glusterfs.key -subj "/CN=gluster2" -out /etc/ssl/glusterfs.pem
In gluster-client node:
openssl req -new -x509 -key /etc/ssl/glusterfs.key -subj "/CN=gluster-client" -out /etc/ssl/glusterfs.pem


=====================================================

9. On another box, I concatenate all of the .pem certificates into a .ca file.

Bring all .pem files locally:

scp gluster1:/etc/ssl/glusterfs.pem gluster01.pem
scp gluster2:/etc/ssl/glusterfs.pem gluster02.pem
scp gluster-client:/etc/ssl/glusterfs.pem gluster-client.pem

For storage nodes, I concatenate all .pem certificates (including the client's .pem):

cat gluster01.pem gluster02.pem gluster-client.pem > glusterfs-nodes.ca

For server clients:

cat gluster01.pem gluster02.pem > glusterfs-client.ca


=====================================================


10. Put glusterfs-nodes.ca file on all the storage nodes (this includes storage nodes .pem + client's .pem):

scp glusterfs-nodes.ca gluster1:/etc/ssl/glusterfs.ca
scp glusterfs-nodes.ca gluster2:/etc/ssl/glusterfs.ca

Put glusterfs-client.ca file on all the storage nodes (this includes only storage nodes .pem):

scp glusterfs-client.ca gluster-client:/etc/ssl/glusterfs.ca


=====================================================


11. Enable management encryption on each node (command ran on gluster1, gluster2, gluster-client):


touch /var/lib/glusterd/secure-access


=====================================================


12. Start, enable and check status of glusterd on gluster1 and gluster2:

systemctl start glusterd
systemctl enable glusterd
systemctl status glusterd

=====================================================

13. Configure trusted storage pool (TSP):

>From gluster1:


gluster peer status


gluster peer probe gluster2

=====================================================

14. Create the replicated gluster volume but don't start it:


gluster volume create vol01 replica 2 transport tcp gluster1:/data/glusterfs/gluster1/vol01/brick1 gluster2:/data/glusterfs/gluster2/vol01/brick1

=====================================================


15. Setup SSL/TLS access to the volume:

gluster volume set vol01 auth.ssl-allow 'gluster01,gluster02,gluster-client'


gluster volume set vol01 client.ssl on
gluster volume set vol01 server.ssl on

gluster volume set vol01 network.ping-timeout "5"

gluster volume start vol01


=====================================================


16. Mount the volume on gluster-client:

mount -t glusterfs gluster1:/vol01 /mnt


After mounting the volume, df -h has the correct output:

[root@gluster-client mnt]# df -h
Filesystem       Size  Used Avail Use% Mounted on
/dev/sda1         48G   36G  9.3G  80% /
devtmpfs         1.9G     0  1.9G   0% /dev
tmpfs            1.9G     0  1.9G   0% /dev/shm
tmpfs            1.9G  8.5M  1.9G   1% /run
tmpfs            1.9G     0  1.9G   0% /sys/fs/cgroup
tmpfs            379M     0  379M   0% /run/user/0
gluster1:/vol01   15G  187M   15G   2% /mnt


=====================================================

17. If I try to copy an archive, it throws an error after transferring part of the file and the /mnt folder becomes inaccessible for a while:

[root@gluster-client mnt]# cp /root/mybee.tar.gz /mnt/
cp: error writing ‘/mnt/mybee.tar.gz’: Transport endpoint is not connected
cp: failed to extend ‘/mnt/mybee.tar.gz’: Transport endpoint is not connected
cp: failed to close ‘/mnt/mybee.tar.gz’: Transport endpoint is not connected
[root@gluster-client mnt]# ll /mnt
ls: cannot open directory .: Transport endpoint is not connected


=====================================================

This is the log from gluster-client for the mnt mountpoint (mnt.log):


[2018-06-29 07:52:03.869382] I [MSGID: 100030] [glusterfsd.c:2625:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 4.0.2 (args: /usr/sbin/glusterfs --process-name fuse --volfile-server=gluster1 --volfile-id=/vol01 /mnt)
[2018-06-29 07:52:03.878709] I [socket.c:4470:socket_init] 0-glusterfs: SSL support on the I/O path is ENABLED
[2018-06-29 07:52:03.878782] I [socket.c:4473:socket_init] 0-glusterfs: SSL support for glusterd is ENABLED
[2018-06-29 07:52:03.878795] I [socket.c:4490:socket_init] 0-glusterfs: using private polling thread
[2018-06-29 07:52:03.879108] E [socket.c:4541:socket_init] 0-glusterfs: failed to open /etc/ssl/dhparam.pem, DH ciphers are disabled
[2018-06-29 07:52:03.885718] I [MSGID: 101190] [event-epoll.c:609:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-06-29 07:52:03.901983] I [MSGID: 101190] [event-epoll.c:609:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2018-06-29 07:52:03.902092] I [socket.c:4470:socket_init] 0-vol01-client-1: SSL support on the I/O path is ENABLED
[2018-06-29 07:52:03.902117] I [socket.c:4473:socket_init] 0-vol01-client-1: SSL support for glusterd is ENABLED
[2018-06-29 07:52:03.902142] I [socket.c:4490:socket_init] 0-vol01-client-1: using private polling thread
[2018-06-29 07:52:03.902266] E [socket.c:4541:socket_init] 0-vol01-client-1: failed to open /etc/ssl/dhparam.pem, DH ciphers are disabled
[2018-06-29 07:52:03.902977] I [socket.c:4470:socket_init] 0-vol01-client-0: SSL support on the I/O path is ENABLED
[2018-06-29 07:52:03.903018] I [socket.c:4473:socket_init] 0-vol01-client-0: SSL support for glusterd is ENABLED
[2018-06-29 07:52:03.903028] I [socket.c:4490:socket_init] 0-vol01-client-0: using private polling thread
[2018-06-29 07:52:03.903161] E [socket.c:4541:socket_init] 0-vol01-client-0: failed to open /etc/ssl/dhparam.pem, DH ciphers are disabled
[2018-06-29 07:52:03.903756] I [MSGID: 114020] [client.c:2300:notify] 0-vol01-client-0: parent translators are ready, attempting connect on transport
[2018-06-29 07:52:03.908432] I [MSGID: 114020] [client.c:2300:notify] 0-vol01-client-1: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
  1: volume vol01-client-0
  2:     type protocol/client
  3:     option ping-timeout 5
  4:     option remote-host gluster1
  5:     option remote-subvolume /data/glusterfs/gluster1/vol01/brick1
  6:     option transport-type socket
  7:     option transport.address-family inet
  8:     option transport.socket.ssl-enabled on
  9:     option transport.tcp-user-timeout 0
 10:     option transport.socket.keepalive-time 20
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-count 9
 13:     option send-gids true
 14: end-volume
 15:
 16: volume vol01-client-1
 17:     type protocol/client
 18:     option ping-timeout 5
 19:     option remote-host gluster2
 20:     option remote-subvolume /data/glusterfs/gluster2/vol01/brick1
 21:     option transport-type socket
 22:     option transport.address-family inet
 23:     option transport.socket.ssl-enabled on
 24:     option transport.tcp-user-timeout 0
 25:     option transport.socket.keepalive-time 20
 26:     option transport.socket.keepalive-interval 2
 27:     option transport.socket.keepalive-count 9
 28:     option send-gids true
 29: end-volume
 30:
 31: volume vol01-replicate-0
 32:     type cluster/replicate
 33:     option afr-pending-xattr vol01-client-0,vol01-client-1
 34:     option use-compound-fops off
 35:     subvolumes vol01-client-0 vol01-client-1
 36: end-volume
 37:
 38: volume vol01-dht
 39:     type cluster/distribute
 40:     option lock-migration off
 41:     option force-migration off
 42:     subvolumes vol01-replicate-0
 43: end-volume
 44:
 45: volume vol01-write-behind
 46:     type performance/write-behind
 47:     subvolumes vol01-dht
 48: end-volume
 49:
 50: volume vol01-read-ahead
 51:     type performance/read-ahead
 52:     subvolumes vol01-write-behind
 53: end-volume
 54:
 55: volume vol01-readdir-ahead
 56:     type performance/readdir-ahead
 57:     option parallel-readdir off
 58:     option rda-request-size 131072
 59:     option rda-cache-limit 10MB
 60:     subvolumes vol01-read-ahead
 61: end-volume
 62:
 63: volume vol01-io-cache
 64:     type performance/io-cache
 65:     subvolumes vol01-readdir-ahead
 66: end-volume
 67:
 68: volume vol01-quick-read
 69:     type performance/quick-read
 70:     subvolumes vol01-io-cache
 71: end-volume
 72:
 73: volume vol01-open-behind
 74:     type performance/open-behind
 75:     subvolumes vol01-quick-read
 76: end-volume
 77:
 78: volume vol01-md-cache
 79:     type performance/md-cache
 80:     subvolumes vol01-open-behind
 81: end-volume
 82:
 83: volume vol01
 84:     type debug/io-stats
 85:     option log-level INFO
 86:     option latency-measurement off
 87:     option count-fop-hits off
 88:     subvolumes vol01-md-cache
 89: end-volume
 90:
 91: volume meta-autoload
 92:     type meta
 93:     subvolumes vol01
 94: end-volume
 95:
+------------------------------------------------------------------------------+
[2018-06-29 07:52:03.917518] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:03.918459] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:03.919289] I [rpc-clnt.c:2071:rpc_clnt_reconfig] 0-vol01-client-0: changing port to 49152 (from 0)
[2018-06-29 07:52:03.922903] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-1: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:03.923922] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-1: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:03.925115] I [rpc-clnt.c:2071:rpc_clnt_reconfig] 0-vol01-client-1: changing port to 49152 (from 0)
[2018-06-29 07:52:03.930023] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:03.930756] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:03.931859] I [MSGID: 114046] [client-handshake.c:1176:client_setvolume_cbk] 0-vol01-client-0: Connected to vol01-client-0, attached to remote volume '/data/glusterfs/gluster1/vol01/brick1'.
[2018-06-29 07:52:03.931894] I [MSGID: 108005] [afr-common.c:5081:__afr_handle_child_up_event] 0-vol01-replicate-0: Subvolume 'vol01-client-0' came back up; going online.
[2018-06-29 07:52:03.938903] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-1: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:03.939803] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-1: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:03.941300] I [MSGID: 114046] [client-handshake.c:1176:client_setvolume_cbk] 0-vol01-client-1: Connected to vol01-client-1, attached to remote volume '/data/glusterfs/gluster2/vol01/brick1'.
[2018-06-29 07:52:03.942884] I [fuse-bridge.c:4234:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.24 kernel 7.22
[2018-06-29 07:52:03.942907] I [fuse-bridge.c:4864:fuse_graph_sync] 0-fuse: switched to graph 0
[2018-06-29 07:52:03.946767] W [socket.c:592:__socket_rwv] 0-vol01-client-0: readv on 192.168.10.233:49152 failed (Input/output error)
[2018-06-29 07:52:03.946791] E [socket.c:2785:socket_poller] 0-vol01-client-0: socket_poller 192.168.10.233:49152 failed (Input/output error)
[2018-06-29 07:52:03.946837] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-0: disconnected from vol01-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:03.947264] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GlusterFS 4.x v1) op(LOOKUP(27)) called at 2018-06-29 07:52:03.945958 (xid=0xb)
[2018-06-29 07:52:03.947297] W [MSGID: 114031] [client-rpc-fops_v2.c:2540:client4_0_lookup_cbk] 0-vol01-client-0: remote operation failed. Path: / (00000000-0000-0000-0000-000000000001) [Transport endpoint is not connected]
[2018-06-29 07:52:03.947455] I [MSGID: 109005] [dht-selfheal.c:2328:dht_selfheal_directory] 0-vol01-dht: Directory selfheal failed: Unable to form layout for directory /
[2018-06-29 07:52:07.885389] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:07.886457] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:07.887452] I [rpc-clnt.c:2071:rpc_clnt_reconfig] 0-vol01-client-0: changing port to 49152 (from 0)
[2018-06-29 07:52:07.900627] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:07.901594] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:07.903220] I [MSGID: 114046] [client-handshake.c:1176:client_setvolume_cbk] 0-vol01-client-0: Connected to vol01-client-0, attached to remote volume '/data/glusterfs/gluster1/vol01/brick1'.
[2018-06-29 07:52:13.062404] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-0: disconnected from vol01-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:13.062780] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GlusterFS 4.x v1) op(WRITE(13)) called at 2018-06-29 07:52:13.048761 (xid=0x22)
[2018-06-29 07:52:13.062826] W [MSGID: 114031] [client-rpc-fops_v2.c:658:client4_0_writev_cbk] 0-vol01-client-0: remote operation failed [Transport endpoint is not connected]
[2018-06-29 07:52:13.063017] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GlusterFS 4.x v1) op(WRITE(13)) called at 2018-06-29 07:52:13.049162 (xid=0x23)
[2018-06-29 07:52:13.063175] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GlusterFS 4.x v1) op(WRITE(13)) called at 2018-06-29 07:52:13.049865 (xid=0x24)
[2018-06-29 07:52:13.063319] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GlusterFS 4.x v1) op(WRITE(13)) called at 2018-06-29 07:52:13.050199 (xid=0x25)
[2018-06-29 07:52:13.063473] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GlusterFS 4.x v1) op(WRITE(13)) called at 2018-06-29 07:52:13.050489 (xid=0x26)
[2018-06-29 07:52:13.063600] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GlusterFS 4.x v1) op(WRITE(13)) called at 2018-06-29 07:52:13.050757 (xid=0x27)
[2018-06-29 07:52:13.063745] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GlusterFS 4.x v1) op(WRITE(13)) called at 2018-06-29 07:52:13.051031 (xid=0x28)
[2018-06-29 07:52:13.063883] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GlusterFS 4.x v1) op(WRITE(13)) called at 2018-06-29 07:52:13.053035 (xid=0x29)
The message "W [MSGID: 114031] [client-rpc-fops_v2.c:658:client4_0_writev_cbk] 0-vol01-client-0: remote operation failed [Transport endpoint is not connected]" repeated 7 times between [2018-06-29 07:52:13.062826] and [2018-06-29 07:52:13.063903]
[2018-06-29 07:52:13.104580] W [MSGID: 114031] [client-rpc-fops_v2.c:1570:client4_0_fxattrop_cbk] 0-vol01-client-0: remote operation failed
[2018-06-29 07:52:13.109230] W [socket.c:592:__socket_rwv] 0-vol01-client-1: writev on 192.168.10.234:49152 failed (No data available)
[2018-06-29 07:52:13.109282] E [socket.c:2777:socket_poller] 0-vol01-client-1: poll error on socket
[2018-06-29 07:52:13.109340] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-1: disconnected from vol01-client-1. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:13.109357] E [MSGID: 108006] [afr-common.c:5158:__afr_handle_child_down_event] 0-vol01-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up.
[2018-06-29 07:52:13.109461] I [MSGID: 108006] [afr-common.c:5444:afr_local_init] 0-vol01-replicate-0: no subvolumes up
[2018-06-29 07:52:13.109679] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GlusterFS 4.x v1) op(FSTAT(25)) called at 2018-06-29 07:52:13.104513 (xid=0x42)
[2018-06-29 07:52:13.109700] W [MSGID: 114031] [client-rpc-fops_v2.c:1260:client4_0_fstat_cbk] 0-vol01-client-1: remote operation failed [Transport endpoint is not connected]
[2018-06-29 07:52:13.109854] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GlusterFS 4.x v1) op(FXATTROP(34)) called at 2018-06-29 07:52:13.104729 (xid=0x43)
[2018-06-29 07:52:13.109877] W [MSGID: 114031] [client-rpc-fops_v2.c:1570:client4_0_fxattrop_cbk] 0-vol01-client-1: remote operation failed
[2018-06-29 07:52:13.109897] E [MSGID: 114031] [client-rpc-fops_v2.c:1352:client4_0_finodelk_cbk] 0-vol01-client-0: remote operation failed [Transport endpoint is not connected]
[2018-06-29 07:52:13.110036] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GlusterFS 4.x v1) op(WRITE(13)) called at 2018-06-29 07:52:13.105623 (xid=0x44)
[2018-06-29 07:52:13.110053] W [MSGID: 114031] [client-rpc-fops_v2.c:658:client4_0_writev_cbk] 0-vol01-client-1: remote operation failed [Transport endpoint is not connected]
[2018-06-29 07:52:13.110232] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GlusterFS 4.x v1) op(FSTAT(25)) called at 2018-06-29 07:52:13.105658 (xid=0x45)
[2018-06-29 07:52:13.110250] W [MSGID: 114031] [client-rpc-fops_v2.c:1260:client4_0_fstat_cbk] 0-vol01-client-1: remote operation failed [Transport endpoint is not connected]
[2018-06-29 07:52:13.110386] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GlusterFS 4.x v1) op(FSTAT(25)) called at 2018-06-29 07:52:13.106862 (xid=0x46)
[2018-06-29 07:52:13.110530] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GlusterFS 4.x v1) op(FSTAT(25)) called at 2018-06-29 07:52:13.108103 (xid=0x47)
[2018-06-29 07:52:13.110660] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GlusterFS 4.x v1) op(FSTAT(25)) called at 2018-06-29 07:52:13.108969 (xid=0x48)
[2018-06-29 07:52:13.110793] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GlusterFS 4.x v1) op(FXATTROP(34)) called at 2018-06-29 07:52:13.109256 (xid=0x49)
The message "W [MSGID: 114031] [client-rpc-fops_v2.c:1260:client4_0_fstat_cbk] 0-vol01-client-1: remote operation failed [Transport endpoint is not connected]" repeated 3 times between [2018-06-29 07:52:13.110250] and [2018-06-29 07:52:13.110675]
[2018-06-29 07:52:13.110808] W [MSGID: 114031] [client-rpc-fops_v2.c:1570:client4_0_fxattrop_cbk] 0-vol01-client-1: remote operation failed
[2018-06-29 07:52:13.110933] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GlusterFS 4.x v1) op(FXATTROP(34)) called at 2018-06-29 07:52:13.109284 (xid=0x4a)
The message "W [MSGID: 114031] [client-rpc-fops_v2.c:1570:client4_0_fxattrop_cbk] 0-vol01-client-1: remote operation failed" repeated 3 times between [2018-06-29 07:52:13.110808] and [2018-06-29 07:52:14.878874]
[2018-06-29 07:52:14.878895] E [MSGID: 114031] [client-rpc-fops_v2.c:1352:client4_0_finodelk_cbk] 0-vol01-client-1: remote operation failed [Transport endpoint is not connected]
[2018-06-29 07:52:14.879095] I [MSGID: 108006] [afr-common.c:5444:afr_local_init] 0-vol01-replicate-0: no subvolumes up
[2018-06-29 07:52:14.879217] W [fuse-bridge.c:2427:fuse_writev_cbk] 0-glusterfs-fuse: 76: WRITE => -1 gfid=a8848be6-1c05-4a72-88a7-5209eeef7c2c fd=0x7f7c50023a68 (Transport endpoint is not connected)
[2018-06-29 07:52:14.880509] W [fuse-bridge.c:1406:fuse_err_cbk] 0-glusterfs-fuse: 77: FLUSH() ERR => -1 (Transport endpoint is not connected)
The message "I [MSGID: 108006] [afr-common.c:5444:afr_local_init] 0-vol01-replicate-0: no subvolumes up" repeated 11 times between [2018-06-29 07:52:14.879095] and [2018-06-29 07:52:17.908056]
[2018-06-29 07:52:17.908066] E [MSGID: 101046] [dht-common.c:1501:dht_lookup_dir_cbk] 0-vol01-dht: dict is null
[2018-06-29 07:52:17.908175] W [fuse-bridge.c:896:fuse_attr_cbk] 0-glusterfs-fuse: 79: LOOKUP() / => -1 (Transport endpoint is not connected)
[2018-06-29 07:52:18.758975] I [MSGID: 108006] [afr-common.c:5444:afr_local_init] 0-vol01-replicate-0: no subvolumes up
[2018-06-29 07:52:18.759155] I [MSGID: 108006] [afr-common.c:5444:afr_local_init] 0-vol01-replicate-0: no subvolumes up
[2018-06-29 07:52:18.759165] E [MSGID: 101046] [dht-common.c:1501:dht_lookup_dir_cbk] 0-vol01-dht: dict is null
[2018-06-29 07:52:18.759214] W [fuse-bridge.c:896:fuse_attr_cbk] 0-glusterfs-fuse: 80: LOOKUP() / => -1 (Transport endpoint is not connected)
[2018-06-29 07:52:21.662626] I [MSGID: 108006] [afr-common.c:5444:afr_local_init] 0-vol01-replicate-0: no subvolumes up
[2018-06-29 07:52:21.662756] I [MSGID: 108006] [afr-common.c:5444:afr_local_init] 0-vol01-replicate-0: no subvolumes up
[2018-06-29 07:52:21.662765] E [MSGID: 101046] [dht-common.c:1501:dht_lookup_dir_cbk] 0-vol01-dht: dict is null
[2018-06-29 07:52:21.662811] W [fuse-bridge.c:896:fuse_attr_cbk] 0-glusterfs-fuse: 81: LOOKUP() / => -1 (Transport endpoint is not connected)
[2018-06-29 07:52:21.663567] I [MSGID: 108006] [afr-common.c:5444:afr_local_init] 0-vol01-replicate-0: no subvolumes up
[2018-06-29 07:52:21.663630] W [fuse-bridge.c:3205:fuse_statfs_cbk] 0-glusterfs-fuse: 82: ERR => -1 (Transport endpoint is not connected)
[2018-06-29 07:52:23.896362] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:23.896687] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-0: disconnected from vol01-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:23.897321] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:23.896427 (xid=0x2a)
[2018-06-29 07:52:23.897353] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-0: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:23.903241] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-1: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:23.903566] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-1: disconnected from vol01-client-1. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:23.904166] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:23.903281 (xid=0x4b)
[2018-06-29 07:52:23.904199] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-1: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:26.909819] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:26.910175] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-0: disconnected from vol01-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:26.910858] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:26.909871 (xid=0x2b)
[2018-06-29 07:52:26.910888] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-0: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:26.913921] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-1: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:26.914163] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-1: disconnected from vol01-client-1. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:26.914417] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:26.913961 (xid=0x4c)
[2018-06-29 07:52:26.914443] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-1: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:29.921628] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:29.921867] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-0: disconnected from vol01-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:29.922215] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:29.921674 (xid=0x2c)
[2018-06-29 07:52:29.922257] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-0: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:29.926462] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-1: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:29.926774] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-1: disconnected from vol01-client-1. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:29.927415] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:29.926499 (xid=0x4d)
[2018-06-29 07:52:29.927447] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-1: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:32.935218] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:32.935553] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-0: disconnected from vol01-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:32.936196] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:32.935271 (xid=0x2d)
[2018-06-29 07:52:32.936230] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-0: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:32.938940] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-1: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:32.939232] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-1: disconnected from vol01-client-1. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:32.939497] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:32.938982 (xid=0x4e)
[2018-06-29 07:52:32.939526] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-1: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:35.948789] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:35.949172] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-0: disconnected from vol01-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:35.949860] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:35.948839 (xid=0x2e)
[2018-06-29 07:52:35.949901] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-0: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:35.952154] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-1: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:35.952408] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-1: disconnected from vol01-client-1. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:35.952670] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:35.952199 (xid=0x4f)
[2018-06-29 07:52:35.952699] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-1: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:38.959369] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:38.959733] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-0: disconnected from vol01-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:38.960536] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:38.959426 (xid=0x2f)
[2018-06-29 07:52:38.960573] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-0: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:38.963381] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-1: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:38.963618] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-1: disconnected from vol01-client-1. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:38.963877] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:38.963428 (xid=0x50)
[2018-06-29 07:52:38.963906] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-1: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:41.969152] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:41.969397] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-0: disconnected from vol01-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:41.969731] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:41.969204 (xid=0x30)
[2018-06-29 07:52:41.969769] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-0: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:41.974130] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-1: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:41.974381] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-1: disconnected from vol01-client-1. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:41.974669] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:41.974198 (xid=0x51)
[2018-06-29 07:52:41.974698] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-1: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:44.984465] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:44.984781] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-0: disconnected from vol01-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:44.985444] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:44.984514 (xid=0x31)
[2018-06-29 07:52:44.985477] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-0: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:44.990039] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-1: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:44.990352] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-1: disconnected from vol01-client-1. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:44.990729] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:44.990094 (xid=0x52)
[2018-06-29 07:52:44.990771] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-1: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:47.994448] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:47.994684] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-0: disconnected from vol01-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:47.994947] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:47.994499 (xid=0x32)
[2018-06-29 07:52:47.994984] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-0: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:48.002544] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-1: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:48.002809] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-1: disconnected from vol01-client-1. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:48.003388] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:48.002577 (xid=0x53)
[2018-06-29 07:52:48.003416] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-1: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:49.440101] I [MSGID: 108006] [afr-common.c:5444:afr_local_init] 0-vol01-replicate-0: no subvolumes up
[2018-06-29 07:52:49.440257] I [MSGID: 108006] [afr-common.c:5444:afr_local_init] 0-vol01-replicate-0: no subvolumes up
[2018-06-29 07:52:49.440266] E [MSGID: 101046] [dht-common.c:1501:dht_lookup_dir_cbk] 0-vol01-dht: dict is null
[2018-06-29 07:52:49.440310] W [fuse-bridge.c:896:fuse_attr_cbk] 0-glusterfs-fuse: 83: LOOKUP() / => -1 (Transport endpoint is not connected)
[2018-06-29 07:52:50.182948] I [MSGID: 108006] [afr-common.c:5444:afr_local_init] 0-vol01-replicate-0: no subvolumes up
[2018-06-29 07:52:50.183108] I [MSGID: 108006] [afr-common.c:5444:afr_local_init] 0-vol01-replicate-0: no subvolumes up
[2018-06-29 07:52:50.183116] E [MSGID: 101046] [dht-common.c:1501:dht_lookup_dir_cbk] 0-vol01-dht: dict is null
[2018-06-29 07:52:50.197572] W [fuse-bridge.c:896:fuse_attr_cbk] 0-glusterfs-fuse: 84: LOOKUP() / => -1 (Transport endpoint is not connected)
[2018-06-29 07:52:50.575887] I [MSGID: 108006] [afr-common.c:5444:afr_local_init] 0-vol01-replicate-0: no subvolumes up
[2018-06-29 07:52:50.576030] I [MSGID: 108006] [afr-common.c:5444:afr_local_init] 0-vol01-replicate-0: no subvolumes up
[2018-06-29 07:52:50.576039] E [MSGID: 101046] [dht-common.c:1501:dht_lookup_dir_cbk] 0-vol01-dht: dict is null
[2018-06-29 07:52:50.576087] W [fuse-bridge.c:896:fuse_attr_cbk] 0-glusterfs-fuse: 85: LOOKUP() / => -1 (Transport endpoint is not connected)
[2018-06-29 07:52:51.012626] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-0: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:51.012924] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-0: disconnected from vol01-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:51.013615] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-0: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:51.012667 (xid=0x33)
[2018-06-29 07:52:51.013647] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-0: received RPC status error [Transport endpoint is not connected]
[2018-06-29 07:52:51.018855] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol01-client-1: error returned while attempting to connect to host:(null), port:0
[2018-06-29 07:52:51.019093] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-vol01-client-1: disconnected from vol01-client-1. Client process will keep trying to connect to glusterd until brick's port is available
[2018-06-29 07:52:51.019378] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7c73b8ae5b] (--> /lib64/libgfrpc.so.0(+0xce4e)[0x7f7c73955e4e] (--> /lib64/libgfrpc.so.0(+0xcf6e)[0x7f7c73955f6e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7c7395760d] (--> /lib64/libgfrpc.so.0(+0xf178)[0x7f7c73958178] ))))) 0-vol01-client-1: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-06-29 07:52:51.018893 (xid=0x54)
[2018-06-29 07:52:51.019408] W [MSGID: 114032] [client-handshake.c:1620:client_dump_version_cbk] 0-vol01-client-1: received RPC status error [Transport endpoint is not connected]



=====================================================

After trying to copy the archive, I end up with different sized files on the two bricks:

[root@gluster1 brick1]# pwd
/data/glusterfs/gluster1/vol01/brick1
[root@gluster1 brick1]# ll
total 384
-rw------- 2 root root 393216 Jun 29 10:52 mybee.tar.gz



[root@gluster2 brick1]# pwd
/data/glusterfs/gluster2/vol01/brick1
[root@gluster2 brick1]# ll
total 1280
-rw------- 2 root root 1310720 Jun 29 10:52 mybee.tar.gz


=====================================================

These are the informations regarding the gluster volume:

[root@gluster1 glusterfs]# gluster volume info vol01

Volume Name: vol01
Type: Replicate
Volume ID: d3b39520-3fbb-4cfc-b736-338d7726b5a1
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: gluster1:/data/glusterfs/gluster1/vol01/brick1
Brick2: gluster2:/data/glusterfs/gluster2/vol01/brick1
Options Reconfigured:
network.ping-timeout: 5
server.ssl: on
client.ssl: on
auth.ssl-allow: gluster01,gluster02,gluster-client
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off

=====================================================

Here is the peers information:

[root@gluster1 brick1]# gluster peer status
Number of Peers: 1

Hostname: gluster2
Uuid: 9ccac8e3-71c1-43e4-b723-fb7374be8a1d
State: Peer in Cluster (Connected)


=====================================================

Here is the volume status:

[root@gluster1 brick1]# gluster volume status vol01
Status of volume: vol01
Gluster process                             TCP Port  RDMA Port Online  Pid
------------------------------------------------------------------------------
Brick gluster1:/data/glusterfs/gluster1/vol
01/brick1                                   49152     0 Y       11108
Brick gluster2:/data/glusterfs/gluster2/vol
01/brick1                                   49152     0 Y       10942
Self-heal Daemon on localhost               N/A       N/A Y       11133
Self-heal Daemon on gluster2                N/A       N/A Y       10968

Task Status of Volume vol01
------------------------------------------------------------------------------
There are no active volume tasks


=====================================================


These are the installed packages on gluster-client:

[root@gluster-client glusterfs]# rpm -qa | grep "gluster\|fuse"
centos-release-gluster40-1.0-2.el7.centos.x86_64
glusterfs-client-xlators-4.0.2-1.el7.x86_64
fuse-2.9.2-10.el7.x86_64
glusterfs-4.0.2-1.el7.x86_64
glusterfs-fuse-4.0.2-1.el7.x86_64
fuse-libs-2.9.2-10.el7.x86_64
glusterfs-libs-4.0.2-1.el7.x86_64



And these are the installed packages on gluster1 and gluster2 storage nodes:


[root@gluster1 glusterfs]# rpm -qa | grep "gluster\|fuse"
glusterfs-api-4.0.2-1.el7.x86_64
centos-release-gluster40-1.0-2.el7.centos.x86_64
glusterfs-libs-4.0.2-1.el7.x86_64
glusterfs-client-xlators-4.0.2-1.el7.x86_64
glusterfs-cli-4.0.2-1.el7.x86_64
glusterfs-fuse-4.0.2-1.el7.x86_64
glusterfs-server-4.0.2-1.el7.x86_64
glusterfs-4.0.2-1.el7.x86_64




Can anyone point me in the right direction? What am I doing wrong? The mount works fine until I try to copy an archive, multiple smaller files or a bigger file on it (meaning it shows correctly in df -Th and I can create several files with "touch file1 file2..."). Basically, after any data transfer, I get these errors.


I followed the indications from the redhat page:

https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.1/html/administration_guide/chap-network_encryption

UPDATE 1:
I tried doing the exact same steps in Gluster 3.12 and had no problem. The steps worked and SSL/TLS was enabled. There was no transport error or anything and I also checked if SSL/TLS was enabled. Afterwards, I also tried with the new release 4.1 and the problem persists (same error with "Transport endpoint is not connected").
Let me know if you need any other info. Any help is much appreciated.

Thank you.
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users



_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users
[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux