Hi,
The email is talking about many issues. Let me ask a few questions to get a whole picture.
1. are the peers are in the connected state now? or they still in the rejected state?
2. What led you to see "locking failed" messages? We would like to if there is a reproducer and fix the issue if any.
3. Another transaction in progress message appears when there is already a operation going on. Are you seeing this when there is no such transaction going on?
4. When did you hit the timedouts? Did you tried to look at the pstack output of glusterd process? If so, please share the pstack output.
On Tue, Apr 21, 2020 at 7:08 PM <nico@xxxxxxxxxx> wrote:
Hi all.
We're using 3 nodes Gluster 7.3 (2 + 1 arbiter), yesterday node 2 was rejected from cluster and I applied following steps to fix : https://staged-gluster-docs.readthedocs.io/en/release3.7.0beta1/Administrator%20Guide/Resolving%20Peer%20Rejected/
I also saw https://docs.gluster.org/en/latest/Troubleshooting/troubleshooting-glusterd/ but solution isn't compatible as cluster.max-op-version doesn't exist and all op-version are the same on all 3 nodes.
After renewing SSL certs and several restart all volumes came back online but glusterd log file on all 3 nodes is filled with nothing else than following 3 lines :
[2020-04-21 13:05:19.478913] I [socket.c:4347:ssl_setup_connection_params] 0-socket.management: SSL support on the I/O path is ENABLED
[2020-04-21 13:05:19.478972] I [socket.c:4350:ssl_setup_connection_params] 0-socket.management: SSL support for glusterd is ENABLED
[2020-04-21 13:05:19.478986] I [socket.c:4360:ssl_setup_connection_params] 0-socket.management: using certificate depth 1
Moreover, I have "Locking failed", "Another transaction is in progress" and "Error : Request timed out" on gluster volume status volxxx command.
All SSL certs on clients have also been renewed and all volumes were remounted. All 3 nodes were alternatively restarted (glusterd) and rebooted.
The cluster is not in production environment but there's about ~250 clients for ~75 volumes, I don't know how to troubleshoot and fix this problem, if anyone has an idea.
________
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users
Thanks,
Sanju________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://bluejeans.com/441850968 Gluster-users mailing list Gluster-users@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-users
