Hi all, I am running replica 3 on SSDs with 10G networking, everything works OK but VMs stored in Gluster volume occasionally freeze with “Task XY blocked for more than 120 seconds”. Only solution is to poweroff (hard) VM and than boot it up again. I am unable to SSH and also login with console, its stuck probably on some disk operation. No error/warning logs or messages are store in VMs logs. KVM/Libvirt(qemu) using libgfapi and fuse mount to access VM disks on replica volume. Can someone advice how to debug this problem or what can cause these issues? It’s really annoying, I’ve tried to google everything but nothing came up. I’ve tried changing virtio-scsi-pci to virtio-blk-pci disk drivers, but its not related. BR, Martin  These are volume settings : Volume ID: b021bbb6-fa99-4cc7-88f6-49152a22cb9e Status: Started Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: node1:/imagestore/brick1 Brick2: node2:/imagestore/brick1 Brick3: node3:/imagestore/brick1 Options Reconfigured: performance.client-io-threads: on performance.quick-read: off performance.read-ahead: off performance.io-cache: off performance.stat-prefetch: on cluster.min-free-disk: 10% cluster.server-quorum-type: server cluster.quorum-type: auto cluster.eager-lock: enable cluster.data-self-heal-algorithm: full network.remote-dio: enable network.ping-timeout: 30 diagnostics.count-fop-hits: on diagnostics.latency-measurement: on client.event-threads: 4 server.event-threads: 4 storage.owner-gid: 9869 storage.owner-uid: 9869 server.allow-insecure: on nfs.disable: on performance.readdir-ahead: on |
_______________________________________________ Community Meeting Calendar: APAC Schedule - Every 2nd and 4th Tuesday at 11:30 AM IST Bridge: https://bluejeans.com/836554017 NA/EMEA Schedule - Every 1st and 3rd Tuesday at 01:00 PM EDT Bridge: https://bluejeans.com/486278655 Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-devel
