VMs blocked for more than 120 seconds

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi all,

I am running replica 3 on SSDs with 10G networking, everything works OK but VMs stored in Gluster volume occasionally freeze with “Task XY blocked for more than 120 seconds”.
Only solution is to poweroff (hard) VM and than boot it up again. I am unable to SSH and also login with console, its stuck probably on some disk operation. No error/warning logs or messages are store in VMs logs.

KVM/Libvirt(qemu) using libgfapi and fuse mount to access VM disks on replica volume. Can someone advice  how to debug this problem or what can cause these issues? 
It’s really annoying, I’ve tried to google everything but nothing came up. I’ve tried changing virtio-scsi-pci to virtio-blk-pci disk drivers, but its not related.

BR,
Martin


These are volume settings :

Type: Replicate
Volume ID: b021bbb6-fa99-4cc7-88f6-49152a22cb9e
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: node1:/imagestore/brick1
Brick2: node2:/imagestore/brick1
Brick3: node3:/imagestore/brick1
Options Reconfigured:
performance.client-io-threads: on
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
performance.stat-prefetch: on
cluster.min-free-disk: 10%
cluster.server-quorum-type: server
cluster.quorum-type: auto
cluster.eager-lock: enable
cluster.data-self-heal-algorithm: full
network.remote-dio: enable
network.ping-timeout: 30
diagnostics.count-fop-hits: on
diagnostics.latency-measurement: on
client.event-threads: 4
server.event-threads: 4
storage.owner-gid: 9869
storage.owner-uid: 9869
server.allow-insecure: on
nfs.disable: on
performance.readdir-ahead: on



_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users

[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux