On Fri, Jul 5, 2013 at 7:00 AM, Stuart Ford <stuart.ford@xxxxxxxxxxxx> wrote:
Do you share the same VLAN for guest traffic with NFS or iSCSI used by vmware for storing VMDKs?
Dear community
Twice today our PG 9.1 server has caused a "soft lockup", with a kernel
message like this:
[1813775.496127] BUG: soft lockup - CPU#3 stuck for 73s! [postgres:18723]
Full dmesg output - http://pastebin.com/YdWSmNUp
The incidents were approximately two hours apart and the server was
momentarily unavailable before coming back again, with no restore actions
required - it just carried on. It's never done this before, I've checked
in the logs. The server is about 3 weeks old and runs Debian 7 under
VMWare.
Does anybody know what could cause this, and, if so, is it something to be
concerned about and what can be done to stop it?
Before I looked at your pastebin, I was going to ask "What kind of storage are the VMDKs on? If they are on NFS, iSCSI or FC, could the NAS/SAN be experiencing a problem?"
But I see in the stack trace that the kernel thread hung in "vmxnet3_xmit_frame" (sending an Ethernet frame on your virtual NIC).
Describe your vmware network topology.
Do you share the same VLAN for guest traffic with NFS or iSCSI used by vmware for storing VMDKs?
Are there any errors recorded on the Ethernet switch connected to your VMWare servers?
What path does a packet take to get from a postgresql server process in your VM to a client?
What version of VMWare are you running?
If you are managing it with vCenter, are there any alarms of events in the VMWare logs?
