Hi Craig/Tom, I've managed to trap the full stack trace this time - 2 processes chewing up 25% each (1 core each on a quad core server), while SELECT * FROM pg_stat_activity revealed they were <IDLE>. I also confirmed that the two runaway processes were started by a developer remotely connecting pgAdmin via SSL. It appears that no actual queries were run, just the connection established. Unfortunately I couldn't confirm if the connections were dirty disconnected. Relevant DLL versions are: 8.3\bin\LIBEAY32.DLL - 0.9.8.5 8.3\bin\SSLEAY32.DLL - 0.9.8.5 Other DLLS - standard for Windows 2003 Server SP2 The juiciest stack traces I captured (in no particular order) were: ntkrnlpa.exe!KiUnexpectedInterrupt+0x48 ntkrnlpa.exe!KeWaitForSingleObject+0x346 ntkrnlpa.exe!ZwYieldExecution+0x3514 hal.dll!KfRaiseIrql+0xe5 hal.dll!KeRaiseIrqlToSynchLevel+0x8d hal.dll!HalEndSystemInterrupt+0x67 hal.dll!HalInitializeProcessor+0x856 LIBEAY32.dll!lh_doall_arg+0x1c4 ntkrnlpa.exe!KiUnexpectedInterrupt+0x48 ntkrnlpa.exe!KeWaitForSingleObject+0x346 ntkrnlpa.exe!ZwYieldExecution+0x3514 hal.dll!KfRaiseIrql+0xe5 hal.dll!KeRaiseIrqlToSynchLevel+0x8d hal.dll!KfLowerIrql+0x62 ntkrnlpa.exe!ZwYieldExecution+0x163a ntkrnlpa.exe!KeInsertQueueApc+0x57 ntkrnlpa.exe!IoCancelIrp+0x27d hal.dll!HalEndSystemInterrupt+0x6e hal.dll!HalInitializeProcessor+0x856 ntkrnlpa.exe!IofCallDriver+0x45 ntkrnlpa.exe!NtWriteFile+0x2943 ntkrnlpa.exe!NtWriteFile+0x36cb ntkrnlpa.exe!NtDeviceIoControlFile+0x2a ntkrnlpa.exe!KeReleaseInStackQueuedSpinLockFromDpcLevel+0xb64 ntdll.dll!KiFastSystemCallRet WS2_32.dll!WSARecv+0x65 WSOCK32.dll!recv+0x31 LIBEAY32.dll!BIO_sock_should_retry+0x57 postgres.exe!my_sock_read+0x1b LIBEAY32.dll!BIO_read+0x6f SSLEAY32.dll!SSLv3_client_method+0x1ee1 SSLEAY32.dll!SSLv3_client_method+0x22ea mswsock.dll!StartWsdpService+0x500 SSLEAY32.dll!SSLv3_client_method+0x225a SSLEAY32.dll!SSLv3_client_method+0x2a15 postgres.exe!pgwin32_waitforsinglesocket+0x1ed postgres.exe!secure_read+0x26 postgres.exe!pq_recvbuf+0x71 postgres.exe!pq_getbyte+0x15 postgres.exe!SocketBackend+0x6 postgres.exe!PostgresMain+0xbf8 postgres.exe!BackendRun+0x200 postgres.exe!SubPostmasterMain+0x21d postgres.exe!main+0x177 postgres.exe!__tmainCRTStartup+0x10f kernel32.dll!ProcessIdToSessionId+0x209 ntkrnlpa.exe!ZwYieldExecution+0x163a ntkrnlpa.exe!KeSetEvent+0xcc ntkrnlpa.exe!PsLookupThreadByThreadId+0x54bc ntkrnlpa.exe!KiDeliverApc+0xbb ntkrnlpa.exe!ZwYieldExecution+0x3801 ntkrnlpa.exe!KeWaitForSingleObject+0x346 ntkrnlpa.exe!ZwYieldExecution+0x3514 ntkrnlpa.exe!KiCheckForKernelApcDelivery+0x1c ntkrnlpa.exe!wctomb+0x4229 ntkrnlpa.exe!IofCallDriver+0x45 ntkrnlpa.exe!NtWriteFile+0x2943 ntkrnlpa.exe!NtWriteFile+0x36cb ntkrnlpa.exe!NtDeviceIoControlFile+0x2a ntkrnlpa.exe!KeReleaseInStackQueuedSpinLockFromDpcLevel+0xb64 ntdll.dll!KiFastSystemCallRet WS2_32.dll!WSARecv+0x65 WSOCK32.dll!recv+0x31 LIBEAY32.dll!BIO_sock_should_retry+0x57 postgres.exe!my_sock_read+0x1b LIBEAY32.dll!BIO_read+0x6f SSLEAY32.dll!SSLv3_client_method+0x1ee1 SSLEAY32.dll!SSLv3_client_method+0x22ea mswsock.dll!StartWsdpService+0x500 SSLEAY32.dll!SSLv3_client_method+0x225a SSLEAY32.dll!SSLv3_client_method+0x2a15 postgres.exe!pgwin32_waitforsinglesocket+0x1ed postgres.exe!secure_read+0x26 postgres.exe!pq_recvbuf+0x71 postgres.exe!pq_getbyte+0x15 postgres.exe!SocketBackend+0x6 postgres.exe!PostgresMain+0xbf8 postgres.exe!BackendRun+0x200 postgres.exe!SubPostmasterMain+0x21d postgres.exe!main+0x177 postgres.exe!__tmainCRTStartup+0x10f kernel32.dll!ProcessIdToSessionId+0x209 ntkrnlpa.exe!KiUnexpectedInterrupt+0x48 ntkrnlpa.exe!KeWaitForSingleObject+0x346 ntkrnlpa.exe!ZwYieldExecution+0x3514 ntkrnlpa.exe!KiCheckForKernelApcDelivery+0x1c ntkrnlpa.exe!NtWriteFile+0x3195 ntkrnlpa.exe!NtDeviceIoControlFile+0x2a ntkrnlpa.exe!KeReleaseInStackQueuedSpinLockFromDpcLevel+0xb64 ntdll.dll!KiFastSystemCallRet WS2_32.dll!WSARecv+0x65 WSOCK32.dll!recv+0x31 LIBEAY32.dll!BIO_sock_should_retry+0x57 postgres.exe!my_sock_read+0x1b LIBEAY32.dll!BIO_read+0x6f SSLEAY32.dll!SSLv3_client_method+0x1ee1 SSLEAY32.dll!SSLv3_client_method+0x22ea mswsock.dll!StartWsdpService+0x500 SSLEAY32.dll!SSLv3_client_method+0x225a SSLEAY32.dll!SSLv3_client_method+0x2a15 postgres.exe!pgwin32_waitforsinglesocket+0x1ed postgres.exe!secure_read+0x26 postgres.exe!pq_recvbuf+0x71 postgres.exe!pq_getbyte+0x15 postgres.exe!SocketBackend+0x6 postgres.exe!PostgresMain+0xbf8 postgres.exe!BackendRun+0x200 postgres.exe!SubPostmasterMain+0x21d postgres.exe!main+0x177 postgres.exe!__tmainCRTStartup+0x10f kernel32.dll!ProcessIdToSessionId+0x209 I'd appreciate any help diagnosing this problem - cutting off remote access via SSL isn't the ideal solution. Regards, -Brendan -----Original Message----- From: Craig Ringer [mailto:craig@xxxxxxxxxxxxxxxxxxxxx] Sent: Wednesday, 5 August 2009 5:44 PM To: Brendan Hill Cc: 'Tom Lane'; pgsql-general@xxxxxxxxxxxxxx Subject: RE: Idle processes chewing up CPU? On Wed, 2009-08-05 at 16:44 +1000, Brendan Hill wrote: > Hi Craig, > > Sorry, I had the stack trace so I thought it was enough. I'll make sure the > debug environment is set up and post the full stack traces again. No worries. Sorry it cost you time. I've extended the wiki article on win32 debug info to (hopefully) explain how to identify a useful stack trace, or at least a likely bogus one. If you feel like a look I'd value your opinion especially when it comes to clarity/readability. http://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQ L_backend_on_Windows -- Craig Ringer -- Sent via pgsql-general mailing list (pgsql-general@xxxxxxxxxxxxxx) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
