Re: User concurrency thresholding: where do I look?

"Jignesh K. Shah" <J.K.Shah@xxxxxxx> · Thu, 26 Jul 2007 17:17:55 -0400

Will try 16 and 32  CLOGBUFFER tomorrow:

But here is locks data again with about increased time profiling (about 
2 minutes) for the connection with about 2000 users:

bash-3.00# time ./4_lwlock_waits.d 13583
^C

            Lock Id            Mode           Count
      ProcArrayLock          Shared               5
         XidGenLock       Exclusive              13
    CLogControlLock          Shared              14
         XidGenLock          Shared              15
    CLogControlLock       Exclusive              21
      WALInsertLock       Exclusive              77
       WALWriteLock       Exclusive             175
      ProcArrayLock       Exclusive             275

            Lock Id   Combined Time (ns)
         XidGenLock            194966200
      WALInsertLock            517955000
    CLogControlLock            679665100
       WALWriteLock           2838716200
      ProcArrayLock          44181002600

Top Wait time   seems to come from the following code path for 
ProcArrayLock:

            Lock Id            Mode           Count
      ProcArrayLock       Exclusive              21

            Lock Id   Combined Time (ns)
      ProcArrayLock           5255937500

            Lock Id   Combined Time (ns)

             postgres`LWLockAcquire+0x1f0
             postgres`CommitTransaction+0x104
             postgres`CommitTransactionCommand+0xbc
             postgres`finish_xact_command+0x78
             postgres`exec_execute_message+0x42c
             postgres`PostgresMain+0x1838
             postgres`BackendRun+0x2f8
             postgres`ServerLoop+0x680
             postgres`PostmasterMain+0xda8
             postgres`main+0x3d0
             postgres`_start+0x17c
      5255937500

Regards,
Jignesh

Simon Riggs wrote:
On Thu, 2007-07-26 at 15:44 -0400, Jignesh K. Shah wrote:

BEAUTIFUL!!!

Using the Patch I can now go upto 1300 users without dropping.. But now 
it still repeats at 1300-1350 users..

OK, can you try again with 16 and 32 buffers please?  We need to know
how many is enough and whether this number needs to be variable via a
parameter, or just slightly larger by default. Thanks.

I corrected the Lock Descriptions based on what I got from lwlock.h and 
retried the whole scalability again with profiling again.. This time it 
looks like the ProcArrayLock

That's what I would expect with that many users.

             Lock Id            Mode           Count
          XidGenLock       Exclusive               1
     CLogControlLock          Shared               2
          XidGenLock          Shared               2
        WALWriteLock       Exclusive               4
       WALInsertLock       Exclusive               8
     CLogControlLock       Exclusive               9
       ProcArrayLock       Exclusive               9

...but as Tom mentioned, we need to do longer runs now so these counts
get to somewhere in the hundreds so we have some statistical validity.

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster