Re: 389 Deadlock

Rich Megginson <rmeggins@xxxxxxxxxx> · Mon, 22 Jul 2013 09:02:06 -0600



    On 07/22/2013 08:46 AM, Jeffrey Dunham
      wrote:

    
      Here's thread 31: 

        (gdb) thread 31

        [Switching to thread 31 (Thread 0x2b0e92901940 (LWP 10739))]#0 
        0x00002b0df2847654 in __lll_lock_wait () from
        /lib64/libpthread.so.0

        (gdb) bt

        #0  0x00002b0df2847654 in __lll_lock_wait () from
        /lib64/libpthread.so.0

        #1  0x00002b0df2842f80 in _L_lock_1233 () from
        /lib64/libpthread.so.0

        #2  0x00002b0df2842f03 in pthread_mutex_lock () from
        /lib64/libpthread.so.0

        #3  0x00002b0df2204289 in PR_Lock () from /usr/lib64/libnspr4.so

        #4  0x0000000000411e88 in ber_sockbuf_free ()

        #5  0x00002b0df04d652a in ?? () from
        /usr/lib64/dirsrv/libslapd.so.0

        #6  0x00002b0df04d6f1e in send_ldap_result_ext () from
        /usr/lib64/dirsrv/libslapd.so.0

        #7  0x00002b0df04d7421 in send_ldap_result () from
        /usr/lib64/dirsrv/libslapd.so.0

        #8  0x00002b0df04c58c4 in slapi_send_ldap_result () from
        /usr/lib64/dirsrv/libslapd.so.0

        #9  0x000000000042524e in ?? ()

        #10 0x00002b0df04c70f7 in plugin_call_exop_plugins () from
        /usr/lib64/dirsrv/libslapd.so.0

        #11 0x000000000041841c in ber_sockbuf_free ()

        #12 0x00000000004137ad in ber_sockbuf_free ()

        #13 0x00002b0df22098cd in ?? () from /usr/lib64/libnspr4.so

        #14 0x00002b0df284083d in start_thread () from
        /lib64/libpthread.so.0

        #15 0x00002b0df2b2af8d in clone () from /lib64/libc.so.6

      
    For hangs/deadlocks, please follow this:

    
    http://port389.org/wiki/FAQ#Debugging_Hangs

    
    1.2.11 has improved deadlock detection and handling

    
        On Mon, Jul 22, 2013 at 7:42 AM, Ludwig
          Krispenz <lkrispen@xxxxxxxxxx>
          wrote:

          
             could you add the bt
              for thread 31 ?
              
                
                  On 07/22/2013 03:58 PM, Jeffrey Dunham wrote:

                  
                      Hello,

                        
                        Starting a day ago we have been noticing
                        deadlocks on select 389 servers.  It does not
                        happen on all of them (10% of servers have been
                        affected currently, and no repeat offenders).

                        
                      Just looking for any hints on where to look
                        or what could be causing this issue.

                      
                      Version: 389-Directory/1.2.10.14

                        
                        From GDB:

                        0x00002b0df2847654 in __lll_lock_wait () from
                        /lib64/libpthread.so.0

                        (gdb) info threads

                          40 Thread 0x2b0e86c91940 (LWP 10689) 
                        0x00002b0df2b24122 in select () from
                        /lib64/libc.so.6

                          39 Thread 0x2b0e87692940 (LWP 10690) 
                        0x00002b0df2b24122 in select () from
                        /lib64/libc.so.6

                          38 Thread 0x2b0e88093940 (LWP 10691) 
                        0x00002b0df2b24122 in select () from
                        /lib64/libc.so.6

                          37 Thread 0x2b0e88a94940 (LWP 10692) 
                        0x00002b0df2b24122 in select () from
                        /lib64/libc.so.6

                          36 Thread 0x2b0e89495940 (LWP 10734) 
                        0x00002b0df2845019 in pthread_cond_wait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          35 Thread 0x2b0e89e96940 (LWP 10735) 
                        0x00002b0df2845019 in pthread_cond_wait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          34 Thread 0x2b0e8a897940 (LWP 10736) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          33 Thread 0x2b0e8b298940 (LWP 10737) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          32 Thread 0x2b0e91f00940 (LWP 10738) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          31 Thread 0x2b0e92901940 (LWP 10739) 
                        0x00002b0df2847654 in __lll_lock_wait () from
                        /lib64/libpthread.so.0

                          30 Thread 0x2b0e93302940 (LWP 10740) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          29 Thread 0x2b0e93d03940 (LWP 10741) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          28 Thread 0x2b0e94704940 (LWP 10742) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          27 Thread 0x2b0e95105940 (LWP 10743) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          26 Thread 0x2b0e95b06940 (LWP 10744) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          25 Thread 0x2b0e96507940 (LWP 10745) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          24 Thread 0x2b0e96f08940 (LWP 10746) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          23 Thread 0x2b0e97909940 (LWP 10747) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          22 Thread 0x2b0e9830a940 (LWP 10748) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          21 Thread 0x2b0e98d0b940 (LWP 10749) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          20 Thread 0x2b0e9970c940 (LWP 10750) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          19 Thread 0x2b0e9a10d940 (LWP 10751) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          18 Thread 0x2b0e9ab0e940 (LWP 10752) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          17 Thread 0x2b0e9b50f940 (LWP 10753) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          16 Thread 0x2b0e9bf10940 (LWP 10754) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          15 Thread 0x2b0e9c911940 (LWP 10755) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          14 Thread 0x2b0e9d312940 (LWP 10756) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          13 Thread 0x2b0e9dd13940 (LWP 10757) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          12 Thread 0x2b0e9e714940 (LWP 10758) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          11 Thread 0x2b0e9f115940 (LWP 10759) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          10 Thread 0x2b0e9fb16940 (LWP 10760) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          9 Thread 0x2b0ea0517940 (LWP 10761) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          8 Thread 0x2b0ea0f18940 (LWP 10762) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          7 Thread 0x2b0ea1919940 (LWP 10763) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          6 Thread 0x2b0ea231a940 (LWP 10764) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          5 Thread 0x2b0ea2d1b940 (LWP 10765) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          4 Thread 0x2b0ea371c940 (LWP 10766) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          3 Thread 0x2b0ea411d940 (LWP 10767) 
                        0x00002b0df2845280 in pthread_cond_timedwait@@GLIBC_2.3.2
                        () from /lib64/libpthread.so.0

                          2 Thread 0x2b0ea4b1e940 (LWP 10768) 
                        0x00002b0df2b24122 in select () from
                        /lib64/libc.so.6

                        * 1 Thread 0x2b0df452a7e0 (LWP 10666) 
                        0x00002b0df2847654 in __lll_lock_wait () from
                        /lib64/libpthread.so.0

                        (gdb) bt

                        #0  0x00002b0df2847654 in __lll_lock_wait ()
                        from /lib64/libpthread.so.0

                        #1  0x00002b0df2842f80 in _L_lock_1233 () from
                        /lib64/libpthread.so.0

                        #2  0x00002b0df2842f03 in pthread_mutex_lock ()
                        from /lib64/libpthread.so.0

                        #3  0x00002b0df2204289 in PR_Lock () from
                        /usr/lib64/libnspr4.so

                        #4  0x0000000000415955 in ber_sockbuf_free ()

                        #5  0x00000000004166f1 in ber_sockbuf_free ()

                        #6  0x000000000041d7d4 in ?? ()

                        #7  0x00002b0df2a739c4 in __libc_start_main ()
                        from /lib64/libc.so.6

                        #8  0x000000000040c9b9 in ber_sockbuf_free ()

                        #9  0x00007fff5cd57a98 in ?? ()

                        #10 0x0000000000000000 in ?? ()

                        
                    --
389 users mailing list
389-users@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/389-users
                  
              
            --

            389 users mailing list

            389-users@xxxxxxxxxxxxxxxxxxxxxxx

            https://admin.fedoraproject.org/mailman/listinfo/389-users

          
      --
389 users mailing list
389-users@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/389-users
    
    
--
389 users mailing list
389-users@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/389-users