Hello,
I am using pjlib (namely the NAT framework) in an application I'm writing. So far it is working great, with one exception. I have incorporated much of the ice_demo.c example code, of course making some changes to make it usable from a C++ class.
I get a deadlock from within the library that seems to result from a race of some sort (it does not happen all the time). When the deadlock does not occur, the application works perfectly otherwise. The problem occurs, it seems, when pjlib calls back into the application upon receiving a packet. My callback blocks waiting for another thread to release my mutex:
#0 __lll_lock_wait() at /usr/lib/libpthread.so.0
#1 _L_lock_553() at /usr/lib/libpthread.so.0
#2 pthread_mutex_lock() at /usr/lib/libpthread.so.0
#3 __gthread_mutex_lock() at /usr/include/c++/4.9.0/x86_64-unknown-linux-gnu/bits/gthr-default.h:748
#4 std::mutex::lock() at /usr/include/c++/4.9.0/mutex:134
#5 std::lock_guard<std::mutex>::lock_guard() at /usr/include/c++/4.9.0/mutex:376
(snip)
#13 IceLink::cb_on_rx_data() at /home/ghilliard/Code/pseudotcp/IceLink.cpp:213
#14 cb_on_rx_data_stub() at /home/ghilliard/Code/pseudotcp/IceLink.cpp:164
#15 pj_ice_sess_on_rx_pkt() at /home/ghilliard/Code/pjproject/src/pjproject-2.2.1/pjnath/src/pjnath/ice_session.c:3008
#16 stun_on_rx_data() at /home/ghilliard/Code/pjproject/src/pjproject-2.2.1/pjnath/src/pjnath/ice_strans.c:1446
#17 on_data_recvfrom() at /home/ghilliard/Code/pjproject/src/pjproject-2.2.1/pjnath/src/pjnath/stun_sock.c:957
#18 ioqueue_on_read_complete() at /home/ghilliard/Code/pjproject/src/pjproject-2.2.1/pjlib/src/pj/activesock.c:495
#19 ioqueue_dispatch_read_event() at /home/ghilliard/Code/pjproject/src/pjproject-2.2.1/pjlib/src/pj/ioqueue_common_abs.c:591
#20 pj_ioqueue_poll() at /home/ghilliard/Code/pjproject/src/pjproject-2.2.1/pjlib/src/pj/ioqueue_select.c:963
#21 IceLink::handle_events() at /home/ghilliard/Code/pseudotcp/IceLink.cpp:415
(snip)
My mutex is locked by another thread which is attempting to call into PJLIB to send a packet. This send thread blocks on the grp_lock object:
#0 __lll_lock_wait() at /usr/lib/libpthread.so.0
#1 _L_lock_598() at /usr/lib/libpthread.so.0
#2 pthread_mutex_lock() at /usr/lib/libpthread.so.0
#3 pj_mutex_lock() at /home/ghilliard/Code/pjproject/src/pjproject-2.2.1/pjlib/src/pj/os_core_unix.c:1243
#4 grp_lock_acquire() at /home/ghilliard/Code/pjproject/src/pjproject-2.2.1/pjlib/src/pj/lock.c:290
#5 pj_grp_lock_acquire() at /home/ghilliard/Code/pjproject/src/pjproject-2.2.1/pjlib/src/pj/lock.c:437
#6 pj_ice_sess_send_data() at /home/ghilliard/Code/pjproject/src/pjproject-2.2.1/pjnath/src/pjnath/ice_session.c:2903
#7 pj_ice_strans_sendto() at /home/ghilliard/Code/pjproject/src/pjproject-2.2.1/pjnath/src/pjnath/ice_strans.c:1197
#8 IceLink::icedemo_send_data() at /home/ghilliard/Code/pseudotcp/IceLink.cpp:702
(snip)
Inspection of the object indicates that it is locked by the first thread, even though that thread has released the lock (in ice_session.c:3004)! It seems that the object still is "level 2" as printed by a debug message with verbose logging:
Mutex released by thread thr0x7f446b6b8700 (level=2)
This is the last mention of that object in the log; these two threads are the only ones with pjlib in their backtraces. The only thing that I can think that might be wrong with my code is that I am playing somewhat fast and loose with the pjlib thread registration function. My layer is designed so that any thread might call its send functions, so I'm currently doing this at the beginning of any function that calls into pjlib:
pj_thread_t *thread;
if(!pj_thread_is_registered()) {
pj_thread_desc d;
pj_thread_register(0, d, &thread);
}
(My reasoning being that once the call is done, pjlib will never need to reference the thread again.) Am I doing all this correctly? Any guidance would be very appreciated.
Thanks,
George Hilliard
