FWIW, from your stack trace, it looks like something happened between the original call to pj_mutex_lock() and pj_mutex_unlock(), which makes PJLIB thinks that the thread no longer holds the mutex. This could be caused by different thread is calling unlock(), or something wipes out the thread descriptor, for example a memory overwrite bug, or maybe a call to pj_thread_destroy(). Best regards, ?Benny On Tue, Sep 14, 2010 at 2:56 PM, Adrian Georgescu <ag at ag-projects.com> wrote: > Hi Benny, > > It is true that we have done modifications though in parts not related with this and your support was always to the point. > > Trying to isolate the possible cause for this illusive problem that I still need to solve, I wish to ask the list if anyone else is using more than one conference bridge at a time in their application and if so if they encountered such a problem? > > An excerpt of the problem is here: > > Exception Type: ?EXC_CRASH (SIGABRT) > Exception Codes: 0x0000000000000000, 0x0000000000000000 > Crashed Thread: ?6 > > Application Specific Information: > Assertion failed: (mutex->owner == pj_thread_this()), function pj_mutex_unlock, file ../src/pj/os_core_unix.c, line 1234. > > Thread 6 Crashed: > 0 ? libSystem.B.dylib ? ? ? ? ? ? ? ? ? 0x929b728e __semwait_signal_nocancel + 10 > 1 ? libSystem.B.dylib ? ? ? ? ? ? ? ? ? 0x929b7172 nanosleep$NOCANCEL$UNIX2003 + 166 > 2 ? libSystem.B.dylib ? ? ? ? ? ? ? ? ? 0x92a3203a usleep$NOCANCEL$UNIX2003 + 61 > 3 ? libSystem.B.dylib ? ? ? ? ? ? ? ? ? 0x92a536f0 abort + 105 > 4 ? libSystem.B.dylib ? ? ? ? ? ? ? ? ? 0x92a4028c __pthread_markcancel + 0 > 5 ? _core.so ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x05b7870b pj_mutex_unlock + 283 > 6 ? _core.so ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x05aeb3ec get_frame + 556 > 7 ? _core.so ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x05afc767 play_cb + 71 > 8 ? _core.so ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x05b0dc70 PaPlayerCallback + 416 > 9 ? _core.so ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x05b72a18 AdaptingOutputOnlyProcess + 216 > 10 ?_core.so ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x05b734b6 PaUtil_EndBufferProcessing + 550 > 11 ?_core.so ? ? ? ? ? ? ? ? ? ? ? ? ? ?0x05b6b05c AudioIOProc + 2652 > 12 ?...pple.audio.units.Components ? ? ?0x70008c2f 0x70000000 + 35887 > 13 ?...pple.audio.units.Components ? ? ?0x70009830 0x70000000 + 38960 > 14 ?...pple.audio.units.Components ? ? ?0x70008d08 0x70000000 + 36104 > 15 ?....audio.toolbox.AudioToolbox ? ? ?0x92245f5b AudioConverterChain::CallInputProc(unsigned long) + 881 > 16 ?....audio.toolbox.AudioToolbox ? ? ?0x92245a83 AudioConverterChain::FillBufferFromInputProc(unsigned long*, CABufferList*) + 95 > 17 ?....audio.toolbox.AudioToolbox ? ? ?0x92245a11 BufferedAudioConverter::GetInputBytes(unsigned long, unsigned long&, CABufferList const*&) + 199 > 18 ?....audio.toolbox.AudioToolbox ? ? ?0x922458cd CBRConverter::RenderOutput(CABufferList*, unsigned long, unsigned long&, AudioStreamPacketDescription*) + 129 > 19 ?....audio.toolbox.AudioToolbox ? ? ?0x92245677 BufferedAudioConverter::FillBuffer(unsigned long&, AudioBufferList&, AudioStreamPacketDescription*) + 315 > 20 ?....audio.toolbox.AudioToolbox ? ? ?0x922457df AudioConverterChain::RenderOutput(CABufferList*, unsigned long, unsigned long&, AudioStreamPacketDescription*) + 157 > 21 ?....audio.toolbox.AudioToolbox ? ? ?0x92245677 BufferedAudioConverter::FillBuffer(unsigned long&, AudioBufferList&, AudioStreamPacketDescription*) + 315 > 22 ?....audio.toolbox.AudioToolbox ? ? ?0x922453ec AudioConverterFillComplexBuffer + 268 > 23 ?...pple.audio.units.Components ? ? ?0x70008df5 0x70000000 + 36341 > 24 ?...pple.audio.units.Components ? ? ?0x700082b4 0x70000000 + 33460 > 25 ?...pple.audio.units.Components ? ? ?0x70006b64 0x70000000 + 27492 > 26 ?...pple.audio.units.Components ? ? ?0x700057c3 0x70000000 + 22467 > 27 ?...pple.audio.units.Components ? ? ?0x7000b9fb AUGenericOutputEntry + 7172 > 28 ?com.apple.audio.CoreAudio ? ? ? ? ? 0x9795b454 HP_IOProc::Call(AudioTimeStamp const&, AudioTimeStamp const&, AudioBufferList const*, AudioTimeStamp const&, AudioBufferList*) + 374 > 29 ?com.apple.audio.CoreAudio ? ? ? ? ? 0x9795b1ba IOA_Device::CallIOProcs(AudioTimeStamp const&, AudioTimeStamp const&, AudioTimeStamp const&) + 370 > 30 ?com.apple.audio.CoreAudio ? ? ? ? ? 0x9795afb6 HP_IOThread::PerformIO(AudioTimeStamp const&, double) + 620 > 31 ?com.apple.audio.CoreAudio ? ? ? ? ? 0x9795839a HP_IOThread::WorkLoop() + 2506 > 32 ?com.apple.audio.CoreAudio ? ? ? ? ? 0x979579cb HP_IOThread::ThreadEntry(HP_IOThread*) + 17 > 33 ?com.apple.audio.CoreAudio ? ? ? ? ? 0x979578e2 CAPThread::Entry(CAPThread*) + 140 > 34 ?libSystem.B.dylib ? ? ? ? ? ? ? ? ? 0x9297781d _pthread_start + 345 > 35 ?libSystem.B.dylib ? ? ? ? ? ? ? ? ? 0x929776a2 thread_start + 34 > > > Regards, > Adrian > > On Sep 14, 2010, at 1:02 AM, Benny Prijono wrote: > >> On Mon, Sep 13, 2010 at 5:44 PM, Adrian Georgescu <ag at ag-projects.com> wrote: >>> >>> We are using this model in SIP SIMPLE client SDK and ?Blink (multiple conference bridges) and we run into random crashes caused by multithreading issues. Hard to debug and impossible to reproduce deterministically but its happens sooner or later after running for awhile. Probably nobody else have done this before in production environments and this scenario was never properly tested, this is why I am mentioning it here. >>> >> >> Hi Adrian, >> >> True I haven't tried this scenario myself, but conferencing is one of >> the thing that's central to PJSUA-LIB's design, hence it's been used >> by everybody and I think it's fair to say it's stable. Now when you're >> supposed to just create multiple instances of the same thing, I would >> expect it to stay stable if done properly. >> >> And we've discussed your problem few times off the list. Apart from >> this, you also have done major modifications elsewhere to the library, >> so it could be that too. And all this time, I never saw a crash dump, >> so it's impossible to say what this is. >> >> Best regards >> Benny >> >> _______________________________________________ >> Visit our blog: http://blog.pjsip.org >> >> pjsip mailing list >> pjsip at lists.pjsip.org >> http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org >> > > > _______________________________________________ > Visit our blog: http://blog.pjsip.org > > pjsip mailing list > pjsip at lists.pjsip.org > http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org >
