Hello,
We have found probably a bug in ptlib during testing the t38modem
application (ptlib+h323plus+t38modem).
We got this backtrace:
root@maxi:/tmp# gdb /usr/sbin/t38modem core.6382.104
GNU gdb 6.4-debian
Copyright 2005 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you
are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu"...Using host libthread_db
library "/lib/libthread_db.so.1".
Failed to read a valid object file image from memory.
Core was generated by `/usr/sbin/t38modem -p
ttyc0,ttyc1,ttyc2,ttyc3,ttyc4,ttyc5,ttyc6,ttyc7,ttyc8,tty'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /lib/libresolv.so.2...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /usr/lib/libldap_r.so.2...done.
Loaded symbols for /usr/lib/libldap_r.so.2
Reading symbols from /usr/lib/liblber.so.2...done.
Loaded symbols for /usr/lib/liblber.so.2
Reading symbols from /lib/libpthread.so.0...done.
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /usr/lib/libsasl2.so.2...done.
Loaded symbols for /usr/lib/libsasl2.so.2
Reading symbols from /usr/lib/libssl.so.0.9.8...done.
Loaded symbols for /usr/lib/libssl.so.0.9.8
Reading symbols from /usr/lib/libcrypto.so.0.9.8...done.
Loaded symbols for /usr/lib/libcrypto.so.0.9.8
Reading symbols from /usr/lib/libexpat.so.1...done.
Loaded symbols for /usr/lib/libexpat.so.1
Reading symbols from /usr/lib/libSDL-1.2.so.0...done.
Loaded symbols for /usr/lib/libSDL-1.2.so.0
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /usr/lib/libstdc++.so.6...done.
Loaded symbols for /usr/lib/libstdc++.so.6
Reading symbols from /lib/libm.so.6...done.
Loaded symbols for /lib/libm.so.6
Reading symbols from /lib/libgcc_s.so.1...done.
Loaded symbols for /lib/libgcc_s.so.1
Reading symbols from /lib/libc.so.6...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/libcrypt.so.1...done.
Loaded symbols for /lib/libcrypt.so.1
Reading symbols from /usr/lib/libgnutls.so.12...done.
Loaded symbols for /usr/lib/libgnutls.so.12
Reading symbols from /lib/ld-linux-x86-64.so.2...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /usr/lib/libz.so.1...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /usr/lib/libasound.so.2...done.
Loaded symbols for /usr/lib/libasound.so.2
Reading symbols from /usr/lib/libX11.so.6...done.
Loaded symbols for /usr/lib/libX11.so.6
Reading symbols from /usr/lib/libXext.so.6...done.
Loaded symbols for /usr/lib/libXext.so.6
Reading symbols from /usr/lib/libtasn1.so.2...done.
Loaded symbols for /usr/lib/libtasn1.so.2
Reading symbols from /usr/lib/libgcrypt.so.11...done.
Loaded symbols for /usr/lib/libgcrypt.so.11
Reading symbols from /usr/lib/libgpg-error.so.0...done.
Loaded symbols for /usr/lib/libgpg-error.so.0
Reading symbols from /usr/lib/libXau.so.6...done.
Loaded symbols for /usr/lib/libXau.so.6
Reading symbols from /lib/libnsl.so.1...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /lib/libnss_compat.so.2...done.
Loaded symbols for /lib/libnss_compat.so.2
Reading symbols from /lib/libnss_nis.so.2...done.
Loaded symbols for /lib/libnss_nis.so.2
Reading symbols from /lib/libnss_files.so.2...done.
Loaded symbols for /lib/libnss_files.so.2
#0 0x00002b3e71de4312 in __gnu_cxx::__exchange_and_add () from
/usr/lib/libstdc++.so.6
(gdb) bt full
#0 0x00002b3e71de4312 in __gnu_cxx::__exchange_and_add () from
/usr/lib/libstdc++.so.6
No symbol table info available.
#1 0x00000000005c4ee1 in PAtomicInteger::operator++ (this=0x4) at
critsec.h:243
No locals.
#2 0x000000000077cf8a in PContainer (this=0x40cf2080, cont=@0xafc3c0)
at ../common/contain.cxx:739
No locals.
#3 0x000000000046fdbd in PAbstractArray (this=0x40cf2080, c=@0xafc3c0)
at array.h:183
No locals.
#4 0x000000000047a19b in PBaseArray (this=0x40cf2080) at array.h:390
No locals.
#5 0x0000000000789315 in PCharArray (this=0x40cf2080) at array.h:733
No locals.
#6 0x000000000077d571 in PString (this=0x40cf2080, str=@0xafc3c0) at
contain.inl:241
No locals.
#7 0x000000000075c559 in PThread::PX_ThreadEnd (arg=0xafc3b0) at
tlibthrd.cxx:1402
threadName = {<PCharArray> = {<PBaseArray<char>> =
{<PAbstractArray> = {<PContainer> = {<PObject> = {_vptr.PObject =
0x870d90}, reference = 0xf0}, elementSize = 0,
theArray = 0x100000000 <Address 0x100000000 out of bounds>,
allocatedDynamically = 1}, <No data fields>}, <No data fields>}, <No
data fields>}
process = (class PProcess &) @0xabc420: {<PThread> = {<PObject> =
{_vptr.PObject = 0x7bfad0}, autoDelete = 0,
threadName = {<PCharArray> = {<PBaseArray<char>> = {<PAbstractArray>
= {<PContainer> = {<PObject> = {_vptr.PObject = 0x8710b0}, reference =
0xabaf90}, elementSize = 1, theArray = 0xabc970 "",
allocatedDynamically = 1}, <No data fields>}, <No data
fields>}, <No data fields>},
traceStream = {<PString> = {<PCharArray> = {<PBaseArray<char>> =
{<PAbstractArray> = {<PContainer> = {<PObject> = {_vptr.PObject =
0x871458}, reference = 0xac2530}, elementSize = 1,
theArray = 0xabc990 "", allocatedDynamically = 1}, <No data
fields>}, <No data fields>}, <No data fields>}, <> = {<No data fields>},
<No data fields>}, traceLevel = 0,
traceBlockIndentLevel = 0, PX_origStackSize = 0, PX_priority =
PThread::NormalPriority, PX_threadId = 47547219615632, PX_suspendMutex =
{__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0,
__kind = 0, __spins = 0}, __size = '\0' <repeats 39 times>,
__align = 0}, PX_suspendCount = 0, PX_firstTimeStart = 0, ending = 0,
unblockPipe = {3, 4}}, static p_argc = 21,
static p_argv = 0x7fff39b8a398, static p_envp = 0x7fff39b8a448,
terminationValue = 0, manufacturer = {<PCharArray> = {<PBaseArray<char>>
= {<PAbstractArray> = {<PContainer> = {<PObject> = {
_vptr.PObject = 0x8710b0}, reference = 0xabc9b0},
elementSize = 1, theArray = 0xabcb40 "OpenH323 Project",
allocatedDynamically = 1}, <No data fields>}, <No data fields>}, <No
data fields>},
productName = {<PCharArray> = {<PBaseArray<char>> = {<PAbstractArray> =
{<PContainer> = {<PObject> = {_vptr.PObject = 0x8710b0}, reference =
0xabcb60}, elementSize = 1, theArray = 0xabcb80 "T38Modem",
allocatedDynamically = 1}, <No data fields>}, <No data
fields>}, <No data fields>}, majorVersion = 1, minorVersion = 0, status
= PProcess::ReleaseCode, buildNumber = 0,
executableFile = {<PString> = {<PCharArray> = {<PBaseArray<char>> =
{<PAbstractArray> = {<PContainer> = {<PObject> = {_vptr.PObject =
0x8691d0}, reference = 0xabcba0}, elementSize = 1,
theArray = 0xabcbc0 "", allocatedDynamically = 1}, <No data
fields>}, <No data fields>}, <No data fields>}, <No data fields>},
configurationPaths = {<PList<PString>> = {<PAbstractList> =
{<PCollection> = {<PContainer> = {<PObject> = {_vptr.PObject =
0x871670}, reference = 0xabcbe0}, <No data fields>},
info = 0xabcc00}, <No data fields>}, <No data fields>}, arguments
= {<PObject> = {_vptr.PObject = 0x86f190},
argumentArray = {<PArray<PString>> = {<PArrayObjects> =
{<PCollection> = {<PContainer> = {<PObject> = {_vptr.PObject =
0x871590}, reference = 0xabcee0}, <No data fields>},
theArray = 0xabcf00}, <No data fields>}, <No data fields>},
optionLetters = {<PCharArray> = {<PBaseArray<char>> = {<PAbstractArray>
= {<PContainer> = {<PObject> = {_vptr.PObject = 0x8710b0},
reference = 0xabcc30}, elementSize = 1, theArray = 0xabcc80
"", allocatedDynamically = 1}, <No data fields>}, <No data fields>}, <No
data fields>},
optionNames = {<PArray<PString>> = {<PArrayObjects> = {<PCollection>
= {<PContainer> = {<PObject> = {_vptr.PObject = 0x871590}, reference =
0xabcce0}, <No data fields>},
theArray = 0xabcd00}, <No data fields>}, <No data fields>},
optionCount = {<PBaseArray<int>> = {<PAbstractArray> = {<PContainer> =
{<PObject> = {_vptr.PObject = 0x7bfbf0}, reference = 0xabcd50},
elementSize = 4, theArray = 0x0, allocatedDynamically = 1}, <No
data fields>}, <No data fields>},
optionString = {<PArray<PString>> = {<PArrayObjects> = {<PCollection>
= {<PContainer> = {<PObject> = {_vptr.PObject = 0x871590}, reference =
0xabcd70}, <No data fields>},
theArray = 0xabcd90}, <No data fields>}, <No data fields>},
parameterIndex = {<PBaseArray<int>> = {<PAbstractArray> = {<PContainer>
= {<PObject> = {_vptr.PObject = 0x7bfbf0},
reference = 0xabcde0}, elementSize = 4, theArray = 0xabd960
"", allocatedDynamically = 1}, <No data fields>}, <No data fields>},
shift = 0},
timers = {<PList<PTimer>> = {<PAbstractList> = {<PCollection> =
{<PContainer> = {<PObject> = {_vptr.PObject = 0x86a0b0}, reference =
0xabce00}, <No data fields>}, info = 0xabce20}, <No data fields>},
listMutex = {<PSync> = {<PObject> = {_vptr.PObject = 0x869d90},
lockerId = 1092376928}, mutex = {__data = {__lock = 0, __count = 0,
__owner = 0, __nusers = 0, __kind = 1, __spins = 0},
__size = '\0' <repeats 16 times>, "\001", '\0' <repeats 22
times>, __align = 0}}, processingMutex = {<PSync> = {<PObject> =
{_vptr.PObject = 0x869d90}, lockerId = 1092376928}, mutex = {__data = {
__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 1,
__spins = 0}, __size = '\0' <repeats 16 times>, "\001", '\0' <repeats 22
times>, __align = 0}},
inTimeoutMutex = {<PSync> = {<PObject> = {_vptr.PObject = 0x869d90},
lockerId = 1092376928}, mutex = {__data = {__lock = 0, __count = 0,
__owner = 0, __nusers = 0, __kind = 1, __spins = 0},
__size = '\0' <repeats 16 times>, "\001", '\0' <repeats 22
times>, __align = 0}}, lastSample = {<PObject> = {_vptr.PObject =
0x868990}, milliseconds = 1203507183364}, currentTimer = 0x0},
programStartTime = {<PObject> = {_vptr.PObject = 0x868a10}, theTime =
1203506350, microseconds = 785187}, maxHandles = 2048, pxSignals = 0,
configFiles = 0xabdab0,
activeThreads = {<PAbstractDictionary> = {<PHashTable> = {<PCollection>
= {<PContainer> = {<PObject> = {_vptr.PObject = 0x869f10}, reference =
0xabce50}, <No data fields>},
hashTable = 0xabce70}, <No data fields>}, <No data fields>},
threadMutex = {<PSync> = {<PObject> = {_vptr.PObject = 0x869d90},
lockerId = 1087318368}, mutex = {__data = {__lock = 2, __count = 1,
__owner = 21470, __nusers = 1, __kind = 1, __spins = 0}, __size =
"\002\000\000\000\001\000\000\000�S\000\000\001\000\000\000\001", '\0'
<repeats 22 times>, __align = 4294967298}},
breakBlock = {<PSemaphore> = {<PSync> = {<PObject> = {_vptr.PObject =
0x869e10}, lockerId = 18446744073709551615}, initialVar = 0, maxCountVar
= 0, pxClass = PSemaphore::PXSyncPoint, mutex = {__data = {
__lock = 0, __count = 0, __owner = 0, __nusers = 1, __kind = 0,
__spins = 0}, __size = '\0' <repeats 12 times>, "\001", '\0' <repeats 26
times>, __align = 0}, condVar = {__data = {__lock = 0,
__futex = 1659, __total_seq = 830, __wakeup_seq = 829,
__woken_seq = 829, __mutex = 0xabc8d8, __nwaiters = 2, __broadcast_seq =
0},
__size =
"\000\000\000\000{\006\000\000>\003\000\000\000\000\000\000=\003\000\000\000\000\000\000=\003\000\000\000\000\000\000�ȫ\000\000\000\000\000\002\000\000\000\000\000\000",
__align = 7125350744064}, semId = {__size = '\0' <repeats 31
times>, __align = 0}}, signalCount = 0}, housekeepingThread = 0xaedfc0}
thread = (PThread *) 0xafc3b0
id = 1087318368
#8 0x000000000075c35e in PThread::PX_ThreadStart (arg=0xafc3b0) at
tlibthrd.cxx:1364
__cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf = {7805982,
1087316448, 47547187568752, 1091568592, 0, 1091568592, 1087316224,
7717523}, __mask_was_saved = 0}}, __pad = {0x40cf2200, 0x0, 0x1,
0x2b3e714a46c0}}
---Type <return> to continue, or q <return> to quit---
__cancel_routine = (void (*)(void *)) 0x75c36e
<PThread::PX_ThreadEnd(void*)>
__cancel_arg = (void *) 0xafc3b0
not_first_call = 0
thread = (PThread *) 0xafc3b0
#9 0x00002b3e7139a0fa in start_thread () from /lib/libpthread.so.0
No symbol table info available.
#10 0x00002b3e72287ce2 in clone () from /lib/libc.so.6
No symbol table info available.
#11 0x0000000000000000 in ?? ()
No symbol table info available.
(gdb)
After long debugging we have found that the issue is in ptlib and the
following patch fixes it:
--- ptlib_v2_0_1/src/ptlib/unix/tlibthrd.cxx.orig 2008-03-19
22:13:56.000000000 +0100
+++ ptlib_v2_0_1/src/ptlib/unix/tlibthrd.cxx 2008-03-20
09:25:41.000000000 +0100
@@ -1387,10 +1387,11 @@
// delete the thread if required, note this is done this way to avoid
// a race condition, the thread ID cannot be zeroed before the if!
+ PString threadName = thread->threadName;
if (thread->autoDelete) {
thread->PX_threadId = 0; // Prevent terminating terminated thread
process.threadMutex.Signal();
- PTRACE(5, "PWLib\tEnded thread " << thread << ' ' <<
thread->threadName);
+ PTRACE(5, "PWLib\tEnded thread " << thread << ' ' << threadName);
/* It is now safe to delete this thread. Note that this thread
is deleted after the process.threadMutex.Signal(), which means
@@ -1399,7 +1400,6 @@
}
else {
thread->PX_threadId = 0;
- PString threadName = thread->threadName;
process.threadMutex.Signal();
PTRACE(5, "PWLib\tEnded thread " << thread << ' ' << threadName);
}
It seems to be a race condition similar to what have been resolved
earlier (see the comments and cvs history).
After testing the patched version the issue seems to be over.
I would like thank Simon for helping with the debugging and taking care
of t38modem for h323plus.
Kind regards,
Tamas
ps: system info:
ptlib-2.0.1, h323plus-1.20.2, t38modem from h323plus contrib. Ubuntu
Dapper Drake LTS (6.06), x86_64,
Linux maxi 2.6.23.9 #1 SMP PREEMPT Thu Dec 6 11:32:16 CET 2007 x86_64
GNU/Linux
