Hi guys, I have a small example (reduced from a large server application, that fork()s and pthread_create()s a lot) where I experience a mudflapth problem, attached. The program is linked with the mudflapth library using the commands below. It is a PostgreSQL based application and I only see the problem in the small example when accessing the database (opened or not), but I doubt that it has something to do with PostgreSQL, rather some (stupid) little thing I do wrong. I just don't seem to be able to deduct anything sensible from the dumps. Mudflap catches the 'error' during the pthread_cancel() call. I have of course asked on the PostgreSQL list, but none of these guys seems too knowledgeable about mudflap ... perhaps just better programmers than me, not needing that kind of stuff ;-) The delay just before the pthread_cancel(), i.e. sleep(10), is rather critical for the problem to appear and you might have to change it to something less. On some very slow machines I wasn't able to produce the problem. I ended up using the latest gcc version, gcc-4.4.0. All machines I have tested on is running some version of kernel 2.6 with nptl thread library. In the large application I get spurious SIGSEGV (or SIGABRT with mudflap) and in most of these cases my stack is corrupted. That's why looked into using mudflap. I am now also using -fstack-protector, but that does not catch any errors. $ ecpg crashex.pgc $ /usr/local/Packages/gcc-4.4.0/bin/gcc -O0 -c -fmudflap -fmudflapth -fomit-frame-pointer -B/usr/local/Packages/gcc-4.4.0/bin/ -Wwrite-strings -std=gnu89 -ggdb -fPIC -Wall -I/usr/local/Packages/pgsql/include -I./Modules -I./ -o crashex.o crashex.c $ /usr/local/Packages/gcc-4.4.0/bin/gcc -O0 -B/usr/local/Packages/gcc-4.4.0/bin/ -Wl -o crashex crashex.o -L/usr/local/Packages/pgsql/lib -lecpg -lpq -lmudflapth -lpthread -ldl And this is the output from running the program: leif$ LD_LIBRARY_PATH=/usr/local/Packages/gcc-4.4.0/lib/ ./crashex Couldn't open somename@localhost:5432 2+2=0. *** glibc detected *** /home/leif/tmp/crashex: free(): invalid pointer: 0x081f3958 *** ======= Backtrace: ========= /lib32/libc.so.6[0xf7c30615] /lib32/libc.so.6(cfree+0x90)[0xf7c34080] /usr/local/Packages/gcc-4.4.0/lib/libmudflapth.so.0(__real_free+0x3f1)[0xf7d39061] /lib32/libecpg.so.6[0xf7e3fb5c] /lib32/libpthread.so.0[0xf7d1bbb0] /lib32/libpthread.so.0[0xf7d1c509] /lib32/libc.so.6(clone+0x5e)[0xf7c9d08e] ======= Memory map: ======== 08048000-0804a000 r-xp 00000000 08:0a 1671173 /home/leif/tmp/crashex 0804a000-0804b000 rwxp 00001000 08:0a 1671173 /home/leif/tmp/crashex 0804b000-081f8000 rwxp 0804b000 00:00 0 [heap] f71eb000-f71ec000 ---p f71eb000 00:00 0 f71ec000-f79ec000 rwxp f71ec000 00:00 0 f79ec000-f79f5000 r-xp 00000000 08:01 97934 /lib32/libnss_files-2.7.so f79f5000-f79f7000 rwxp 00008000 08:01 97934 /lib32/libnss_files-2.7.so f79f7000-f79ff000 r-xp 00000000 08:01 97936 /lib32/libnss_nis-2.7.so f79ff000-f7a01000 rwxp 00007000 08:01 97936 /lib32/libnss_nis-2.7.so f7a01000-f7a15000 r-xp 00000000 08:01 97931 /lib32/libnsl-2.7.so f7a15000-f7a17000 rwxp 00013000 08:01 97931 /lib32/libnsl-2.7.so f7a17000-f7a19000 rwxp f7a17000 00:00 0 f7a19000-f7a20000 r-xp 00000000 08:01 97932 /lib32/libnss_compat-2.7.so f7a20000-f7a22000 rwxp 00006000 08:01 97932 /lib32/libnss_compat-2.7.so f7a22000-f7a2c000 r-xp 00000000 08:08 227374 /usr/lib32/libgcc_s.so.1 f7a2c000-f7a2d000 rwxp 0000a000 08:08 227374 /usr/lib32/libgcc_s.so.1 f7a2d000-f7a2f000 rwxp f7a2d000 00:00 0 f7a2f000-f7a38000 r-xp 00000000 08:01 97927 /lib32/libcrypt-2.7.so f7a38000-f7a3a000 rwxp 00008000 08:01 97927 /lib32/libcrypt-2.7.so f7a3a000-f7a61000 rwxp f7a3a000 00:00 0 f7a61000-f7b4a000 r-xp 00000000 08:01 98015 /lib32/libcrypto.so.0.9.7 f7b4a000-f7b5c000 rwxp 000e8000 08:01 98015 /lib32/libcrypto.so.0.9.7 f7b5c000-f7b5f000 rwxp f7b5c000 00:00 0 f7b5f000-f7b8d000 r-xp 00000000 08:01 98021 /lib32/libssl.so.0.9.7 f7b8d000-f7b90000 rwxp 0002d000 08:01 98021 /lib32/libssl.so.0.9.7 f7b90000-f7bb3000 r-xp 00000000 08:01 97929 /lib32/libm-2.7.so f7bb3000-f7bb5000 rwxp 00023000 08:01 97929 /lib32/libm-2.7.so f7bb5000-f7bb6000 rwxp f7bb5000 00:00 0 f7bb6000-f7bc2000 r-xp 00000000 08:01 97971 /lib32/libpgtypes.so.3.0 f7bc2000-f7bc4000 rwxp 0000c000 08:01 97971 /lib32/libpgtypes.so.3.0 f7bc4000-f7d0d000 r-xp 00000000 08:01 97925 /lib32/libc-2.7.so f7d0d000-f7d0e000 r-xp 00149000 08:01 97925 /lib32/libc-2.7.so f7d0e000-f7d10000 rwxp 0014a000 08:01 97925 /lib32/libc-2.7.so f7d10000-f7d13000 rwxp f7d10000 00:00 0 f7d13000-f7d15000 r-xp 00000000 08:01 97928 /lib32/libdl-2.7.so f7d15000-f7d17000 rwxp 00001000 08:01 97928 /lib32/libdl-2.7.so f7d17000-f7d2b000 r-xp 00000000 08:01 97939 /lib32/libpthread-2.7.so f7d2b000-f7d2d000 rwxp 00013000 08:01 97939 /lib32/libpthread-2.7.so f7d2d000-f7d2f000 rwxp f7d2d000 00:00 0 f7d2f000-f7d49000 r-xp 00000000 08:09 934128 /usr/local/Packages/gcc-4.4.0/lib/libmudflapth.so.0.0.0 f7d49000-f7d4c000 rwxp 0001a000 08:09 934128 /usr/local/Packages/gcc-4.4.0/lib/libmudflapth.so.0.0.0 f7d4c000-f7e1b000 rwxp f7d4c000 00:00 0 f7e1b000-f7e35000 r-xp 00000000 08:01 98012 /lib32/libpq.so.5.0 f7e35000-f7e36000 rwxp 0001a000 08:01 98012 /lib32/libpq.so.5.0 f7e36000-f7e44000 r-xp 00000000 08:01 97969 /lib32/libecpg.so.6.0 f7e44000-f7f05000 rwxp 0000d000 08:01 97969 /lib32/libecpg.so.6.0 f7f18000-f7f1b000 rwxp f7f18000 00:00 0 f7f1b000-f7f38000 r-xp 00000000 08:01 97922 /lib32/ld-2.7.so f7f38000-f7f3a000 rwxp 0001c000 08:01 97922 /lib32/ld-2.7.so fff3f000-fff59000 rwxp 7ffffffe5000 00:00 0 [stack] ffffe000-fffff000 r-xp ffffe000 00:00 0 [vdso] Aborted (core dumped) leif$ gdb ~/tmp/crashex core.30920 GNU gdb 6.8-debian Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu"... warning: Can't read pathname for load map: Input/output error. Reading symbols from /lib32/libecpg.so.6...done. Loaded symbols for /lib32/libecpg.so.6 Reading symbols from /lib32/libpq.so.5...done. Loaded symbols for /lib32/libpq.so.5 Reading symbols from /usr/local/Packages/gcc-4.4.0/lib/libmudflapth.so.0...done. Loaded symbols for /usr/local/Packages/gcc-4.4.0/lib/libmudflapth.so.0 Reading symbols from /lib32/libpthread.so.0...done. Loaded symbols for /lib32/libpthread.so.0 Reading symbols from /lib32/libdl.so.2...done. Loaded symbols for /lib32/libdl.so.2 Reading symbols from /lib32/libc.so.6...done. Loaded symbols for /lib32/libc.so.6 Reading symbols from /lib32/libpgtypes.so.3...done. Loaded symbols for /lib32/libpgtypes.so.3 Reading symbols from /lib32/libm.so.6...done. Loaded symbols for /lib32/libm.so.6 Reading symbols from /lib32/libssl.so.0...done. Loaded symbols for /lib32/libssl.so.0 Reading symbols from /lib32/libcrypto.so.0...done. Loaded symbols for /lib32/libcrypto.so.0 Reading symbols from /lib32/libcrypt.so.1...done. Loaded symbols for /lib32/libcrypt.so.1 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /usr/lib32/libgcc_s.so.1...done. Loaded symbols for /usr/lib32/libgcc_s.so.1 Reading symbols from /lib32/libnss_compat.so.2...done. Loaded symbols for /lib32/libnss_compat.so.2 Reading symbols from /lib32/libnsl.so.1...done. Loaded symbols for /lib32/libnsl.so.1 Reading symbols from /lib32/libnss_nis.so.2...done. Loaded symbols for /lib32/libnss_nis.so.2 Reading symbols from /lib32/libnss_files.so.2...done. Loaded symbols for /lib32/libnss_files.so.2 warning: Lowest section in system-supplied DSO at 0xffffe000 is .hash at ffffe0b4 Program terminated with signal 6, Aborted. [New process 30922] [New process 30920] #0 0xffffe405 in __kernel_vsyscall () (gdb) bt #0 0xffffe405 in __kernel_vsyscall () #1 0xf7bef335 in raise () from /lib32/libc.so.6 #2 0xf7bf0cb1 in abort () from /lib32/libc.so.6 #3 0xf7c286ec in ?? () from /lib32/libc.so.6 #4 0xf7c30615 in ?? () from /lib32/libc.so.6 #5 0xf7c34080 in free () from /lib32/libc.so.6 #6 0xf7d39061 in free (buf=0x81f3958) at ../../../libmudflap/mf-hooks1.c:241 #7 0xf7e3fb5c in ecpg_sqlca_key_destructor () from /lib32/libecpg.so.6 #8 0xf7d1bbb0 in __nptl_deallocate_tsd () from /lib32/libpthread.so.0 #9 0xf7d1c509 in start_thread () from /lib32/libpthread.so.0 #10 0xf7c9d08e in clone () from /lib32/libc.so.6 (gdb) As you might have noticed, this particular run is on a 64bit architecture (Ubuntu 8.04) and the crashex program is generated on a 32bit machine with gcc-4.4.0. I have tried with PostgreSQL version 8.3.5 and 8.3.7. All give the same result, though the specific program addresses of course might differ from system to system. Please help, Leif
Attachment:
crashex.pgc
Description: Binary data
