backtrace_symbols_fd() can trigger a call to malloc()

[Stefan Puiu <stefan.puiu@xxxxxxxxx>]

Hi Michael,

Currently, the backtrace(3) manual page says this about backtrace_symbols_fd():

back‐
       trace_symbols_fd() does not call malloc(3), and so can be
employed in situations where the latter function might fail.


However, I watched a video of a presentation about signal handling and
the speaker was saying that calling backtrace() can trigger a call to
malloc - indirectly. That happens because the backtrace*() functions
are part of libgcc, which gets dynamically loaded whenever needed;
dynamic loading would, in turn, trigger a malloc. The talk can be
found here: http://free-electrons.com/pub/video/2008/ols/ols2008-gilad-ben-yossef-fault-handlers.ogg

I decided to test it out, and it seems that this is still true (at
least on Ubuntu 12.04). I compiled the attached program (I used
CXXFLAGS = '-Wall -g -std=c++0x', the -std= part is not really needed)
and ran it through gdb, putting a breakpoint on the line where
backtrace is called. When that breakpoint is hit, I set a breakpoint
on malloc, continued and voila:

Breakpoint 2, __GI___libc_malloc (bytes=36) at malloc.c:2910
2910    malloc.c: No such file or directory.
(gdb) bt
#0  __GI___libc_malloc (bytes=36) at malloc.c:2910
#1  0x00007ffff7de294e in local_strdup (s=0x7ffff7ff2574
"/lib/x86_64-linux-gnu/libgcc_s.so.1") at dl-load.c:162
#2  _dl_map_object (loader=0x7ffff7ff94c0, name=0x7ffff7b94918
"libgcc_s.so.1", type=2, trace_mode=0, mode=-1879048191,
nsid=<optimized out>) at dl-load.c:2473
#3  0x00007ffff7ded84b in dl_open_worker (a=0x7fffffffdcb0) at dl-open.c:225
#4  0x00007ffff7de9176 in _dl_catch_error (objname=0x7fffffffdcf8,
errstring=0x7fffffffdd00, mallocedp=0x7fffffffdd0f,
operate=0x7ffff7ded700 <dl_open_worker>, args=0x7fffffffdcb0) at
dl-error.c:178
#5  0x00007ffff7ded31a in _dl_open (file=0x7ffff7b94918
"libgcc_s.so.1", mode=-2147483647, caller_dlopen=0x7ffff7b260a9,
nsid=-2, argc=1, argv=<optimized out>, env=0x7fffffffe068) at
dl-open.c:639
#6  0x00007ffff7b4bc62 in do_dlopen (ptr=0x7fffffffdeb0) at dl-libc.c:89
#7  0x00007ffff7de9176 in _dl_catch_error (objname=0x7fffffffdee0,
errstring=0x7fffffffded0, mallocedp=0x7fffffffdeef,
operate=0x7ffff7b4bc20 <do_dlopen>, args=0x7fffffffdeb0) at
dl-error.c:178
#8  0x00007ffff7b4bd24 in dlerror_run (args=0x7fffffffdeb0,
operate=0x7ffff7b4bc20 <do_dlopen>) at dl-libc.c:48
#9  __GI___libc_dlopen_mode (name=<optimized out>, mode=<optimized
out>) at dl-libc.c:165
#10 0x00007ffff7b260a9 in init () at ../sysdeps/x86_64/../ia64/backtrace.c:53
#11 __GI___backtrace (array=<optimized out>, size=20) at
../sysdeps/x86_64/../ia64/backtrace.c:104
#12 0x000000000040066c in g () at backtrace.cpp:20
#13 0x000000000040068c in f () at backtrace.cpp:26
#14 0x0000000000400697 in main () at backtrace.cpp:31

Here's a tentative patch for backtrace.3 against git:


diff --git a/man3/backtrace.3 b/man3/backtrace.3
index 1b21794..5344d58 100644
--- a/man3/backtrace.3
+++ b/man3/backtrace.3
@@ -114,7 +114,8 @@ it writes the strings, one per line, to the file descriptor
 .BR backtrace_symbols_fd ()
 does not call
 .BR malloc (3),
-and so can be employed in situations where the latter function might fail.
+and so can be employed in situations where the latter function might
+fail, but see NOTES.
 .SH RETURN VALUE
 .BR backtrace ()
 returns the number of addresses returned in
@@ -156,6 +157,21 @@ violated.
 Inlined functions do not have stack frames.
 .IP *
 Tail-call optimization causes one stack frame to replace another.
+.IP *
+.BR backtrace ()
+and
+.BR backtrace_symbols_fd ()
+don't call
+.BR malloc ()
+explicitly, but they are part of libgcc, which gets loaded dynamically
+when first used. Dynamic loading usually triggers a call to
+.BR malloc ().
+So you need to call
+.BR backtrace ()
+or some other libgcc function early in your program to trigger loading
+of libgcc in order to make sure that later calls don't trigger the
+indirect memory allocation (for instance when called from a signal
+handler).
 .PP
 The symbol names may be unavailable without the use of special linker
 options.

====

Thanks,
Stefan.

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

#include <stdio.h>
#include <execinfo.h>
#include <stdlib.h>

static const size_t maxBacktraceSize = 20;
static void *backtraceInfo[maxBacktraceSize];

void g()
{
    int fd = open("/tmp/vrz.log", O_WRONLY | O_CREAT |O_TRUNC);
    if (fd < 0) {
        perror("open");
        exit(1);
    }
    int len = backtrace(backtraceInfo, maxBacktraceSize);
    backtrace_symbols_fd(backtraceInfo, len, fd);
}

void f()
{
    g();
}

int main()
{
    f();
    return 0;
}