Does it also happen when you restart with 'mktree' ?
(you can try the latest tree which is temporarily ni branches
ckpt-v16-x86 in both linux-cr and user-cr)
Oren.
Nathan Lynch wrote:
> (Latest commit is a5e53f3... Define clone_with_pids syscall)
>
> I have a pretty simple bash script (included below) which usually
> restarts successfully, but occasionally it gets a segfault after
> restart, maybe 20% of the time. I'm using the ckpt and rstr commands
> from user-cr.git. Some examples (output of show_signal_msg in
> arch/x86/mm/fault.c):
>
> bash-simple.sh[3608]: segfault at 0 ip 00363472 sp bfb63854 error 4 in libc-2.9.so[2e2000+16e000]
> bash-simple.sh[3728]: segfault at 0 ip 00358d03 sp bfe375f8 error 4 in libc-2.9.so[2e2000+16e000]
> bash-simple.sh[3756]: segfault at 14 ip 0030d14a sp bfe9c45c error 6 in libc-2.9.so[2e2000+16e000]
> bash-simple.sh[3812]: segfault at 14 ip 003633e6 sp bf9623b4 error 4 in libc-2.9.so[2e2000+16e000]
> bash-simple.sh[4049]: segfault at 0 ip 002fd054 sp bfbdfab4 error 6 in libc-2.9.so[2e2000+16e000]
>
> Typical /proc/pid/maps (from before checkpoint):
>
> 002bd000-002dd000 r-xp 00000000 08:03 11449 /lib/ld-2.9.so
> 002de000-002df000 r--p 00020000 08:03 11449 /lib/ld-2.9.so
> 002df000-002e0000 rw-p 00021000 08:03 11449 /lib/ld-2.9.so
> 002e2000-00450000 r-xp 00000000 08:03 196992 /lib/libc-2.9.so
> 00450000-00452000 r--p 0016e000 08:03 196992 /lib/libc-2.9.so
> 00452000-00453000 rw-p 00170000 08:03 196992 /lib/libc-2.9.so
> 00453000-00456000 rw-p 00000000 00:00 0
> 00458000-0045b000 r-xp 00000000 08:03 196999 /lib/libdl-2.9.so
> 0045b000-0045c000 r--p 00002000 08:03 196999 /lib/libdl-2.9.so
> 0045c000-0045d000 rw-p 00003000 08:03 196999 /lib/libdl-2.9.so
> 08047000-080fb000 r-xp 00000000 08:03 11602 /bin/bash
> 080fb000-08100000 rw-p 000b3000 08:03 11602 /bin/bash
> 08100000-08105000 rw-p 00000000 00:00 0
> 08536000-08557000 rw-p 00000000 00:00 0 [heap]
> 46bc0000-46bd6000 r-xp 00000000 08:03 10261 /lib/libtinfo.so.5.6
> 46bd6000-46bd9000 rw-p 00015000 08:03 10261 /lib/libtinfo.so.5.6
> b7dfb000-b7ffb000 r--p 00000000 08:03 123071 /usr/lib/locale/locale-archive
> b7ffb000-b7ffd000 rw-p 00000000 00:00 0
> b8002000-b8003000 rw-p 00000000 00:00 0
> b8003000-b800a000 r--s 00000000 08:03 172333 /usr/lib/gconv/gconv-modules.cache
> bfbcc000-bfbe1000 rw-p 00000000 00:00 0 [stack]
> ffffe000-fffff000 r-xp 00000000 00:00 0 [vdso]
>
> ...and gdb backtrace from the core dump:
>
> (gdb) bt
> #0 0x002fd054 in utf8_internal_loop () at ../iconv/loop.c:332
> #1 __gconv_transform_utf8_internal (step=0x8538570, data=0xbfbdfbac,
> inptrp=0xbfbdfbd0, inend=0x853854b "", outbufstart=0x0,
> irreversible=0xbfbdfbd4, do_flush=0, consume_incomplete=1)
> at ../iconv/skeleton.c:611
> #2 0x00363440 in __mbrtowc (pwc=<value optimized out>, s=0x8538548 " \t\n",
> n=3, ps=<value optimized out>) at mbrtowc.c:82
> #3 0x080b703d in mbrlen () at /usr/include/wchar.h:348
> #4 xstrchr (s=0x8538548 " \t\n", c=48) at xstrchr.c:62
> #5 0x080824f3 in string_extract_verbatim (
> string=0x8539430 "/tmp/bash-4035/step2-go", slen=23, sindex=0xbfbdfccc,
> charlist=0x8538548 " \t\n") at subst.c:961
> #6 0x08082b2e in list_string (string=0x8539430 "/tmp/bash-4035/step2-go",
> separators=0x8538548 " \t\n", quoted=0) at subst.c:1982
> #7 0x08082ee0 in word_split (w=0x8538549, ifs_chars=0x8538548 " \t\n")
> at subst.c:7629
> #8 0x08082f1c in word_list_split (list=<value optimized out>) at subst.c:7647
> #9 0x08087317 in shell_expand_word_list () at subst.c:8056
> #10 expand_word_list_internal (list=<value optimized out>,
> eflags=<value optimized out>) at subst.c:8149
> #11 0x080703b0 in execute_simple_command (simple_command=0x853ada0,
> pipe_in=-1, pipe_out=-1, async=0, fds_to_close=0x85397e0)
> at execute_cmd.c:2881
>
> I've seen xstrchr implicated more than once... I'll lazily speculate
> that some sort of mmx/sse state may be restored incorrectly, assuming
> that glibc is using SIMD instructions for string operations.
>
> Not sure whether this was recently introduced or if it's a long-standing
> bug. I tried backtracking in ckpt-v16-dev history to get a known-good
> starting point for bisect but ran into other problems.
>
>
> #!/bin/bash
>
> set -eu
>
> tmpdir="/tmp/bash-$1"
>
> step1go="$tmpdir/step1-go"
> step1ok="$tmpdir/step1-ok"
> step2go="$tmpdir/step2-go"
> step2ok="$tmpdir/step2-ok"
>
> maps_before="$tmpdir/maps-before"
> maps_after="$tmpdir/maps-after"
>
> pidfile="$tmpdir/pid-there"
>
> logfile="$tmpdir/bash-simple-$$.log"
>
> # slow version of $$ which works across restart
> # should use redirection, not pipe or command substitution
> getpid() {
> bash -c 'echo $PPID'
> }
>
> # close stdin
> exec <&-
>
> # redirect stdio/stderr to file
> exec 1>"$logfile"
> exec 2>&1
>
> ls -l /proc/$$/fd
>
> echo $$ > $pidfile
>
> while [ ! -f $step1go ] ; do : ; done
>
> cat /proc/$$/maps > "$maps_before"
>
> wait
>
> echo "Step 1 OK."
> echo > $step1ok
>
> # wait for checkpoint -- just spin, don't fork a task for sleep
> while [ ! -f $step2go ] ; do : ; done
>
> # restarted
>
> echo "Step 2 OK."
>
> getpid > "$pidfile"
> read mypid < "$pidfile"
>
> cat /proc/$mypid/maps > "$maps_after"
>
> echo > $step2ok
> _______________________________________________
> Containers mailing list
> Containers@xxxxxxxxxxxxxxxxxxxxxxxxxx
> https://lists.linux-foundation.org/mailman/listinfo/containers
>
_______________________________________________
Containers mailing list
Containers@xxxxxxxxxxxxxxxxxxxxxxxxxx
https://lists.linux-foundation.org/mailman/listinfo/containers
