Re: Indirect syscall restarted incorrectly after signal

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 01/12/2015 04:31 PM, Ed Swierk wrote:
I'm trying to track down a strange problem affecting an O32 userspace
program on an N64 MIPS kernel. I'm using a 64-bit kernel from Cavium
that's nominally 3.10.20 but has an assortment of patches, including
rt and grsec and various Cavium stuff.

If you have all that stuff, you should also have access to the OCTEON simulator, that can produce an instruction trace...

If you can create a testcase that fails within fewer than 10^9 instructions on the simulator (on say fewer than 4 CPUs), then it would be child's play to find the problem...

David Daney

My glibc is stock 2.19-13 from
Debian.

The program is written in Go and compiled with gccgo from gcc 4.9.2.
It is using the exec.Command API which is a Go wrapper for fork(),
exec(), wait() and friends. The runtime library (libgo) was changed
sometime before 4.9.2 to call clone() rather than fork() (see
http://patchwork.ozlabs.org/patch/386411/). Presumably for expediency,
the library invokes clone() indirectly via syscall(). Complicating
matters, the clone() calls are invoked from different threads, so the
program also has to deal with handling SIGCHLD whenever one of its
child processes exits.

Most of the time, the indirect clone() call works just fine.
Occasionally, however, the clone() gets interrupted by a signal. When
the signal handler returns, the kernel tries to restart the clone()
syscall by rolling back the program counter and various registers, and
jumping back into userspace at the point the syscall was first
originally called.

Running my program under strace looks like this (minus noise from
other processes/threads):

2530  syscall(0x1018, 0x12, 0, 0, 0, 0, 0 <unfinished ...>
2532  --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=3113,
si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
2530  <... syscall resumed> )           = ? ERESTARTNOINTR (To be restarted)
2530  syscall(0x12, 0, 0, 0, 0, 0, 0)   = -1 ENOSYS (Function not implemented)
2532  rt_sigreturn( <unfinished ...>
2532  <... rt_sigreturn resumed> )      = 2

syscall(0x1018) (where 0x1018 is the syscall number for clone on
32-bit MIPS) first returns ERESTARTNOINTR (as expected, this never
actually propagates back to userspace). But the next attempt uses
syscall number 0x22, which returns ENOSYS because there's no such
syscall.

I assume it is no coincidence that 0x12 is the first argument to the
original syscall.

For comparison, when I compile my program against the original libgo
which calls fork() and run it under strace I see the following:

16791 clone( <unfinished ...>
16792 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=17006,
si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
16792 rt_sigreturn( <unfinished ...>
16791 <... clone resumed> child_stack=0,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x3bc843c8) = ? ERESTARTNOINTR (To be restarted)
16792 <... rt_sigreturn resumed> )      = 0
16791 clone( <unfinished ...>
16791 <... clone resumed> child_stack=0,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x3bc843c8) = 17008

Note that the second call to clone() has exactly the same arguments as
the first one, and returns the new PID as expected.

I spent quite some time digging into the syscall code in the kernel
and glibc, but couldn't figure out who is supposed to shift arguments
and push some of them to the stack and others to registers, and so on.

I tried the same experiment with a 64-bit little-endian userspace from
the Debian mips64el repository and a gcc 4.9.2 toolchain targeting
mips64el. The program works fine. So the problem appears limited to
O32 userspace on N64 kernel (not clear whether endianness is an
issue).

I can prepare a self-contained test case, but thought I'd first ask if
this symptom rings a bell with anyone on the list.

--Ed








[Index of Archives]     [Linux MIPS Home]     [LKML Archive]     [Linux ARM Kernel]     [Linux ARM]     [Linux]     [Git]     [Yosemite News]     [Linux SCSI]     [Linux Hams]

  Powered by Linux