When pipes are used for communication between trace-cmd main
thread and per-cpu recorder threads, there is a possible race
condition in stop_threads(), which can cause a deadlock between
the main thread and cpu recorder thread:
In trace_stream_read(), the select() call can return 0 if threads
have no data to send. This will force stop_threads() to stop reading
the thread's pipes and enter a waitpid() loop, to wait for all threads
to be terminated. However, there is a case when some threads are still
flushing its data - tracecmd_flush_recording() tries a blocking write()
to the pipe. A dead lock appears - the cpu thread is blocked in write(),
as its buffer is full and no one is reading it. The main thread is blocked
in waitpid(), to wait the same thread to exit.
The deadlock can be (randomly) observed with the command
"trace-cmd profile -p function -F sleep 10"
The proposed fix increases select timeout from 0 to 1 second, to ensure
the threads are flushed its data before going in waitpid() loop.
Signed-off-by: Tzvetomir Stoyanov <tstoyanov@xxxxxxxxxx>
---
tracecmd/trace-record.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c
index 4523128..9aef5c3 100644
--- a/tracecmd/trace-record.c
+++ b/tracecmd/trace-record.c
@@ -626,7 +626,7 @@ static void delete_thread_data(void)
static void stop_threads(enum trace_type type)
{
- struct timeval tv = { 0, 0 };
+ struct timeval tv = { 1, 0 };
int ret;
int i;
--
2.20.1
|