Hi,
I was trying to get some sched traces on a 160 cpu box yesterday. Trace-cmd
failed with
# ./tracecmd/trace-cmd record -e "sched:*" sleep 2
none
trace-cmd: Invalid argument
Failed filter of /sys/kernel/tracing/events/sched/sched_switch/filter
trace-cmd: No such file or directory
can not stat 'trace.dat.cpu0'
#
Which can be seen better with strace
[pid 97653] open("/sys/kernel/tracing/events/sched/sched_swap_numa/filter", O_WRONLY|O_TRUNC) = 5
[pid 97653] write(5, "(common_pid!=97652)&&(common_pid"..., 3358) = 3358
[pid 97653] close(5) = 0
[pid 97653] open("/sys/kernel/tracing/events/sched/sched_switch/filter", O_WRONLY|O_TRUNC) = 5
[pid 97653] write(5, "(common_pid!=97652)&&(common_pid"..., 6398) = -1 EINVAL (Invalid argument)
[pid 97653] close(5) = 0
The filter file can only take a max write of length PAGE_SIZE.
The extra pid filtering added for "next_pid" more or less doubles length
and pushes it over the 4k limit.
WRITE: /sys/kernel/tracing/events/sched/sched_switch/filter, len 6718, data "(common_pid!=100199)&&(common_pid!=100198)&&(common_pid!=100197)&&(common_pid!=100196)&&(common_pid!=100195)&&(common_pid!=100194)&&(common_pid!=100193)&&(common_pid!=100192)&&(common_pid!=100191) ... 160 of these ...
&&(common_pid!=100040)||(next_pid!=100199)&&(next_pid!=100198)&&(next_pid!=100197)&&(next_pid!=100196)&&(next_pid!=100195)&&(next_pid!=100194)&&(next_pid!=100193)&&(next_pid!=100192)... 160 of these...
I suppose the answer is don't run on a system with that many cpus :)
But I wonder if it would be possible to have the threads each handle say 8 cpu
files or something.
Or maybe have the kernel filter accept an "all_pid" that covered common_pid, next_pid, pid to reduce
the number of items needed in there?
Thanks,
Phil
--
|