----- On Feb 3, 2018, at 4:43 PM, Linus Torvalds torvalds@xxxxxxxxxxxxxxxxxxxx wrote: > On Sat, Feb 3, 2018 at 9:04 AM, Mathieu Desnoyers > <mathieu.desnoyers@xxxxxxxxxxxx> wrote: >> >> The approach proposed here will introduce an expectation that internal >> function signatures never change in the kernel, else it would break user-space >> tools hooking on those functions. > > No, I really don't think so. > > There's two reasons for that: > > The first is purely about kernel development. I, and every sane kernel > engineer, will simply laugh in the face of somebody who comes to us > and says "hey, I had this script that did low-level function tracing > on your kernel, and then you changed something, and now the random > function I was tracing has a new name and different arguments". > > We'll just go "yeah, tough, change your script". Or more likely, not > even bother to reply at all. > > But the bigger issue is actually simply just psychology. Exactly > *because* this is all implicit, and there are no explicit trace > points, it's _obvious_ to any user that there isn't something > long-term dependable that they hang their hat on. > > Everybody *understands* that this is like a debugger: if you have a > gdb script that shows some information, and then you go around and > change the source code, then *obviously* you'll have to change your > debugger script too. You don't keep the source code static just to > make your gdb script happy., That would be silly. > > In contrast, the explicit tracepoints really made people believe that > they have some long-term meaning. > > So yes, we'll make it obvious that hell no, random kernel functions > are not a long-term ABI. But honestly, I don't think we even need to > have a lot of "education" on this, simply because it's so obvious that > anybody who thinks it's some ABI is not going to be somebody we'll > have to worry about. > > Because the kind of person thinking "Ooh, this is a stable ABI" won't > be doing interesting work anyway. That kind of person will be sitting > in a corner eating paste, not doing interesting kernel tracing. I agree with your arguments. A consequence of those arguments is that function-based tracing should be expected to be used by kernel engineers and experts who can adapt their scripts to follow code changes, and tune the script based on their specific kernel version and configuration. On the other hand, system administrators and application developers who wish to tune and understand their workload will more likely fetch their analysis scripts/tools from a third-party. If those scripts hook on functions/arguments, it becomes really hard to ensure those will work with a myriad of kernel versions, configurations, and toolchain versions out there. The good news is that Steven's approach should allow us to deal with use-cases that are specific to kernel developers by simply using function tracing. That should take care of most "one off" and very specialized use-cases. My genuine concern here is that tools targeting sysadmins and application developers start hooking on kernel functions, and all goes well for a few months or years until someone changes that part of the code. Then all those wonderfully useful scripts that users depend on will end up being broken: in the best scenario those tools will detect the change and require to be updated, but in the worse case the tools will simply print bogus results that people will take for granted. This should therefore leave a door open to adding new tracepoints: cases where the data gathered is shown to be useful enough for tools targeting an audience wider than just kernel developers. To improve over the current situation, we should think about documenting some rules about how those tools should cope with tracepoints changing over time (event version, tools backward compatibility, and so on), and make sure the ABI exposes the information required to help tools cope with change. Thanks, Mathieu -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com -- To unsubscribe from this list: send the line "unsubscribe linux-trace-users" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html