On Thu, Sep 05, 2019 at 01:35:07PM -0400, Steven Rostedt wrote: > > > [ Added Tom ] > > On Thu, 5 Sep 2019 09:03:01 -0700 > Suren Baghdasaryan <surenb@xxxxxxxxxx> wrote: > > > On Thu, Sep 5, 2019 at 7:43 AM Michal Hocko <mhocko@xxxxxxxxxx> wrote: > > > > > > [Add Steven] > > > > > > On Wed 04-09-19 12:28:08, Joel Fernandes wrote: > > > > On Wed, Sep 4, 2019 at 11:38 AM Michal Hocko <mhocko@xxxxxxxxxx> wrote: > > > > > > > > > > On Wed 04-09-19 11:32:58, Joel Fernandes wrote: > > > [...] > > > > > > but also for reducing > > > > > > tracing noise. Flooding the traces makes it less useful for long traces and > > > > > > post-processing of traces. IOW, the overhead reduction is a bonus. > > > > > > > > > > This is not really anything special for this tracepoint though. > > > > > Basically any tracepoint in a hot path is in the same situation and I do > > > > > not see a point why each of them should really invent its own way to > > > > > throttle. Maybe there is some way to do that in the tracing subsystem > > > > > directly. > > > > > > > > I am not sure if there is a way to do this easily. Add to that, the fact that > > > > you still have to call into trace events. Why call into it at all, if you can > > > > filter in advance and have a sane filtering default? > > > > > > > > The bigger improvement with the threshold is the number of trace records are > > > > almost halved by using a threshold. The number of records went from 4.6K to > > > > 2.6K. > > > > > > Steven, would it be feasible to add a generic tracepoint throttling? > > > > I might misunderstand this but is the issue here actually throttling > > of the sheer number of trace records or tracing large enough changes > > to RSS that user might care about? Small changes happen all the time > > but we are likely not interested in those. Surely we could postprocess > > the traces to extract changes large enough to be interesting but why > > capture uninteresting information in the first place? IOW the > > throttling here should be based not on the time between traces but on > > the amount of change of the traced signal. Maybe a generic facility > > like that would be a good idea? > > You mean like add a trigger (or filter) that only traces if a field has > changed since the last time the trace was hit? Hmm, I think we could > possibly do that. Perhaps even now with histogram triggers? Hey Steve, Something like an analog to digitial coversion function where you lose the granularity of the signal depending on how much trace data: https://www.globalspec.com/ImageRepository/LearnMore/20142/9ee38d1a85d37fa23f86a14d3a9776ff67b0ec0f3b.gif so like, if you had a counter incrementing with values after the increments as: 1,3,4,8,12,14,30 and say 5 is the threshold at which to emit a trace, then you would get 1,8,12,30. So I guess what is need is a way to reduce the quantiy of trace data this way. For this usecase, the user mostly cares about spikes in the counter changing that accurate values of the different points. thanks, - Joel
