On Thu, Sep 05, 2019 at 01:47:05PM -0400, Joel Fernandes wrote: > On Thu, Sep 05, 2019 at 01:35:07PM -0400, Steven Rostedt wrote: > > > > > > [ Added Tom ] > > > > On Thu, 5 Sep 2019 09:03:01 -0700 > > Suren Baghdasaryan <surenb@xxxxxxxxxx> wrote: > > > > > On Thu, Sep 5, 2019 at 7:43 AM Michal Hocko <mhocko@xxxxxxxxxx> wrote: > > > > > > > > [Add Steven] > > > > > > > > On Wed 04-09-19 12:28:08, Joel Fernandes wrote: > > > > > On Wed, Sep 4, 2019 at 11:38 AM Michal Hocko <mhocko@xxxxxxxxxx> wrote: > > > > > > > > > > > > On Wed 04-09-19 11:32:58, Joel Fernandes wrote: > > > > [...] > > > > > > > but also for reducing > > > > > > > tracing noise. Flooding the traces makes it less useful for long traces and > > > > > > > post-processing of traces. IOW, the overhead reduction is a bonus. > > > > > > > > > > > > This is not really anything special for this tracepoint though. > > > > > > Basically any tracepoint in a hot path is in the same situation and I do > > > > > > not see a point why each of them should really invent its own way to > > > > > > throttle. Maybe there is some way to do that in the tracing subsystem > > > > > > directly. > > > > > > > > > > I am not sure if there is a way to do this easily. Add to that, the fact that > > > > > you still have to call into trace events. Why call into it at all, if you can > > > > > filter in advance and have a sane filtering default? > > > > > > > > > > The bigger improvement with the threshold is the number of trace records are > > > > > almost halved by using a threshold. The number of records went from 4.6K to > > > > > 2.6K. > > > > > > > > Steven, would it be feasible to add a generic tracepoint throttling? > > > > > > I might misunderstand this but is the issue here actually throttling > > > of the sheer number of trace records or tracing large enough changes > > > to RSS that user might care about? Small changes happen all the time > > > but we are likely not interested in those. Surely we could postprocess > > > the traces to extract changes large enough to be interesting but why > > > capture uninteresting information in the first place? IOW the > > > throttling here should be based not on the time between traces but on > > > the amount of change of the traced signal. Maybe a generic facility > > > like that would be a good idea? > > > > You mean like add a trigger (or filter) that only traces if a field has > > changed since the last time the trace was hit? Hmm, I think we could > > possibly do that. Perhaps even now with histogram triggers? > > > Hey Steve, > > Something like an analog to digitial coversion function where you lose the > granularity of the signal depending on how much trace data: > https://www.globalspec.com/ImageRepository/LearnMore/20142/9ee38d1a85d37fa23f86a14d3a9776ff67b0ec0f3b.gif s/how much trace data/what the resolution is/ > so like, if you had a counter incrementing with values after the increments > as: 1,3,4,8,12,14,30 and say 5 is the threshold at which to emit a trace, > then you would get 1,8,12,30. > > So I guess what is need is a way to reduce the quantiy of trace data this > way. For this usecase, the user mostly cares about spikes in the counter > changing that accurate values of the different points. s/that accurate/than accurate/ I think Tim, Suren, Dan and Michal are all saying the same thing as well. thanks, - Joel
