On Mon, Feb 10, 2020 at 05:26:16PM +0530, Gaurav Kohli wrote:
> Hi,
>
> In Linux kernel, everywhere we are using notification chains to notify for
> any kernel events, But we don't have any debugging or profiling mechanism to
> know which callback is taking time or currently we are stuck on which call
> back(without dumps it is difficult to say for last problem)
Callbacks are a mess, I agree.
> Below are the few ways, which we can implement to profile callback on need
> basis:
>
> 1) Use trace event before and after callback:
>
> static int notifier_call_chain(struct notifier_block **nl,
> unsigned long val, void *v,
> int nr_to_call, int *nr_calls)
> {
> int ret = NOTIFY_DONE;
> struct notifier_block *nb, *next_nb;
>
>
> + trace_event for entry of callback
> ret = nb->notifier_call(nb, val, v);
> + trace_event for exit of callback
Ick.
> }
> return ret;
> }
>
> 2) Or use pr_debug instead of trace_event
>
> 3) Both of the above approach has certain problems, like it will dump
> callback for each notifier chain, which might flood trace buffer or dmesg.
>
> So we can use bool variable to control that and dump the required
> notification chain only.
>
> Some thing like below we can use:
>
> struct srcu_notifier_head {
> struct mutex mutex;
> struct srcu_struct srcu;
> struct notifier_block __rcu *head;
> + bool debug_callback;
> };
>
>
> static int notifier_call_chain(struct notifier_block **nl,
> unsigned long val, void *v,
> - int nr_to_call, int *nr_calls)
> + int nr_to_call, int *nr_calls, bool
> debug_callback)
> {
> int ret = NOTIFY_DONE;
> struct notifier_block *nb, *next_nb;
> @@ -526,6 +526,7 @@ void srcu_init_notifier_head(struct srcu_notifier_head
> *nh)
> if (init_srcu_struct(&nh->srcu) < 0)
> BUG();
> nh->head = NULL;
> + nh->debug_callback = false; -> by default it would be false for
> every notifier chain.
>
> 4) we can also think of something pre and post function, before and after
> each callback, And we can enable only for those who wants to profile.
>
> Please let us what approach we can use, or please suggest some debugging
> mechanism for the same.
Why not just pay attention to the specific notifier you want? Trace
when the specific blocking_notifier_call_chain() is called.
What specific notifier call chain is causing you problems that you need
to debug?
thanks,
greg k-h
