On Thu, Sep 29, 2016 at 01:50:34PM +0300, Nikolay Borisov wrote: > > > On 09/29/2016 01:27 PM, Paul E. McKenney wrote: > > On Thu, Sep 29, 2016 at 10:11:09AM +0300, Nikolay Borisov wrote: > [SNIP] > > >> What in particular should I be looking for in ftrace? tracing the stacks > >> on the stuck cpu? > > > > To start with, how about the sequence of functions that the stuck > > CPU is executing? > > Unfortunately I do not know how to reproduce the issue, but it is being > reproduced byt our production load - which is creating backups in this > case. They are created by rsyncing files to a loop-back attached files > wihch are then unmounted and unmapped.From this crash it is evident that > the hang occurs while a volume is being unmounted. > > But the callstack is in my hang report, no? I have the crashdump with me > so if you are interested in anything in particular I can go look for it. > I believe an inode eviction was requested, since destroy_inode, which > utilizes ext4_i_callback is called in the eviction + some errors paths. > And this eviction is executed on this particular CPU. What in particular > are you looking for? > > Unfortunately it's impossible for me to run: > > trace-cmd record -p function_graph -F <command that causes the issue> Given that the hang appears to involve a few common functions, is it possible to turn ftrace on for those functions (and related ones) at boot time? Or using sysfs just after boot time? Given that you cannot reproduce at will, your earlier suggestion of enabling tracing of stacks might also make a lot of sense. Thanx, Paul > [SNIP] -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>