On Tue 22-11-16 15:30:56, Michal Hocko wrote:
> On Tue 22-11-16 10:56:51, Nikolay Borisov wrote:
> >
> >
> > On 11/21/2016 07:31 AM, Michal Hocko wrote:
> > > Hi,
> > > I am sorry for a late response, but I was offline until this weekend. I
> > > will try to get to this email ASAP but it might take some time.
> >
> > No worries. I did some further digging up and here is what I got, which
> > I believe is rather strange:
> >
> > struct scan_control {
> > nr_to_reclaim = 32,
> > gfp_mask = 37880010,
> > order = 0,
> > nodemask = 0x0,
> > target_mem_cgroup = 0xffff8823990d1400,
> > priority = 7,
> > may_writepage = 1,
> > may_unmap = 1,
> > may_swap = 0,
> > may_thrash = 1,
> > hibernation_mode = 0,
> > compaction_ready = 0,
> > nr_scanned = 0,
> > nr_reclaimed = 0
> > }
> >
> > Parsing: 37880010
> > #define ___GFP_HIGHMEM 0x02
> > #define ___GFP_MOVABLE 0x08
> > #define ___GFP_IO 0x40
> > #define ___GFP_FS 0x80
> > #define ___GFP_HARDWALL 0x20000
> > #define ___GFP_DIRECT_RECLAIM 0x400000
> > #define ___GFP_KSWAPD_RECLAIM 0x2000000
> >
> > And initial_priority is 12 (DEF_PRIORITY). Given that nr_scanned is 0
> > and priority is 7 this means we've gone 5 times through the do {} while
> > in do_try_to_free_pages. Also total_scanned seems to be 0. Here is the
> > zone which was being reclaimed :
> >
> > http://sprunge.us/hQBi
>
> LRUs on that zones seem to be empty from a quick glance. kmem -z in the
> crash can give you per zone counters much more nicely.
>
> > So what's strange is that the softlockup occurred but then the code
> > proceeded (as evident from the subsequent stack traces), yet inspecting
> > the reclaim progress it seems rather sad (no progress at all)
>
> Unless I have misread the data above it seems something has either
> isolated all LRU pages for some time or there simply are none while the
> reclaim is desperately trying to make some progress. In any case this
> sounds less than a happy system...
Btw. how do you configure memcgs that the FS workload runs in?
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxx. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>