Hi Koki, Yes, the patch is here: https://lkml.org/lkml/2018/1/12/600 It has not been reviewed yet. Pavel On Wed, Jan 31, 2018 at 12:28 PM, Koki.Sanagi@xxxxxxxxxxxxxx <Koki.Sanagi@xxxxxxxxxxxxxx> wrote: > Pavel, > > I assume you are working on the fix. > Do you have any progress ? > > Koki > >>>-----Original Message----- >>>From: Mel Gorman [mailto:mgorman@xxxxxxxxxxxxxxxxxxx] >>>Sent: Wednesday, December 06, 2017 5:50 AM >>>To: Pavel Tatashin <pasha.tatashin@xxxxxxxxxx> >>>Cc: Michal Hocko <mhocko@xxxxxxxxxx>; YASUAKI ISHIMATSU >>><yasu.isimatu@xxxxxxxxx>; Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>; >>>Linux Memory Management List <linux-mm@xxxxxxxxx>; linux- >>>kernel@xxxxxxxxxxxxxxx; Sanagi, Koki <Koki.Sanagi@xxxxxxxxxxxxxx>; Steve >>>Sistare <steven.sistare@xxxxxxxxxx> >>>Subject: Re: [PATCH] mm, meminit: Serially initialise deferred memory if >>>trace_buf_size is specified >>> >>>On Wed, Nov 29, 2017 at 10:41:59PM -0500, Pavel Tatashin wrote: >>>> Hi Mel, >>>> >>>> Thank you very much for your feedback, my replies below: >>>> >>>> > A lack of involvement from admins is indeed desirable. For example, >>>> > while I might concede on using a disable-everything-switch, I would >>>> > not be happy to introduce a switch that specified how much memory >>>> > per node to initialise. >>>> > >>>> > For the forth approach, I really would be only thinking of a blunt >>>> > "initialise everything instead of going OOM". I was wary of making >>>> > things too complicated and I worried about some side-effects I'll cover later. >>>> >>>> I see, I misunderstood your suggestion. Switching to serial >>>> initialization when OOM works, however, boot time becomes >>>> unpredictable, with some configurations boot is fast with others it is >>>> slow. All of that depends on whether predictions in >>>> reset_deferred_meminit() were good or not which is not easy to debug >>>> for users. Also, overtime predictions in reset_deferred_meminit() can >>>> become very off, and I do not think that we want to continuously >>>> adjust this function. >>>> >>> >>>You could increase the probabilty of a report by doing a WARN_ON_ONCE if the >>>serialised meminit is used. >>> >>>> >> With this approach we could always init a very small amount of >>>> >> struct pages, and allow the rest to be initialized on demand as >>>> >> boot requires until deferred struct pages are initialized. Since, >>>> >> having deferred pages feature assumes that the machine is large, >>>> >> there is no drawback of having some extra byte of dead code, >>>> >> especially that all the checks can be permanently switched of via >>>> >> static branches once deferred init is complete. >>>> >> >>>> > >>>> > This is where I fear there may be dragons. If we minimse the number >>>> > of struct pages and initialise serially as necessary, there is a >>>> > danger that we'll allocate remote memory in cases where local memory >>>> > would have done because a remote node had enough memory. >>>> >>>> True, but is not what we have now has the same issue as well? If one >>>> node is gets out of memory we start using memory from another node, >>>> before deferred pages are initialized? >>>> >>> >>>It's possible but I'm not aware of it happening currently. >>> >>>> To offset that risk, it would be >>>> > necessary at boot-time to force allocations from local node where >>>> > possible and initialise more memory as necessary. That starts >>>> > getting complicated because we'd need to adjust gfp-flags in the >>>> > fast path with init-and-retry logic in the slow path and that could >>>> > be a constant penalty. We could offset that in the fast path by >>>> > using static branches >>>> >>>> I will try to implement this, and see how complicated the patch will >>>> be, if it gets too complicated for the problem I am trying to solve we >>>> can return to one of your suggestions. >>>> >>>> I was thinking to do something like this: >>>> >>>> Start with every small amount of initialized pages in every node. >>>> If allocation fails, initialize enough struct pages to cover this >>>> particular allocation with struct pages rounded up to section size but >>>> in every single node. >>>> >>> >>>Ok, just make sure it's all in the slow paths of the allocator when the alternative >>>is to fail the allocation. >>> >>>> > but it's getting more and >>>> > more complex for what is a minor optimisation -- shorter boot times >>>> > on large machines where userspace itself could take a *long* time to >>>> > get up and running (think database reading in 1TB of data from disk as it >>>warms up). >>>> >>>> On M6-32 with 32T [1] of memory it saves over 4 minutes of boot time, >>>> and this is on SPARC with 8K pages, on x86 it would be around of 8 >>>> minutes because of twice as many pages. This feature improves >>>> availability for larger machines quite a bit. Overtime, systems are >>>> growing, so I expect this feature to become a default configuration in >>>> the next several years on server configs. >>>> >>> >>>Ok, when developing the series originally, I had no machine even close to 32T of >>>memory. >>> >>>-- >>>Mel Gorman >>>SUSE Labs > > -- > To unsubscribe, send a message with 'unsubscribe linux-mm' in > the body to majordomo@xxxxxxxxx. For more info on Linux MM, > see: http://www.linux-mm.org/ . > Don't email: <a hrefmailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a> -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>
