On Tue 13-08-19 09:20:51, Khalid Aziz wrote: > On 8/13/19 8:05 AM, Michal Hocko wrote: > > On Mon 12-08-19 19:40:10, Khalid Aziz wrote: > > [...] > >> Patch 1 adds code to maintain a sliding lookback window of (time, number > >> of free pages) points which can be updated continuously and adds code to > >> compute best fit line across these points. It also adds code to use the > >> best fit lines to determine if kernel must start reclamation or > >> compaction. > >> > >> Patch 2 adds code to collect data points on free pages of various orders > >> at different points in time, uses code in patch 1 to update sliding > >> lookback window with these points and kicks off reclamation or > >> compaction based upon the results it gets. > > > > An important piece of information missing in your description is why > > do we need to keep that logic in the kernel. In other words, we have > > the background reclaim that acts on a wmark range and those are tunable > > from the userspace. The primary point of this background reclaim is to > > keep balance and prevent from direct reclaim. Why cannot you implement > > this or any other dynamic trend watching watchdog and tune watermarks > > accordingly? Something similar applies to kcompactd although we might be > > lacking a good interface. > > > > Hi Michal, > > That is a very good question. As a matter of fact the initial prototype > to assess the feasibility of this approach was written in userspace for > a very limited application. We wrote the initial prototype to monitor > fragmentation and used /sys/devices/system/node/node*/compact to trigger > compaction. The prototype demonstrated this approach has merits. > > The primary reason to implement this logic in the kernel is to make the > kernel self-tuning. What makes this particular self-tuning an universal win? In other words there are many ways to analyze the memory pressure and feedback it back that I can think of. It is quite likely that very specific workloads would have very specific demands there. I have seen cases where are trivial increase of min_free_kbytes to normally insane value worked really great for a DB workload because the wasted memory didn't matter for example. > The more knobs we have externally, the more complex > it becomes to tune the kernel externally. I agree on this point. Is the current set of tunning sufficient? What would be missing if not? -- Michal Hocko SUSE Labs
