Hi Michal, Am 11.09.19 um 08:12 schrieb Stefan Priebe - Profihost AG: > Hi Michal, > Am 10.09.19 um 15:24 schrieb Michal Hocko: >> On Tue 10-09-19 15:14:45, Stefan Priebe - Profihost AG wrote: >>> Am 10.09.19 um 15:05 schrieb Stefan Priebe - Profihost AG: >>>> >>>> Am 10.09.19 um 14:57 schrieb Michal Hocko: >>>>> On Tue 10-09-19 14:45:37, Stefan Priebe - Profihost AG wrote: >>>>>> Hello Michal, >>>>>> >>>>>> ok this might take a long time. Attached you'll find a graph from a >>>>>> fresh boot what happens over time (here 17 August to 30 August). Memory >>>>>> Usage decreases as well as cache but slowly and only over time and days. >>>>>> >>>>>> So it might take 2-3 weeks running Kernel 5.3 to see what happens. >>>>> >>>>> No problem. Just make sure to collect the requested data from the time >>>>> you see the actual problem. Btw. you try my very dumb scriplets to get >>>>> an idea of how much memory gets reclaimed due to THP. >>>> >>>> You mean your sed and sort on top of the trace file? No i did not with >>>> the current 5.3 kernel do you think it will show anything interesting? >>>> Which line shows me how much memory gets reclaimed due to THP? >> >> Please re-read http://lkml.kernel.org/r/20190910082919.GL2063@xxxxxxxxxxxxxx >> Each command has a commented output. If you see nunmber of reclaimed >> pages to be large for GFP_TRANSHUGE then you are seeing a similar >> problem. >> >>> Is something like a kernel memory leak possible? Or wouldn't this end up >>> in having a lot of free memory which doesn't seem usable. >> >> I would be really surprised if this was the case. >> >>> I also wonder why a reclaim takes place when there is enough memory. >> >> This is not clear yet and it might be a bug that has been fixed since >> 4.18. That's why we need to see whether the same is pattern is happening >> with 5.3 as well. but except from the btrfs problem the memory consumption looks far better than before. Running 4.19.X: after about 12h cache starts to drop from 30G to 24G Running 5.3-rc8: after about 24h cache is still constant at nearly 30G Greets, Stefan