On Tue, Sep 22, 2020 at 03:12:57PM +0200, Uladzislau Rezki wrote: > > > > > Yes, I do well remember that you are unhappy with this approach. > > > > > Unfortunately, thus far, there is no solution that makes all developers > > > > > happy. You might be glad to hear that we are also looking into other > > > > > solutions, each of which makes some other developers unhappy. So we > > > > > are at least not picking on you alone. :-/ > > > > > > > > No worries I do not feel like a whipping boy here. But do expect me to > > > > argue against the approach. I would also appreciate it if there was some > > > > more information on other attempts, why they have failed. E.g. why > > > > pre-allocation is not an option that works well enough in most > > > > reasonable workloads. > > > Pre-allocating has some drawbacks: > > > > > > a) It is impossible to predict how many pages will be required to > > > cover a demand that is controlled by different workloads on > > > various systems. > > > > Yes, this is not trivial but not a rocket science either. Remember that > > you are relying on a very dumb watermark based pcp pool from the > > allocator. > > > We rely on it, indeed. If the pcp-cache is depleted our special work is > triggered to charge our local cache(few pages) such way will also initiate > the process of pre-featching pages from the buddy allocator populating > the depleted pcp-cache. I do not have any concern here. > It can interfere with ATOMIC allocations in critical paths in extreme circumstances as it potentially puts increased pressure on the emergency reserve as watermarks are bypassed. That adds to the risk of a functional failuure if reclaim fails to make progress. The number of pages are likely to be limited and unpredictable. As it uses any PCP type, it potentially causes fragmention issues. For the last point, the allocations may be transient in the RCU case now but not guaranteed forever. As the API is in gfp.h, it's open to abuse so the next guy that comes along and thinks "I am critical no matter what the name says" will cause problems. While you could argue that would be caught in review, plenty of GFP flag abuses made it through review. Fundamentally, this is simply shifting the problem from RCU to the page allocator because of the locking arrangements and hazard of acquiring zone lock is a raw spinlock is held on RT. It does not even make the timing predictable as an empty PCU list (for example, a full drain in low memory situations) may mean the emergency path is hit anyway. About all it changes is the timing of when the emergency path is hit in some circumstances -- it's not fixing the problem, it's simply changing the shape. > > > > Mimicing a similar implementation shouldn't be all that hard > > and you will get your own pool which doesn't affect other page allocator > > users as much as a bonus. > > > I see your point Michal. As i mentioned before, it is important to avoid of > having such own pools, because the aim is not to waste memory resources. A > page will be returned back to "page allocator" as soon as a scheduler place > our reclaim thread on a CPU and grace period is passed. So, the resource > can be used for other needs. What is important. > As the emergency path and synchronising can be hit no matter what, why not increase the pool temporarily after the emergency path is hit and shrink it again later if necessary? > Otherwise a memory footprint is increased what is bad for low memory > conditions when OOM is involved. OOM would only be a major factor if the size of the pools meant the machine could not even operate or at least was severely degraded. However, depleting the PCPU lists for RCU may slow kswapd making reclaim progress and cause an OOM in itself, or at least an intervention by a userspace monitor that kills non-critical applications in the background when memory pressure exists. > > > As for memory overhead, it is important to reduce it because of > > > embedded devices like phones, where a low memory condition is a > > > big issue. In that sense pre-allocating is something that we strongly > > > would like to avoid. > > > > How big "machines" are we talking about here? I would expect that really > > tiny machines would have hard times to really fill up thousands of pages > > with pointers to free... > > > I mentioned above. We can not rely on static model. We would like to > have a mechanism that gives back ASAP used pages to page allocator > for other needs. > After an emergency, temporarily increase the size of the pool to avoid hitting the emergency path again in the near future. > > > > Would a similar scaling as the page allocator feasible. Really I mostly > > do care about shared nature of the pcp allocator list that one user can > > easily monopolize with this API. > > > I see your concern. pcplist can be monopolized by already existing API: > > while (i < 100) > __get_free_page(GFP_NOWAIT | __GFP_NOWARN); > That's not the same class of abuse as it can go to the buddy lists to refill the correct PCP lists, avoid fragmentation issues, obeys watermarks and wakes kswapd if it's not awake already. -- Mel Gorman SUSE Labs
