On Fri, 13 Jan 2012 13:16:56 +0100 Johannes Weiner <hannes@xxxxxxxxxxx> wrote: > On Thu, Jan 12, 2012 at 10:54:27AM +0900, KAMEZAWA Hiroyuki wrote: > > Thank you for your work and the result seems atractive and code is much > > simpler. My small concerns are.. > > > > 1. This approach may increase latency of direct-reclaim because of priority=0. > > I think strictly speaking yes, but note that with kswapd being less > likely to get stuck in hammering on one group, the need for allocators > to enter direct reclaim itself is reduced. > > However, if this really becomes a problem in real world loads, the fix > is pretty easy: just ignore the soft limit for direct reclaim. We can > still consider it from hard limit reclaim and kswapd. > > > 2. In a case numa-spread/interleave application run in its own container, > > pages on a node may paged-out again and again becasue of priority=0 > > if some other application runs in the node. > > It seems difficult to use soft-limit with numa-aware applications. > > Do you have suggestions ? > > This is a question about soft limits in general rather than about this > particular patch, right? > Partially, yes. My concern is related to "1". Assume an application is binded to some cpu/node and try to allocate memory. If its memcg's usage is over softlimit, this application will play bad because newly allocated memory will be reclaim target soon, again.... > And if I understand correctly, the problem you are referring to is > this: an application and parts of a soft-limited container share a > node, the soft limit setting means that the container's pages on that > node are reclaimed harder. At that point, the container's share on > that node becomes tiny, but since the soft limit is oblivious to > nodes, the expansion of the other application pushes the soft-limited > container off that node completely as long as the container stays > above its soft limit with the usage on other nodes. > > What would you think about having node-local soft limits that take the > node size into account? > > local_soft_limit = soft_limit * node_size / memcg_size > > The soft limit can be exceeded globally, but the container is no > longer pushed off a node on which it's only occupying a small share of > memory. > Yes, I think this kind of care is required. What is the 'node_size' here ? size of pgdat ? size of per-node usage in the memcg ? > Putting it into proportion of the memcg size, not overall memory size > has the following advantages: > > 1. if the container is sitting on only one of several available > nodes without exceeding the limit globally, the memcg will not be > reclaimed harder just because it has a relatively large share of the > node. > > 2. if the soft limit excess is ridiculously high, the local soft > limits will be pushed down, so the tolerance for smaller shares on > nodes goes down in proportion to the global soft limit excess. > > Example: > > 4G soft limit * 2G node / 4G container = 2G node-local limit > > The container is globally within its soft limit, so the local limit is > at least the size of the node. It's never reclaimed harder compared > to other applications on the node. > > 4G soft limit * 2G node / 5G container = ~1.6G node-local limit > > Here, it will experience more pressure initially, but it will level > off when the shrinking usage and the thereby increasing node-local > soft limit meet. From that point on, the container and the competing > application will be treated equally during reclaim. > > Finally, if the container is 16G in size, i.e. 300% in excess, the > per-node tolerance is at 512M node-local soft limit, which IMO strikes > a good balance between zero tolerance and still applying some stress > to the hugely oversized container when other applications (with > virtually unlimited soft limits) want to run on the same node. > > What do you think? I like the idea. Another idea is changing 'priority' based on per-node stats if not too complicated... Thanks, -Kame -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>