(2012/03/15 20:24), Glauber Costa wrote: > On 03/15/2012 04:16 AM, KAMEZAWA Hiroyuki wrote: >> (2012/03/14 18:46), Glauber Costa wrote: >> >>> On 03/14/2012 04:28 AM, KAMEZAWA Hiroyuki wrote: >>>> IIUC, in general, even in the processes are in a tree, in major case >>>> of servers, their workloads are independent. >>>> I think FLAT mode is the dafault. 'heararchical' is a crazy thing which >>>> cannot be managed. >>> >>> Better pay attention to the current overall cgroups discussions being >>> held by Tejun then. ([RFD] cgroup: about multiple hierarchies) >>> >>> The topic of whether of adapting all cgroups to be hierarchical by >>> deafult is a recurring one. >>> >>> I personally think that it is not unachievable to make res_counters >>> cheaper, therefore making this less of a problem. >>> >> >> >> I thought of this a little yesterday. Current my idea is applying following >> rule for res_counter. >> >> 1. All res_counter is hierarchical. But behavior should be optimized. >> >> 2. If parent res_counter has UNLIMITED limit, 'usage' will not be propagated >> to its parent at _charge_. > > That doesn't seem to make much sense. If you are unlimited, but your > parent is limited, > he has a lot more interest to know about the charge than you do. Sorry, I should write "If all ancestors are umlimited'. If parent is limited, the children should be treated as limited. > So the > logic should rather be the opposite: Don't go around getting locks and > all that if you are unlimited. Your parent might, though. > > I am trying to experiment a bit with billing to percpu counters for > unlimited res_counters. But their inexact nature is giving me quite a > headache. > Personally, I think percpu counter is not the best one. Yes, it will work but... Because of its nature of error range, it has scalability problem. Considering to have a tree like /A/B/Guest0/tasks Guest1/tasks Guest2/tasks Guest4/tasks Guest5/tasks ...... percpu res_counter may work scarable in GuestX level but will conflict in level B. And I don't want to think what happens in 256 cpu system. Error in B will be very big. Another idea is to borrow a resource from memcg to the tasks. i.e.having per-task caching of charges. But it has two problems that draining unused resource is difficult and precise usage is unknown. IMHO, hard-limited resource counter itself may be a problem ;) So, an idea, 'if all ancestors are unlimited, don't propagate charges.' comes to my mind. With this, people use resource in FLAT (but has hierarchical cgroup tree) will not see any performance problem. >> 3. If a res_counter has UNLIMITED limit, at reading usage, it must visit >> all children and returns a sum of them. >> >> Then, >> /cgroup/ >> memory/ (unlimited) >> libivirt/ (unlimited) >> qeumu/ (unlimited) >> guest/(limited) >> >> All dir can show hierarchical usage and the guest will not have >> any lock contention at runtime. > > If we are okay with summing it up at read time, we may as well > keep everything in percpu counters at all times. > If all ancestors are unlimited, we don't need to propagate usage upwards at charging. If one of ancestors are limited, we need to propagate and check usage at charging. >> By this >> 1. no runtime overhead if the parent has unlimited limit. >> 2. All res_counter can show aggregate resource usage of children. >> >> To do this >> 1. res_coutner should have children list by itself. >> >> Implementation problem >> - What should happens when a user set new limit to a res_counter which have >> childrens ? Shouldn't we allow it ? Or take all locks of children and >> update in atomic ? > Well, increasing the limit should be always possible. > > As for the kids, how about: > > - ) Take their locks > - ) scan through them seeing if their usage is bellow the new allowance > -) if it is, then ok > -) if it is not, then try to reclaim (*). Fail if it is not possible. > > (*) May be hard to implement, because we already have the res_counter > lock taken, and the code may get nasty. So maybe it is better just fail > if any of your kids usage is over the new allowance... > Seems enough and seems worth to try. > > >> - memory.use_hierarchy should be obsolete ? > If we're going fully hierarchical, yes. > Another big problem is 'when' we should do this change.. Maybe this 'hierarchical' problem will be good topic in MM summit. Thanks, -Kame -- To unsubscribe from this list: send the line "unsubscribe cgroups" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html