On 12/19/2013 01:21 PM, Michal Hocko wrote: > On Thu 19-12-13 13:16:01, Vladimir Davydov wrote: >> On 12/19/2013 01:10 PM, Michal Hocko wrote: >>> On Thu 19-12-13 10:37:27, Vladimir Davydov wrote: >>>> On 12/18/2013 09:14 PM, Michal Hocko wrote: >>>>> On Wed 18-12-13 17:16:54, Vladimir Davydov wrote: >>>>>> First, in memcg_create_kmem_cache() we should issue the write barrier >>>>>> after the kmem_cache is initialized, but before storing the pointer to >>>>>> it in its parent's memcg_params. >>>>>> >>>>>> Second, we should always issue the read barrier after >>>>>> cache_from_memcg_idx() to conform with the write barrier. >>>>>> >>>>>> Third, its better to use smp_* versions of barriers, because we don't >>>>>> need them on UP systems. >>>>> Please be (much) more verbose on Why. Barriers are tricky and should be >>>>> documented accordingly. So if you say that we should issue a barrier >>>>> always be specific why we should do it. >>>> In short, we have kmem_cache::memcg_params::memcg_caches is an array of >>>> pointers to per-memcg caches. We access it lock-free so we should use >>>> memory barriers during initialization. Obviously we should place a write >>>> barrier just before we set the pointer in order to make sure nobody will >>>> see a partially initialized structure. Besides there must be a read >>>> barrier between reading the pointer and accessing the structure, to >>>> conform with the write barrier. It's all that similar to rcu_assign and >>>> rcu_deref. Currently the barrier usage looks rather strange: >>>> >>>> memcg_create_kmem_cache: >>>> initialize kmem >>>> set the pointer in memcg_caches >>>> wmb() // ??? >>>> >>>> __memcg_kmem_get_cache: >>>> <...> >>>> read_barrier_depends() // ??? >>>> cachep = root_cache->memcg_params->memcg_caches[memcg_id] >>>> <...> >>> Why do we need explicit memory barriers when we can use RCU? >>> __memcg_kmem_get_cache already dereferences within rcu_read_lock. >> Because it's not RCU, IMO. RCU implies freeing the old version after a >> grace period, while kmem_caches are freed immediately. We simply want to >> be sure the kmem_cache is fully initialized. And we do not require >> calling this in an RCU critical section. > And you can use rcu_dereference and rcu_assign for that as well. rcu_dereference() will complain if called outside an RCU critical section, while cache_from_memcg_idx() is called w/o RCU protection from some places. > It hides all the juicy details about memory barriers. IMO, a memory barrier with a good comment looks better than an rcu_dereference() without RCU protection :-) > Besides that nothing prevents us from freeing from rcu callback. Or? It's an overhead we can live without there. The point is that we can access a cache only if it is active. I mean no allocation can go from a cache that has already been destroyed. It would be a bug. So there is no point in introducing RCU-protection for kmem_caches there. It would only confuse, IMO. Thanks. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>