Get rid of preempt_disable() and preempt_enable() when the
preload is done for splitting purpose. The reason is that
calling spin_lock() with disabled preemtion is forbidden in
CONFIG_PREEMPT_RT kernel.
Therefore, we do not guarantee that a CPU is preloaded, instead
we minimize the case when it is not with this change.
For example i run the special test case that follows the preload
pattern and path. 20 "unbind" threads run it and each does
1000000 allocations. Only 3.5 times among 1000000 a CPU was
not preloaded thus. So it can happen but the number is rather
negligible.
Fixes: 82dd23e84be3 ("mm/vmalloc.c: preload a CPU with one object for split purpose")
Signed-off-by: Uladzislau Rezki (Sony) <urezki@xxxxxxxxx>
---
mm/vmalloc.c | 17 ++++++++---------
1 file changed, 8 insertions(+), 9 deletions(-)
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index e92ff5f7dd8b..2ed6fef86950 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -1078,9 +1078,12 @@ static struct vmap_area *alloc_vmap_area(unsigned long size,
retry:
/*
- * Preload this CPU with one extra vmap_area object to ensure
- * that we have it available when fit type of free area is
- * NE_FIT_TYPE.
+ * Preload this CPU with one extra vmap_area object. It is used
+ * when fit type of free area is NE_FIT_TYPE. Please note, it
+ * does not guarantee that an allocation occurs on a CPU that
+ * is preloaded, instead we minimize the case when it is not.
+ * It can happen because of migration, because there is a race
+ * until the below spinlock is taken.
*
* The preload is done in non-atomic context, thus it allows us
* to use more permissive allocation masks to be more stable under
@@ -1089,20 +1092,16 @@ static struct vmap_area *alloc_vmap_area(unsigned long size,
* Even if it fails we do not really care about that. Just proceed
* as it is. "overflow" path will refill the cache we allocate from.
*/
- preempt_disable();
- if (!__this_cpu_read(ne_fit_preload_node)) {
- preempt_enable();
+ if (!this_cpu_read(ne_fit_preload_node)) {
pva = kmem_cache_alloc_node(vmap_area_cachep, GFP_KERNEL, node);
- preempt_disable();
- if (__this_cpu_cmpxchg(ne_fit_preload_node, NULL, pva)) {
+ if (this_cpu_cmpxchg(ne_fit_preload_node, NULL, pva)) {
if (pva)
kmem_cache_free(vmap_area_cachep, pva);
}
}
spin_lock(&vmap_area_lock);
- preempt_enable();
/*
* If an allocation fails, the "vend" address is
--
2.20.1
