Patch "sched/fair: Fix unthrottle_cfs_rq() for leaf_cfs_rq list" has been added to the 5.4-stable tree

[Sasha Levin <sashal@xxxxxxxxxx>]

This is a note to let you know that I've just added the patch titled

    sched/fair: Fix unthrottle_cfs_rq() for leaf_cfs_rq list

to the 5.4-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     sched-fair-fix-unthrottle_cfs_rq-for-leaf_cfs_rq-lis.patch
and it can be found in the queue-5.4 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable@xxxxxxxxxxxxxxx> know about it.



commit 9606e522f217cd13aaeb262501da1b0245adf94a
Author: Vincent Guittot <vincent.guittot@xxxxxxxxxx>
Date:   Wed Dec 2 19:44:23 2020 -0500

    sched/fair: Fix unthrottle_cfs_rq() for leaf_cfs_rq list
    
    [ Upstream commit 39f23ce07b9355d05a64ae303ce20d1c4b92b957 ]
    
    Although not exactly identical, unthrottle_cfs_rq() and enqueue_task_fair()
    are quite close and follow the same sequence for enqueuing an entity in the
    cfs hierarchy. Modify unthrottle_cfs_rq() to use the same pattern as
    enqueue_task_fair(). This fixes a problem already faced with the latter and
    add an optimization in the last for_each_sched_entity loop.
    
    Fixes: fe61468b2cb (sched/fair: Fix enqueue_task_fair warning)
    Reported-by Tao Zhou <zohooouoto@xxxxxxxxxxx>
    Signed-off-by: Vincent Guittot <vincent.guittot@xxxxxxxxxx>
    Signed-off-by: Peter Zijlstra (Intel) <peterz@xxxxxxxxxxxxx>
    Reviewed-by: Phil Auld <pauld@xxxxxxxxxx>
    Reviewed-by: Ben Segall <bsegall@xxxxxxxxxx>
    Link: https://lkml.kernel.org/r/20200513135528.4742-1-vincent.guittot@xxxxxxxxxx
    Signed-off-by: Sasha Levin <sashal@xxxxxxxxxx>

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 200e121101097..3dd7c10d6a582 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -4580,7 +4580,6 @@ void unthrottle_cfs_rq(struct cfs_rq *cfs_rq)
 	struct rq *rq = rq_of(cfs_rq);
 	struct cfs_bandwidth *cfs_b = tg_cfs_bandwidth(cfs_rq->tg);
 	struct sched_entity *se;
-	int enqueue = 1;
 	long task_delta, idle_task_delta;
 
 	se = cfs_rq->tg->se[cpu_of(rq)];
@@ -4604,21 +4603,41 @@ void unthrottle_cfs_rq(struct cfs_rq *cfs_rq)
 	idle_task_delta = cfs_rq->idle_h_nr_running;
 	for_each_sched_entity(se) {
 		if (se->on_rq)
-			enqueue = 0;
+			break;
+		cfs_rq = cfs_rq_of(se);
+		enqueue_entity(cfs_rq, se, ENQUEUE_WAKEUP);
+
+		cfs_rq->h_nr_running += task_delta;
+		cfs_rq->idle_h_nr_running += idle_task_delta;
 
+		/* end evaluation on encountering a throttled cfs_rq */
+		if (cfs_rq_throttled(cfs_rq))
+			goto unthrottle_throttle;
+	}
+
+	for_each_sched_entity(se) {
 		cfs_rq = cfs_rq_of(se);
-		if (enqueue)
-			enqueue_entity(cfs_rq, se, ENQUEUE_WAKEUP);
+
 		cfs_rq->h_nr_running += task_delta;
 		cfs_rq->idle_h_nr_running += idle_task_delta;
 
+
+		/* end evaluation on encountering a throttled cfs_rq */
 		if (cfs_rq_throttled(cfs_rq))
-			break;
+			goto unthrottle_throttle;
+
+		/*
+		 * One parent has been throttled and cfs_rq removed from the
+		 * list. Add it back to not break the leaf list.
+		 */
+		if (throttled_hierarchy(cfs_rq))
+			list_add_leaf_cfs_rq(cfs_rq);
 	}
 
-	if (!se)
-		add_nr_running(rq, task_delta);
+	/* At this point se is NULL and we are at root level*/
+	add_nr_running(rq, task_delta);
 
+unthrottle_throttle:
 	/*
 	 * The cfs_rq_throttled() breaks in the above iteration can result in
 	 * incomplete leaf list maintenance, resulting in triggering the
@@ -4627,7 +4646,8 @@ void unthrottle_cfs_rq(struct cfs_rq *cfs_rq)
 	for_each_sched_entity(se) {
 		cfs_rq = cfs_rq_of(se);
 
-		list_add_leaf_cfs_rq(cfs_rq);
+		if (list_add_leaf_cfs_rq(cfs_rq))
+			break;
 	}
 
 	assert_list_leaf_cfs_rq(rq);