Patch "blk-throttle: Fix io statistics for cgroup v1" has been added to the 6.4-stable tree

[Sasha Levin <sashal@xxxxxxxxxx>]

This is a note to let you know that I've just added the patch titled

    blk-throttle: Fix io statistics for cgroup v1

to the 6.4-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     blk-throttle-fix-io-statistics-for-cgroup-v1.patch
and it can be found in the queue-6.4 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable@xxxxxxxxxxxxxxx> know about it.



commit 489df5e1cb021511fd781b5f4c979afc0f56cfd3
Author: Jinke Han <hanjinke.666@xxxxxxxxxxxxx>
Date:   Mon May 8 01:06:31 2023 +0800

    blk-throttle: Fix io statistics for cgroup v1
    
    [ Upstream commit ad7c3b41e86b59943a903d23c7b037d820e6270c ]
    
    After commit f382fb0bcef4 ("block: remove legacy IO schedulers"),
    blkio.throttle.io_serviced and blkio.throttle.io_service_bytes become
    the only stable io stats interface of cgroup v1, and these statistics
    are done in the blk-throttle code. But the current code only counts the
    bios that are actually throttled. When the user does not add the throttle
    limit, the io stats for cgroup v1 has nothing. I fix it according to the
    statistical method of v2, and made it count all ios accurately.
    
    Fixes: a7b36ee6ba29 ("block: move blk-throtl fast path inline")
    Tested-by: Andrea Righi <andrea.righi@xxxxxxxxxxxxx>
    Signed-off-by: Jinke Han <hanjinke.666@xxxxxxxxxxxxx>
    Acked-by: Muchun Song <songmuchun@xxxxxxxxxxxxx>
    Acked-by: Tejun Heo <tj@xxxxxxxxxx>
    Link: https://lore.kernel.org/r/20230507170631.89607-1-hanjinke.666@xxxxxxxxxxxxx
    Signed-off-by: Jens Axboe <axboe@xxxxxxxxx>
    Signed-off-by: Sasha Levin <sashal@xxxxxxxxxx>

diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c
index aaf9903ad7b2f..fc49be622e05b 100644
--- a/block/blk-cgroup.c
+++ b/block/blk-cgroup.c
@@ -2086,6 +2086,9 @@ void blk_cgroup_bio_start(struct bio *bio)
 	struct blkg_iostat_set *bis;
 	unsigned long flags;
 
+	if (!cgroup_subsys_on_dfl(io_cgrp_subsys))
+		return;
+
 	/* Root-level stats are sourced from system-wide IO stats */
 	if (!cgroup_parent(blkcg->css.cgroup))
 		return;
@@ -2116,8 +2119,7 @@ void blk_cgroup_bio_start(struct bio *bio)
 	}
 
 	u64_stats_update_end_irqrestore(&bis->sync, flags);
-	if (cgroup_subsys_on_dfl(io_cgrp_subsys))
-		cgroup_rstat_updated(blkcg->css.cgroup, cpu);
+	cgroup_rstat_updated(blkcg->css.cgroup, cpu);
 	put_cpu();
 }
 
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index 9d010d867fbf4..7397ff199d669 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -2178,12 +2178,6 @@ bool __blk_throtl_bio(struct bio *bio)
 
 	rcu_read_lock();
 
-	if (!cgroup_subsys_on_dfl(io_cgrp_subsys)) {
-		blkg_rwstat_add(&tg->stat_bytes, bio->bi_opf,
-				bio->bi_iter.bi_size);
-		blkg_rwstat_add(&tg->stat_ios, bio->bi_opf, 1);
-	}
-
 	spin_lock_irq(&q->queue_lock);
 
 	throtl_update_latency_buckets(td);
diff --git a/block/blk-throttle.h b/block/blk-throttle.h
index ef4b7a4de987d..d1ccbfe9f7978 100644
--- a/block/blk-throttle.h
+++ b/block/blk-throttle.h
@@ -185,6 +185,15 @@ static inline bool blk_should_throtl(struct bio *bio)
 	struct throtl_grp *tg = blkg_to_tg(bio->bi_blkg);
 	int rw = bio_data_dir(bio);
 
+	if (!cgroup_subsys_on_dfl(io_cgrp_subsys)) {
+		if (!bio_flagged(bio, BIO_CGROUP_ACCT)) {
+			bio_set_flag(bio, BIO_CGROUP_ACCT);
+			blkg_rwstat_add(&tg->stat_bytes, bio->bi_opf,
+					bio->bi_iter.bi_size);
+		}
+		blkg_rwstat_add(&tg->stat_ios, bio->bi_opf, 1);
+	}
+
 	/* iops limit is always counted */
 	if (tg->has_rules_iops[rw])
 		return true;