Re: block: del_gendisk() vs blk_queue_enter() race condition

[Sergey Senozhatsky <senozhatsky@xxxxxxxxxxxx>]

On (24/10/03 23:00), Sergey Senozhatsky wrote:
> On (24/10/03 06:43), Christoph Hellwig wrote:
[..]
> So that mutex_lock(&disk->open_mutex) right before it potentially can
> deadlock (I think it will).
> 
> My idea, thus far, was to
> 
> 	if (test_bit(GD_OWNS_QUEUE, &disk->state)) }
> 		blk_queue_flag_set(QUEUE_FLAG_DYING, disk->queue);
> 		blk_kick_queue_enter(disk->queue);  // this one simply wake_up_all(&q->mq_freeze_wq);
> 											// if the queue has QUEUE_FLAG_DYING
> 	}
> 
> in del_gendisk() before the very first time del_gendisk() attempts to
> mutex_lock(&disk->open_mutex), because that mutex is already locked
> forever.

Well, just in case, the diff that I have (against 6.6)

---

diff --git a/block/blk-core.c b/block/blk-core.c
index 4f25d2c..470c910 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -304,6 +304,13 @@
 	wake_up_all(&q->mq_freeze_wq);
 }
 
+void blk_kick_queue_enter(struct request_queue *q)
+{
+	if (WARN_ON_ONCE(!blk_queue_dying(q)))
+		return;
+	wake_up_all(&q->mq_freeze_wq);
+}
+
 /**
  * blk_queue_enter() - try to increase q->q_usage_counter
  * @q: request queue pointer
diff --git a/block/genhd.c b/block/genhd.c
index 203c880..3ccc593 100644
--- a/block/genhd.c
+++ b/block/genhd.c
@@ -581,9 +581,6 @@
 	if (test_and_set_bit(GD_DEAD, &disk->state))
 		return;
 
-	if (test_bit(GD_OWNS_QUEUE, &disk->state))
-		blk_queue_flag_set(QUEUE_FLAG_DYING, disk->queue);
-
 	/*
 	 * Stop buffered writers from dirtying pages that can't be written out.
 	 */
@@ -641,6 +638,20 @@
 
 	disk_del_events(disk);
 
+	if (test_bit(GD_OWNS_QUEUE, &disk->state)) {
+		/*
+		 * Set QUEUE_FLAG_DYING before we grab ->open_mutex so that
+		 * blkdev_put() -> release -> blk_queue_enter() can detect
+		 * dead device
+		 */
+		blk_queue_flag_set(QUEUE_FLAG_DYING, disk->queue);
+		/*
+		 * Make sure that ->mq_freeze_wq see QUEUE_FLAG_DYING and
+		 * bail out, unlocking ->open_mutex
+		 */
+		blk_kick_queue_enter(disk->queue);
+	}
+
 	/*
 	 * Prevent new openers by unlinked the bdev inode.
 	 */
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 6f67dbe..a49afe9 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -854,6 +854,7 @@
 extern int blk_queue_enter(struct request_queue *q, blk_mq_req_flags_t flags);
 extern void blk_queue_exit(struct request_queue *q);
 extern void blk_sync_queue(struct request_queue *q);
+void blk_kick_queue_enter(struct request_queue *q);
 
 /* Helper to convert REQ_OP_XXX to its string format XXX */
 extern const char *blk_op_str(enum req_op op);