Re: [PATCH v1 06/10] bcache: stop dc->writeback_rate_update, dc->writeback_thread earlier

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



From: Tang Junhui <tang.junhui@xxxxxxxxxx>

Hello Coly,
Thanks for your works!

Acctually stopping write-back thread and writeback_rate_update work in 
bcache_device_detach() has already done in:
https://github.com/mlyle/linux/commit/397d02e162b8ee11940a4e9f45e16fee0650d64e

Is it nessary to add "rate_update_canceled" to identify that
whether writeback_rate_update work are already stoped or not? 
I think it is ok to call cancel_delayed_work_sync(&dc->writeback_rate_update)
twice before the memory of writeback_rate_update being released (It will return
-ENOENT).

>Delayed worker dc->writeback_rate_update and kernel thread
>dc->writeback_thread reference cache set data structure in their routine,
>Therefor, before they are stopped, cache set should not be release. Other-
>wise, NULL pointer deference will be triggered.
>
>Currenly delayed worker dc->writeback_rate_update and kernel thread
>dc->writeback_thread are stopped in cached_dev_free(). When cache set is
>retiring by too many I/O errors, cached_dev_free() is called when refcount
>of bcache device's closure (disk.cl) reaches 0. In most of cases, last
>refcount of disk.cl is dropped in last line of cached_dev_detach_finish().
>But in cached_dev_detach_finish() before calling closure_put(&dc->disk.cl),
>bcache_device_detach() is called, and inside bcache_device_detach()
>refcount of cache_set->caching is dropped by closure_put(&d->c->caching).
>
>It is very probably this is the last refcount of this closure, so routine
>cache_set_flush() will be called (it is set in __cache_set_unregister()),
>and its parent closure cache_set->cl may also drop its last refcount and
>cache_set_free() is called too. In cache_set_free() the last refcount of
>cache_set->kobj is dropped and then bch_cache_set_release() is called. Now
>in bch_cache_set_release(), the memory of struct cache_set is freeed.
>
>bch_cache_set_release() is called before cached_dev_free(), then there is a
>time window after cache set memory freed and before dc->writeback_thread
>and dc->writeback_rate_update stopped, if one of them is scheduled to run,
>a NULL pointer deference will be triggered.
>
>This patch fixes the above problem by stopping dc->writeback_thread and
>dc->writeback_rate_update earlier in bcache_device_detach() before calling
>closure_put(&d->c->caching). Because cancel_delayed_work_sync() and
>kthread_stop() are synchronized operations, we can make sure cache set
>is available when the delayed work and kthread are stopping.
>
>Because cached_dev_free() can also be called by writing 1 to sysfs file
>/sys/block/bcache<N>/bcache/stop, this code path may not call
>bcache_device_detach() if d-c is NULL. So stopping dc->writeback_thread
>and dc->writeback_rate_update in cached_dev_free() is still necessary. In
>order to avoid stop them twice, dc->rate_update_canceled is added to
>indicate dc->writeback_rate_update is canceled, and dc->writeback_thread
>is set to NULL to indicate it is stopped.
>
>Signed-off-by: Coly Li <colyli@xxxxxxx>
>---
> drivers/md/bcache/bcache.h    |  1 +
> drivers/md/bcache/super.c     | 21 +++++++++++++++++++--
> drivers/md/bcache/writeback.c |  1 +
> 3 files changed, 21 insertions(+), 2 deletions(-)
>
>diff --git a/drivers/md/bcache/bcache.h b/drivers/md/bcache/bcache.h
>index 83c569942bd0..395b87942a2f 100644
>--- a/drivers/md/bcache/bcache.h
>+++ b/drivers/md/bcache/bcache.h
>@@ -322,6 +322,7 @@ struct cached_dev {
> 
>     struct bch_ratelimit    writeback_rate;
>     struct delayed_work    writeback_rate_update;
>+    bool            rate_update_canceled;
> 
>     /*
>      * Internal to the writeback code, so read_dirty() can keep track of
>diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
>index 5401d2356aa3..8912be4165c5 100644
>--- a/drivers/md/bcache/super.c
>+++ b/drivers/md/bcache/super.c
>@@ -696,8 +696,20 @@ static void bcache_device_link(struct bcache_device *d, struct cache_set *c,
> 
> static void bcache_device_detach(struct bcache_device *d)
> {
>+    struct cached_dev *dc;
>+
>     lockdep_assert_held(&bch_register_lock);
> 
>+    dc = container_of(d, struct cached_dev, disk);
>+    if (!IS_ERR_OR_NULL(dc->writeback_thread)) {
>+        kthread_stop(dc->writeback_thread);
>+        dc->writeback_thread = NULL;
>+    }
>+    if (!dc->rate_update_canceled) {
>+        cancel_delayed_work_sync(&dc->writeback_rate_update);
>+        dc->rate_update_canceled = true;
>+    }
>+
>     if (test_bit(BCACHE_DEV_DETACHING, &d->flags)) {
>         struct uuid_entry *u = d->c->uuids + d->id;
> 
>@@ -1071,9 +1083,14 @@ static void cached_dev_free(struct closure *cl)
> {
>     struct cached_dev *dc = container_of(cl, struct cached_dev, disk.cl);
> 
>-    cancel_delayed_work_sync(&dc->writeback_rate_update);
>-    if (!IS_ERR_OR_NULL(dc->writeback_thread))
>+    if (!dc->rate_update_canceled) {
>+        cancel_delayed_work_sync(&dc->writeback_rate_update);
>+        dc->rate_update_canceled = true;
>+    }
>+    if (!IS_ERR_OR_NULL(dc->writeback_thread)) {
>         kthread_stop(dc->writeback_thread);
>+        dc->writeback_thread = NULL;
>+    }
>     if (dc->writeback_write_wq)
>         destroy_workqueue(dc->writeback_write_wq);
> 
>diff --git a/drivers/md/bcache/writeback.c b/drivers/md/bcache/writeback.c
>index 745d9b2a326f..ab2ac3d72393 100644
>--- a/drivers/md/bcache/writeback.c
>+++ b/drivers/md/bcache/writeback.c
>@@ -548,6 +548,7 @@ void bch_cached_dev_writeback_init(struct cached_dev *dc)
>     dc->writeback_rate_i_term_inverse = 10000;
> 
>     INIT_DELAYED_WORK(&dc->writeback_rate_update, update_writeback_rate);
>+    dc->rate_update_canceled = false;
> }
 

Thanks,
Tang Junhui



[Index of Archives]     [Linux RAID]     [Linux SCSI]     [Linux ATA RAID]     [IDE]     [Linux Wireless]     [Linux Kernel]     [ATH6KL]     [Linux Bluetooth]     [Linux Netdev]     [Kernel Newbies]     [Security]     [Git]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Device Mapper]

  Powered by Linux