Great, thanks for the update. Jason On Fri, Apr 13, 2018 at 11:06 PM, Alex Gorbachev <ag@xxxxxxxxxxxxxxxxxxx> wrote: > On Thu, Apr 12, 2018 at 9:38 AM, Alex Gorbachev <ag@xxxxxxxxxxxxxxxxxxx> wrote: >> On Thu, Apr 12, 2018 at 7:57 AM, Jason Dillaman <jdillama@xxxxxxxxxx> wrote: >>> If you run "partprobe" after you resize in your second example, is the >>> change visible in "parted"? >> >> No, partprobe does not help: >> >> root@lumd1:~# parted /dev/nbd2 p >> Model: Unknown (unknown) >> Disk /dev/nbd2: 2147MB >> Sector size (logical/physical): 512B/512B >> Partition Table: loop >> Disk Flags: >> >> Number Start End Size File system Flags >> 1 0.00B 2147MB 2147MB xfs >> >> root@lumd1:~# partprobe >> root@lumd1:~# parted /dev/nbd2 p >> Model: Unknown (unknown) >> Disk /dev/nbd2: 2147MB >> Sector size (logical/physical): 512B/512B >> Partition Table: loop >> Disk Flags: >> >> Number Start End Size File system Flags >> 1 0.00B 2147MB 2147MB xfs >> >> >> >>> >>> On Wed, Apr 11, 2018 at 11:01 PM, Alex Gorbachev <ag@xxxxxxxxxxxxxxxxxxx> wrote: >>>> On Wed, Apr 11, 2018 at 2:13 PM, Jason Dillaman <jdillama@xxxxxxxxxx> wrote: >>>>> I've tested the patch on both 4.14.0 and 4.16.0 and it appears to >>>>> function correctly for me. parted can see the newly added free-space >>>>> after resizing the RBD image and our stress tests once again pass >>>>> successfully. Do you have any additional details on the issues you are >>>>> seeing? >>>> >>>> I recompiled again with 4.14-24 and tested, the resize shows up OK >>>> when the filesystem is not mounted. dmesg shows also the "detected >>>> capacity change" message. However, if I create a filesystem and mount >>>> it, the capacity change is no longer detected. Steps as follows: >>>> >>>> root@lumd1:~# rbd create -s 1024 --image-format 2 matte/n4 >>>> root@lumd1:~# rbd-nbd map matte/n4 >>>> /dev/nbd2 >>>> root@lumd1:~# mkfs.xfs /dev/nbd2 >>>> meta-data=/dev/nbd2 isize=512 agcount=4, agsize=65536 blks >>>> = sectsz=512 attr=2, projid32bit=1 >>>> = crc=1 finobt=1, sparse=0 >>>> data = bsize=4096 blocks=262144, imaxpct=25 >>>> = sunit=0 swidth=0 blks >>>> naming =version 2 bsize=4096 ascii-ci=0 ftype=1 >>>> log =internal log bsize=4096 blocks=2560, version=2 >>>> = sectsz=512 sunit=0 blks, lazy-count=1 >>>> realtime =none extsz=4096 blocks=0, rtextents=0 >>>> root@lumd1:~# parted /dev/nbd2 p >>>> Model: Unknown (unknown) >>>> Disk /dev/nbd2: 1074MB >>>> Sector size (logical/physical): 512B/512B >>>> Partition Table: loop >>>> Disk Flags: >>>> >>>> Number Start End Size File system Flags >>>> 1 0.00B 1074MB 1074MB xfs >>>> >>>> root@lumd1:~# rbd resize --pool matte --image n4 --size 2048 >>>> Resizing image: 100% complete...done. >>>> root@lumd1:~# parted /dev/nbd2 p >>>> Model: Unknown (unknown) >>>> Disk /dev/nbd2: 2147MB >>>> Sector size (logical/physical): 512B/512B >>>> Partition Table: loop >>>> Disk Flags: >>>> >>>> Number Start End Size File system Flags >>>> 1 0.00B 2147MB 2147MB xfs >>>> >>>> -- All is well so far, now let's mount the fs >>>> >>>> root@lumd1:~# mount /dev/nbd2 /mnt >>>> root@lumd1:~# rbd resize --pool matte --image n4 --size 3072 >>>> Resizing image: 100% complete...done. >>>> root@lumd1:~# parted /dev/nbd2 p >>>> Model: Unknown (unknown) >>>> Disk /dev/nbd2: 2147MB >>>> Sector size (logical/physical): 512B/512B >>>> Partition Table: loop >>>> Disk Flags: >>>> >>>> Number Start End Size File system Flags >>>> 1 0.00B 2147MB 2147MB xfs >>>> >>>> -- Now the change is not detected >>>> >>>> >>>>> >>>>> On Wed, Apr 11, 2018 at 12:06 PM, Jason Dillaman <jdillama@xxxxxxxxxx> wrote: >>>>>> I'll give it a try locally and see if I can figure it out. Note that >>>>>> this commit [1] also dropped the call to "bd_set_size" within >>>>>> "nbd_size_update", which seems suspicious to me at initial glance. >>>>>> >>>>>> [1] https://github.com/torvalds/linux/commit/29eaadc0364943b6352e8994158febcb699c9f9b#diff-bc9273bcb259fef182ae607a1d06a142L180 >>>>>> >>>>>> On Wed, Apr 11, 2018 at 11:09 AM, Alex Gorbachev <ag@xxxxxxxxxxxxxxxxxxx> wrote: >>>>>>>> On Wed, Apr 11, 2018 at 10:27 AM, Alex Gorbachev <ag@xxxxxxxxxxxxxxxxxxx> wrote: >>>>>>>>> On Wed, Apr 11, 2018 at 2:43 AM, Mykola Golub <to.my.trociny@xxxxxxxxx> wrote: >>>>>>>>>> On Tue, Apr 10, 2018 at 11:14:58PM -0400, Alex Gorbachev wrote: >>>>>>>>>> >>>>>>>>>>> So Josef fixed the one issue that enables e.g. lsblk and sysfs size to >>>>>>>>>>> reflect the correct siz on change. However, partptobe and parted >>>>>>>>>>> still do not detect the change, complete unmap and remap of rbd-nbd >>>>>>>>>>> device and remount of the filesystem is required. >>>>>>>>>> >>>>>>>>>> Does your rbd-nbd include this fix [1], targeted for v12.2.3? >>>>>>>>>> >>>>>>>>>> [1] http://tracker.ceph.com/issues/22172 >>>>>>>>> >>>>>>>>> It should, the rbd-nbd version is 12.2.4 >>>>>>>>> >>>>>>>>> root@lumd1:~# rbd-nbd -v >>>>>>>>> ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) luminous (stable) >>>>>>>>> _______________________________________________ >>>>>>> >>>>>>> On Wed, Apr 11, 2018 at 10:39 AM, Jason Dillaman <jdillama@xxxxxxxxxx> wrote: >>>>>>>> Do you have a preliminary patch that we can test against? >>>>>>>> Jason >>>>>>> >>>>>>> Hi Jason, >>>>>>> >>>>>>> This is the one Josef did, which fixes detection of the new size via >>>>>>> sysfs (lsblk eetc.), but parted still requires complete unmapping and >>>>>>> remapping of the NBD device to detect the change. >>>>>>> >>>>>>> I built a kernel with this patch based on 4.14.24, and also tested >>>>>>> with the latest mainline 4.16.1 - same behavior. >>>>>>> >>>>>>> cc: stable@xxxxxxxxxxxxxxx >>>>>>> Fixes: 639812a ("nbd: don't set the device size until we're connected") >>>>>>> Signed-off-by: Josef Bacik <jbacik@xxxxxx> >>>>>>> --- >>>>>>> drivers/block/nbd.c | 2 ++ >>>>>>> 1 file changed, 2 insertions(+) >>>>>>> >>>>>>> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c >>>>>>> index 86258b00a1d4..7106b98a35fb 100644 >>>>>>> --- a/drivers/block/nbd.c >>>>>>> +++ b/drivers/block/nbd.c >>>>>>> @@ -243,6 +243,8 @@ static void nbd_size_set(struct nbd_device *nbd, >>>>>>> loff_t blocksize, >>>>>>> struct nbd_config *config = nbd->config; >>>>>>> config->blksize = blocksize; >>>>>>> config->bytesize = blocksize * nr_blocks; >>>>>>> + if (nbd->task_recv != NULL) >>>>>>> + nbd_size_update(nbd); >>>>>>> } >>>>>>> >>>>>>> static void nbd_complete_rq(struct request *req) >>>>>>> > > > Josef submitted three patches to nbd.c today, which completely resolve > the problem with the online resize of rbd-nbd device. For > convenience, I am listing them below: > > Fixes: c6a4759ea0c9 ("nbd: add device refcounting") > cc: stable@xxxxxxxxxxxxxxx > Signed-off-by: Josef Bacik <jbacik@xxxxxx> > Tested-by: Alex Gorbachev <ag@xxxxxxxxxxxxxxxxxxx> > --- > drivers/block/nbd.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c > index 86258b00a1d4..e33da3e6aa20 100644 > --- a/drivers/block/nbd.c > +++ b/drivers/block/nbd.c > @@ -174,8 +174,8 @@ static void nbd_dev_remove(struct nbd_device *nbd) > { > struct gendisk *disk = nbd->disk; > if (disk) { > - del_gendisk(disk); > blk_cleanup_queue(disk->queue); > + del_gendisk(disk); > blk_mq_free_tag_set(&nbd->tag_set); > disk->private_data = NULL; > put_disk(disk); > > cc: stable@xxxxxxxxxxxxxxx > Fixes: 639812a ("nbd: don't set the device size until we're connected") > Signed-off-by: Josef Bacik <jbacik@xxxxxx> > Tested-by: Alex Gorbachev <ag@xxxxxxxxxxxxxxxxxxx> > --- > drivers/block/nbd.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c > index e33da3e6aa20..1520383b12f6 100644 > --- a/drivers/block/nbd.c > +++ b/drivers/block/nbd.c > @@ -243,6 +243,8 @@ static void nbd_size_set(struct nbd_device *nbd, > loff_t blocksize, > struct nbd_config *config = nbd->config; > config->blksize = blocksize; > config->bytesize = blocksize * nr_blocks; > + if (nbd->task_recv != NULL) > + nbd_size_update(nbd); > } > > static void nbd_complete_rq(struct request *req) > > Fixes: 29eaadc ("nbd: stop using the bdev everywhere") > cc: stable@xxxxxxxxxxxxxxx > Signed-off-by: Josef Bacik <jbacik@xxxxxx> > Tested-by: Alex Gorbachev <ag@xxxxxxxxxxxxxxxxxxx> > --- > drivers/block/nbd.c | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c > index 1520383b12f6..61dd95aa47fa 100644 > --- a/drivers/block/nbd.c > +++ b/drivers/block/nbd.c > @@ -231,9 +231,15 @@ static void nbd_size_clear(struct nbd_device *nbd) > static void nbd_size_update(struct nbd_device *nbd) > { > struct nbd_config *config = nbd->config; > + struct block_device *bdev = bdget_disk(nbd->disk, 0); > + > blk_queue_logical_block_size(nbd->disk->queue, config->blksize); > blk_queue_physical_block_size(nbd->disk->queue, config->blksize); > set_capacity(nbd->disk, config->bytesize >> 9); > + if (bdev) { > + bd_set_size(bdev, config->bytesize); > + bdput(bdev); > + } > kobject_uevent(&nbd_to_dev(nbd)->kobj, KOBJ_CHANGE); > } > > @@ -1111,7 +1117,6 @@ static int nbd_start_device_ioctl(struct > nbd_device *nbd, struct block_device *b > if (ret) > return ret; > > - bd_set_size(bdev, config->bytesize); > if (max_part) > bdev->bd_invalidated = 1; > mutex_unlock(&nbd->config_lock); -- Jason _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |