On Thu, Aug 12, 2021 at 5:03 PM Boris Behrens <bb@xxxxxxxxx> wrote:
>
> Hi everybody,
>
> we just stumbled over a problem where the rbd image does not shrink, when
> files are removed.
> This only happenes when the rbd image is partitioned.
>
> * We tested it with centos8/ubuntu20.04 with ext4 and a gpt partition table
> (/boot and /)
> * the kvm device is virtio-scsi-pci with krbd
> * Mount option discard is set
> * command to create large file: dd if=/dev/zero of=testfile bs=64M
> count=1000
> * the image grows in the size we expect
> * when we remove the testfile the rbd image stays at the size
> * we wen recreate the deleted file with the command the rbd image grows
> further
> * using fstrim does not work
> * adding a new disk and initialize the ext4 directly on the disk (wihtout
> partitioning) the trim does work and the rbd image shrinks back to a couple
> GB
> * we use ceph 14.2.21
>
> Does anybody experienced the same issue and maybe know how to solve the
> problem?
Hi Boris,
For discard to work the same way as without partitioning, you need to
make sure that your partitions start at rbd object size boundaries (see
"order" in "rbd info" output). The default object size is 4M (8192
512-byte sectors). Here is an fdisk output for a 512M /boot partition
and the remainder used for /:
Command (m for help): p
Disk /dev/rbd0: 100 GiB, 107374182400 bytes, 209715200 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 65536 bytes / 65536 bytes
Disklabel type: gpt
Disk identifier: 77553519-6CA1-DF48-8242-97F45B9E10C9
Device Start End Sectors Size Type
/dev/rbd0p1 8192 1056767 1048576 512M Linux filesystem
/dev/rbd0p2 1056768 209715166 208658399 99.5G Linux filesystem
Note that "Start" values are wholly divisible by 8192.
Thanks,
Ilya
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
