Re: fstrim issue in VM for cloned rbd image with fast-diff feature

[Youzhong Yang <youzhong@xxxxxxxxx>]

Thanks Jason.

Yes, my concern is that fstrim increases clones' disk usage. The VM didn't use any additional space but fstrim caused its disk usage (in ceph) to go up significantly. Imagine when there are hundreds of VMs, it would soon cause space issue.

If this is expected behavior, does it mean it's better to disable fast-diff from rbd image? I am fine with that.

There is an ugly part of this discard/fstrim feature. At one time while I was doing dd + rm file + fstrim repeatedly, it rendered my VM root file system corrupted. Sadly I couldn't reproduce it again.

Thanks.

On Wed, May 9, 2018 at 11:52 AM, Jason Dillaman <jdillama@xxxxxxxxxx> wrote:

On Wed, May 9, 2018 at 11:39 AM, Youzhong Yang <youzhong@xxxxxxxxx> wrote:
> This is what I did:
>
> # rbd import /var/tmp/debian93-raw.img images/debian93
> # rbd info images/debian93
> rbd image 'debian93':
>  size 81920 MB in 20480 objects
>  order 22 (4096 kB objects)
>  block_name_prefix: rbd_data.384b74b0dc51
>  format: 2
>  features: layering, exclusive-lock, object-map, fast-diff, deep-flatten
>  flags:
>  create_timestamp: Wed May  9 09:31:24 2018
> # rbd snap create images/debian93@snap
> # rbd snap protect images/debian93@snap
> # rbd clone images/debian93@snap vms/debian93.dsk
> # rbd du vms/debian93.dsk
> NAME         PROVISIONED USED
> debian93.dsk      81920M 336M
>
> --- Inside the VM ---
> # df -h /
> Filesystem      Size  Used Avail Use% Mounted on
> /dev/sda1        79G   10G   66G  14% /
> # fstrim -v /
> /: 36.6 GiB (39311650816 bytes) trimmed
>
> --- then rbd du reports ---
> # rbd du vms/debian93.dsk
> NAME         PROVISIONED   USED
> debian93.dsk      81920M 76028M
>
> === If I disable fast-diff feature from images/debian93: ===
> # fstrim -v /
> /: 41 GiB (44059172864 bytes) trimmed
>
> # rbd du vms/debian93.dsk
> warning: fast-diff map is not enabled for debian93.dsk. operation may be
> slow.
> NAME         PROVISIONED  USED
> debian93.dsk      81920M 8612M
>
> === or just flatten vms/debian93.dsk without disabling fast-diff ===
> # rbd du vms/debian93.dsk
> NAME         PROVISIONED   USED
> debian93.dsk      81920M 11992M
>
> # fstrim -v /
> /: 68.7 GiB (73710755840 bytes) trimmed
>
> # rbd du vms/debian93.dsk
> NAME         PROVISIONED   USED
> debian93.dsk      81920M 12000M
>
> Testing environment:
> Ceph: v12.2.5
> OS: Ubuntu 18.04
> QEMU: 2.11
> libvirt: 4.0.0
>
> Is this a known issue? or is the above behavior expected?

What's your concern? I didn't see you state any potential problem. Are
you just concerned that "fstrim" appears to increase your clone's disk
usage? If that's the case, it's expected since "fast-diff" only tracks
the existence of objects (not the per-object usage) and since it's a
cloned image, a discard op results in the creation of a zero-byte
object to "hide" the associated extent within the parent image.

> Thanks,
>
> --Youzhong
>
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>



--
Jason

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com