On Sat, Feb 06, 2021 at 04:11:46PM +0100, Pavel Raiskup wrote:
> I'm trying to overcommit storage space on one our hypervisors (and reclaim
> space if possible after memory peak times) and it seems like kernel in
> Fedora 33 isn't doing automatic discard on swap partitions? The swap is
> enabled by:
>
> $ swapon --discard /dev/sda
>
> The guest (on RHEL 8 HV host) is started with virtio-scsi controller to
> propagate the trim requests to the host storage, the volumes I tried were
> both of raw and qcow2 format.
>
> The 'fstrim -av' isn't propagated to SWAP mount points (expected
> probably). The only thing that helps to shrink the host volume allocated
> for guest swap is 'swapoff -a && swapon --discard /dev/sda' (but I assume
> TRIM is sent to the driver in this case, and that the problem isn't in the
> host libvirt setup).
fstrim will only trim a regular filesystem. AFAIK swap is not
involved in any way when you do fstrim. On the other hand swapon --discard
is correct, but you may need to check that --discard=pages was
selected. I have no idea what particular operations will cause a TRIM
to actually be sent though - it could also depend on the kernel
version. It seems unlikely that the kernel would send TRIM for every
single freed page because that would be very expensive on some
devices.
If you want to monitor what requests are seen on the hypervisor side,
I'd suggest running this command on the hypervisor:
nbdkit memory 10G --filter=log logfile=/var/tmp/log
Modify the guest's libvirt XML (virsh edit guestname) to add:
<disk type='network' device='disk'>
<driver name='qemu' type='raw'/>
<source protocol='nbd'>
<host name='localhost' port='10809'/>
</source>
<target dev='sdb' bus='scsi'/>
</disk>
(You might have to open port 10809 on the firewall, or else use
a Unix domain socket)
Then put the swap on /dev/sdb in the guest.
Then do whatever experiments you need and you should see (or perhaps
not see) Trim requests in /var/tmp/log.
You might nee dto play with <driver ... discard="unmap" ...> setting
in the libvirt XML too. I can't remember if that's the default or
not.
Rich.
> But otherwise, freeing the guest memory manually, minimizing the swap usage,
> etc. (experiments with tmpfs) doesn't result in freeing the allocated storage
> blocks on host.
>
> Am I doing something wrong, is this expected or is this a bug?
>
> FTR, I tested "kernel-core-5.10.11-200.fc33.x86_64" and
> "kernel-core-5.10.12-200.fc33.x86_64" but this is not a regression -- I'm
> experimenting with TRIM in this setup for the first time.
>
> Dmesg output seems to indicate page cluster trim is enabled (c symbol):
> [ 3936.914260] Adding 33554428k swap on /dev/sda. Priority:-2 extents:1 across:33554428k DscFS
>
> Thanks,
> Pavel
>
>
> _______________________________________________
> devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
