Hi,
Thanks all for your quick responses. In my enthusiasm to test I might have been masking the problem, plus not knowing what the output of 'fstrim' should actually show.
Firstly, to answer the question, yes I have the relevant libvirt config, plus have set the correct virtio-scsi settings in the image configuration. I am also not expecting to see anything in userland from a 'df'
After re-running the tests from the linked article after an instance was created, it does appear that trimming isn't working, but the number of objects was growing significantly after logging in (almost doubling) and this was masking the effect of trimming.
tl:dr - it does actually appear to be working, but thank you for the responses and also confirming that there was no Ceph-specific configuration required.
On 29 January 2018 at 12:00, Ric Wheeler <rwheeler@xxxxxxxxxx> wrote:
I might have missed something in the question.Fstrim does not free up space at the user level that you see with a normal df.It is meant to let the block device know about all of the space unused by the file system.Regards,RicOn Jan 29, 2018 11:56 AM, "Wido den Hollander" <wido@xxxxxxxx> wrote:
On 01/29/2018 12:29 PM, Nathan Harper wrote:
Hi,
I don't know if this is strictly a Ceph issue, but hoping someone will be able to shed some light. We have an Openstack environment (Ocata) backed onto a Jewel cluster.
We recently ran into some issues with full OSDs but couldn't work out what was filling up the pools.
It appears that fstrim isn't actually working (or being ignored) within the instances:
[centos@with ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 8.0G 1.2G 6.9G 14% /
plenty of unused space that can be reclaimed:
running fstrim:
[centos@with ~]$ sudo fstrim -a -v
/: 6.9 GiB (7416942592 bytes) trimmed
however, running the command again:
[centos@with ~]$ sudo fstrim -a -v
/: 6.9 GiB (7416942592 bytes) trimmed
It depends on the FS you use inside the VM.
iirc XFS will always tell you the total free space of the FS being trimmed and ext4 only the actual bytes which are trimmed.
The discard option was enabled following:
https://ceph.com/geen-categorie/openstack-and-ceph-rbd-disca rd/
Have I missed anything at the Ceph layer to enable this? Is there anything I can do to see if the request is making it's way through to the Ceph layer?
Try using 'rbd du' and see if it makes a difference.
You also want to check if the XML definition of libvirt has unmap enabled although I expect it does since you don't see any errors.
Wido
______________________________
--
*NathanHarper* // IT Systems Lead
*e: *nathan.harper@xxxxxxxxxxx <mailto:nathan.harper@xxxxxxxx.uk > *t*: 0117 906 1104 *m*: 0787 551 0891 *w: *www.cfms.org.uk <http://www.cfms.org.uk/>
CFMS Services Ltd // Bristol & Bath Science Park // Dirac Crescent // Emersons Green // Bristol // BS16 7FR
CFMS Services Ltd is registered in England and Wales No 05742022 - a subsidiary of CFMS Ltd
CFMS Services Ltd registered office // 43 Queens Square // Bristol // BS1 4QP
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph. com
Nathan Harper // IT Systems Lead
CFMS Services Ltd // Bristol & Bath Science Park // Dirac Crescent // Emersons Green // Bristol // BS16 7FR
CFMS Services Ltd is registered in England and Wales No 05742022 - a subsidiary of CFMS Ltd
CFMS Services Ltd registered office // 43 Queens Square // Bristol // BS1 4QP
CFMS Services Ltd registered office // 43 Queens Square // Bristol // BS1 4QP
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
