Dear Cephalopodians,
we have a classic libvirtd / KVM based virtualization cluster using Ceph-RBD (librbd) as backend and sharing the libvirtd configuration between the nodes via CephFS
(all on Mimic).
To share the libvirtd configuration between the nodes, we have symlinked some folders from /etc/libvirt to their counterparts on /cephfs,
so all nodes see the same configuration.
In general, this works very well (of course, there's a "gotcha": Libvirtd needs reloading / restart for some changes to the XMLs, we have automated that),
but there is one issue caused by Yum's cleverness (that's on CentOS 7). Whenever there's a libvirtd update, unattended upgrades fail, and we see:
Transaction check error:
installing package libvirt-daemon-driver-network-4.5.0-10.el7_6.7.x86_64 needs 2 inodes on the /cephfs filesystem
installing package libvirt-daemon-config-nwfilter-4.5.0-10.el7_6.7.x86_64 needs 18 inodes on the /cephfs filesystem
So it seems yum follows the symlinks and checks the available inodes on /cephfs. Sadly, that reveals:
[root@kvm001 libvirt]# LANG=C df -i /cephfs/
Filesystem Inodes IUsed IFree IUse% Mounted on
ceph-fuse 68 68 0 100% /cephfs
I think that's just because there is no real "limit" on the maximum inodes on CephFS. However, returning 0 breaks some existing tools (notably, Yum).
What do you think? Should CephFS return something different than 0 here to not break existing tools?
Or should the tools behave differently? But one might also argue that if the total number of Inodes matches the used number of Inodes, the FS is indeed "full".
It's just unclear to me who to file a bug against ;-).
Right now, I am just using:
yum -y --setopt=diskspacecheck=0 update
as a manual workaround, but this is naturally rather cumbersome.
Cheers,
Oliver
Attachment:
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
