On Tue, Apr 30, 2019 at 8:01 AM Oliver Freyermuth <freyermuth@xxxxxxxxxxxxxxxxxx> wrote: > > Dear Cephalopodians, > > we have a classic libvirtd / KVM based virtualization cluster using Ceph-RBD (librbd) as backend and sharing the libvirtd configuration between the nodes via CephFS > (all on Mimic). > > To share the libvirtd configuration between the nodes, we have symlinked some folders from /etc/libvirt to their counterparts on /cephfs, > so all nodes see the same configuration. > In general, this works very well (of course, there's a "gotcha": Libvirtd needs reloading / restart for some changes to the XMLs, we have automated that), > but there is one issue caused by Yum's cleverness (that's on CentOS 7). Whenever there's a libvirtd update, unattended upgrades fail, and we see: > > Transaction check error: > installing package libvirt-daemon-driver-network-4.5.0-10.el7_6.7.x86_64 needs 2 inodes on the /cephfs filesystem > installing package libvirt-daemon-config-nwfilter-4.5.0-10.el7_6.7.x86_64 needs 18 inodes on the /cephfs filesystem > > So it seems yum follows the symlinks and checks the available inodes on /cephfs. Sadly, that reveals: > [root@kvm001 libvirt]# LANG=C df -i /cephfs/ > Filesystem Inodes IUsed IFree IUse% Mounted on > ceph-fuse 68 68 0 100% /cephfs > > I think that's just because there is no real "limit" on the maximum inodes on CephFS. However, returning 0 breaks some existing tools (notably, Yum). > > What do you think? Should CephFS return something different than 0 here to not break existing tools? > Or should the tools behave differently? But one might also argue that if the total number of Inodes matches the used number of Inodes, the FS is indeed "full". > It's just unclear to me who to file a bug against ;-). > > Right now, I am just using: > yum -y --setopt=diskspacecheck=0 update > as a manual workaround, but this is naturally rather cumbersome. This is fallout from [1]. See discussion on setting f_free to 0 here [2]. In summary, userland tools are trying to be too clever by looking at f_free. [I could be convinced to go back to f_free = ULONG_MAX if there are other instances of this.] [1] https://github.com/ceph/ceph/pull/23323 [2] https://github.com/ceph/ceph/pull/23323#issuecomment-409249911 -- Patrick Donnelly _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |