On 5/12/22 12:06 AM, Stefan Kooman wrote:
Hi List,
We have quite a few linux kernel clients for CephFS. One of our
customers has been running mainline kernels (CentOS 7 elrepo) for the
past two years. They started out with 3.x kernels (default CentOS 7),
but upgraded to mainline when those kernels would frequently generate
MDS warnings like "failing to respond to capability release". That
worked fine until 5.14 kernel. 5.14 and up would use a lot of CPU and
*way* more bandwidth on CephFS than older kernels (order of
magnitude). After the MDS was upgraded from Nautilus to Octopus that
behavior is gone (comparable CPU / bandwidth usage as older kernels).
However, the newer kernels are now the ones that give "failing to
respond to capability release", and worse, clients get evicted
(unresponsive as far as the MDS is concerned). Even the latest 5.17
kernels have that. No difference is observed between using messenger
v1 or v2. MDS version is 15.2.16.
Surprisingly the latest stable kernels from CentOS 7 work flawlessly
now. Although that is good news, newer operating systems come with
newer kernels.
Does anyone else observe the same behavior with newish kernel clients?
There have some known bugs, which have been fixed or under fixing
recently, even in the mainline and, not sure whether are they related.
Such as [1][2][3][4]. More detail please see ceph-client repo testing
branch [5].
I have never see the "failing to respond to capability release" issue
yet, if you have the MDS logs(debug_mds = 25 and debug_ms = 1) and
kernel debug logs will be better to help debug it further, or provide
the steps to reproduce it.
[1] https://tracker.ceph.com/issues/55332
[2] https://tracker.ceph.com/issues/55421
[3] https://bugzilla.redhat.com/show_bug.cgi?id=2063929
[4] https://tracker.ceph.com/issues/55377
[5] https://github.com/ceph/ceph-client/commits/testing
Thanks
-- Xiubo
Gr. Stefan
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
