Dear devs,
Now bisecting turned out that 3c59366c207e4c6c6569524af606baf017a55c61
is the bad commit for me. Strangely it only affects my dovecot process
accessing data over NFS.
Can you please confirm that this may be a bad commit?
My earlier attached programs may be used to demonstrate/trigger the
issue. It even could be stripped down to minimal operations to trigger
the bug.
Thanks in advance,
Richard
2024-05-23 09:10 időpontban Richard Kojedzinszky ezt írta:
Dear NFS developers,
I am running multiple PODs on a Kubernetes node, they all mount
different NFS shares from the same nfs server. I started to notice
hangups in my dovecot process after I switched to Debian's kernel from
upstream 5.15. You can find Debian bugreport at
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1071501.
So, effectively I am running dovecot in Kubernetes, and dovecot's data
directory is accessed over NFS. Eventually one dovecot process stucks
in nfs4_lookup_revalidate(). From that point, that process cannot be
killed, howewer, other processes can access NFS as normal. Also,
another dovecot process running on the very same node accessing the
same NFS share works too.
Now, I am still in the process of bisecting, howewer, I cannot reliably
trigger the bug. Originally it took a few days after I've noticed a
hanging process. Now I am trying to mimic file operations what dovecot
does in a faster way. Now it seems that it triggers the bug in a few
hours, howewer, during bisects, I can still make mistakes.
I've scheduled many of my applications which use NFS shares to the same
node, to have more NFS load on that node.
I am attaching my simple app which triggers the bug in a few hours, at
least in my lab. I have two dedicated NFS shares for this test case,
and I am running 3 instances of the applications for both shares. Also,
I am running other production applications on the same node which also
use NFS, howewer, I dont experience lockups with them. They are
librenms, prometheus, and a docker private registry. This way I dont
know if running the attached app only is enough to trigger the bug.
Once I have a suspectible commit based on my bisecting process, I will
report it here.
My NFS server is a TrueNAS, based on FreeBSD 13.3.
Thanks in advance,
Richard
