Hi Bruce,
On 28.09.2021 21:39, J. Bruce Fields wrote:
On Tue, Sep 28, 2021 at 07:11:41AM +0200, Salvatore Bonaccorso wrote:
Hi Bruce,
On 27.09.2021 17:53, J. Bruce Fields wrote:
>On Mon, Sep 27, 2021 at 08:10:31AM +0200, Salvatore Bonaccorso wrote:
>>We recently got the following traces on a NFS server, but I'm not sure
>>how to further debug this, any hints?
>
>The server creates and opens a file in two steps, though it should
>really be a single atomic operation.
>
>That means there's a small possibility somebody could intervene and do
>something like change the permissions:
>
>>
>>[5746893.904448] ------------[ cut here ]------------
>>[5746893.910050] nfsd4_process_open2 failed to open
>>newly-created file! status=10008
>
>10008 is NFS4ERR_DELAY, so maybe somebody managed to get a delegation
>before we finished opening?
>
>We should be able to prevent that....
>
>In your setup are there processes quickly opening new files created by
>others?
This is very possible. The NFS server is used as a "scratch" place
accessible from
compute cluster where people can have multiple jobs simultaneously
running through
Slurm and accessing the data. So it is possible that user create new
files from
one running instance and accessing it quickly from the other nodes.
I'm so far was unable to arificially trigger the issue but is there
anything I
can try out to get more information useful for you?
I think the problem's pretty obvious. I'm not sure what the fix should
be.
Hope you come up with the right solution! Thanks for looking into it,
much appreciated.
You can work around it for now by turning off delegations (echo 0
/proc/sys/fs/leases_enable before starting nfsd).
Ack thank you.
Regards,
Salvatore
