----- Original Message ----- > From: "marc eshel" <eshel.marc@xxxxxxxxx> > To: "Tigran Mkrtchyan" <tigran.mkrtchyan@xxxxxxx>, "Olga Kornievskaia" <aglo@xxxxxxxxx> > Cc: "schumaker anna" <schumaker.anna@xxxxxxxxx>, "linux-nfs" <linux-nfs@xxxxxxxxxxxxxxx>, "Trond Myklebust" > <trondmy@xxxxxxxxxxxxxxx> > Sent: Sunday, 11 August, 2024 21:50:36 > Subject: Re: NFS client to pNFS DS > On 8/11/24 12:41 PM, Mkrtchyan, Tigran wrote: >>> On 10. Aug 2024, at 23:20, Olga Kornievskaia <aglo@xxxxxxxxx> wrote: >>> >>> On Fri, Aug 9, 2024 at 12:27 PM marc eshel <eshel.marc@xxxxxxxxx> wrote: >>>> >>>> On 8/9/24 8:15 AM, Olga Kornievskaia wrote: >>>>> On Fri, Aug 9, 2024 at 10:09 AM marc eshel <eshel.marc@xxxxxxxxx> wrote: >>>>>> Thanks for the replies, I am a little rusty with debugging NFS but this what I >>>>>> see when the NFS client tried to create a session with the DS. >>>>>> >>>>>> Ganesha was configured for sec=sys and the client mount had the option sec=sys, >>>>>> I assume flavor 390004 means it was trying to use krb5i. >>>>> For 4.1, the client will always try to do state operations with krb5i >>>>> even when sec=sys when the client detects that it's configured to do >>>>> Kerberos (ie., gssd is running). This context creation is triggered >>>>> regardless of whether the rpc client is used for MDS or DS. >>>>> >>>>> My question to you: is the MDS configured with Kerberos but the DS >>>>> isn't? And also, does this lead to a failure? >>>> Both MDS DS are configured for sec=sys and it is leading to client >>>> switching from DS to MDS so yes, it is pNFS failure. What I see on the >>>> DS is the client creating a session and than imminently destroying it >>>> before committing it. If the is something else that I can debug I will >>>> be happy to. >>> pnfs failure is unexpected. I'm pretty confident that a non-kerberos >>> configured client can do normal pnfs with sec=sys. I can help debug, >> Yes, I can confirm that. All RHEL kernels and weekly -rc kernels from Linus >> works as expected. We mount always with sec=sys, and despite the fact, that >> hosts configured to support kerberos due to AFS and GPFS, the access to DSes is >> never an issue and never tries to use kerberos. >> >> Tigran. > > Hi Tigran, > > It is possible that it is failing back to MDS for another reason and the > error message that I see is not the reason for the failure. Are you > using pNFS with Ganesha and GPFS? > > Marc. Hi Marc, pNFS is used exclusively with dCache. Can you provide the packet capture of failing pNFS traffic, like mount + cp + umount? Best regards, Tigran. > >> >>> if you want to send me a network trace and tracepoint output. Btw what >>> kernel/distro are you using? >>> >>>> Marc. >>>> >>>>>> Jul 30 11:10:58 svl-marcrh-node-1 kernel: RPC: Couldn't create >>>>>> auth handle (flavor 390004) >>>>>> >>>>>> Marc. >>>>>> >>>>>> On 8/9/24 6:06 AM, Anna Schumaker wrote: >>>>>> >>>>>> On Thu, Aug 8, 2024 at 6:07 PM Olga Kornievskaia <aglo@xxxxxxxxx> wrote: >>>>>> >>>>>> On Mon, Aug 5, 2024 at 5:51 PM marc eshel <eshel.marc@xxxxxxxxx> wrote: >>>>>> >>>>>> Hi Trond, >>>>>> >>>>>> Will the Linux NFS client try to us krb5i regardless of the MDS >>>>>> configuration? >>>>>> >>>>>> Is there any option to avoid it? >>>>>> >>>>>> I was under the impression the linux client has no way of choosing a >>>>>> different auth_gss security flavor for the DS than the MDS. Meaning >>>>>> >>>>>> That's a good point, I completely missed that this is specifically for the DS. >>>>>> >>>>>> that if mount command has say sec=krb5i then both MDS and DS >>>>>> connections have to do krb5i and if say the DS isn't configured for >>>>>> Kerberos, then IO would fallback to MDS. I no longer have a pnfs >>>>>> >>>>>> That's what I would expect, too. >>>>>> >>>>>> server to verify whether or not what I say is true but that is what my >>>>>> memory tells me is the case. >>>>>> >>>>>> >>>>>> Thanks, Marc. >>>>>> >>>>>> ul 30 11:10:58 svl-marcrh-node-1 kernel: nfs4_fl_alloc_deviceid_node >>>>>> stripe count 1 >>>>>> Jul 30 11:10:58 svl-marcrh-node-1 kernel: nfs4_fl_alloc_deviceid_node >>>>>> ds_num 1 >>>>>> Jul 30 11:10:58 svl-marcrh-node-1 kernel: RPC: Couldn't create >>>>>> auth handle (flavor 390004) >>>>>>
Attachment:
smime.p7s
Description: S/MIME Cryptographic Signature
