> On May 22, 2018, at 3:11 PM, Olga Kornievskaia <aglo@xxxxxxxxx> wrote: > > On Tue, May 22, 2018 at 5:44 PM, Chuck Lever <chuck.lever@xxxxxxxxxx> wrote: >> >> >>> On May 22, 2018, at 2:21 PM, Olga Kornievskaia <aglo@xxxxxxxxx> wrote: >>> >>> On Tue, May 22, 2018 at 5:02 PM, Chuck Lever <chuck.lever@xxxxxxxxxx> wrote: >>>> >>>> >>>>> On May 22, 2018, at 1:38 PM, Olga Kornievskaia <aglo@xxxxxxxxx> wrote: >>>>> >>>>> On Tue, May 22, 2018 at 4:22 PM, Chuck Lever <chuck.lever@xxxxxxxxxx> wrote: >>>>>> >>>>>> >>>>>>> On May 22, 2018, at 1:17 PM, Olga Kornievskaia <aglo@xxxxxxxxx> wrote: >>>>>>> >>>>>>> On Tue, May 22, 2018 at 4:08 PM, Chuck Lever <chuck.lever@xxxxxxxxxx> wrote: >>>>>>>> >>>>>>>> >>>>>>>>> On May 22, 2018, at 1:03 PM, Olga Kornievskaia <aglo@xxxxxxxxx> wrote: >>>>>>>>> >>>>>>>>> I'm looking for comments on the approach to deal with the following >>>>>>>>> denial-of-service issue. >>>>>>>>> >>>>>>>>> Currently, during the nfs4.0 mount, the code takes the content >>>>>>>>> supplied by the user in the mount command for "clientaddr" and that >>>>>>>>> becomes part of the content of the SETCLIENTID client id. There are no >>>>>>>>> verifications that the supplied address belongs to the client >>>>>>>>> initiating the mount. >>>>>>>>> >>>>>>>>> A denial of services comes from where there are 2 clients with IP A >>>>>>>>> and IP B (bad one). Client IP A mounts and has "IP A" in the >>>>>>>>> SETCLIENTID. Client IP B does a mount and specified "clientaddr=IP A". >>>>>>>>> This causes the server to invalidate the lease for the legitimate >>>>>>>>> client IP A. >>>>>>>> >>>>>>>> Generally if this is a concern, Kerberos can be used during >>>>>>>> the SETCLIENTID to mutually authenticate the client and >>>>>>>> server. Shouldn't that prevent client B from tampering with >>>>>>>> client A's lease? >>>>>>> >>>>>>> It turns out to be a concern by folks (customers) that are using the >>>>>>> code. Kerberos does not help here. Client IP B can have a valid >>>>>>> Kerberos identity and still supply "clientaddr=" not belonging to it >>>>>>> for the SETCLIENTID and interfere with the other's lease. >>>>>> >>>>>> SETCLIENTID is associated with a client ID string and a Kerberos >>>>>> principal. The server is supposed to deny a client with the same >>>>>> string (and perhaps the same callback information) but a different >>>>>> Kerberos identity from purging an existing lease belonging to a >>>>>> different principal. NFS4ERR_CLID_INUSE. >>>>>> >>>>>> Are you saying the two clients have exactly the same host >>>>>> principal? That seems... wrong. >>>>>> >>>>> >>>>> Are you sure client ID is associated with a Kerberos principal? >>>>> >>>>> Looking ta the code that constructs the clientid content. I don't see >>>>> that cl_nodename takes in principal identity. >>>>> scnprintf(str, len, "Linux NFSv%u.%u %s", >>>>> clp->rpc_ops->version, clp->cl_minorversion, >>>>> clp->cl_rpcclient->cl_nodename); >>>> >>>> That's correct. >>>> >>>> Normally the Linux client picks up the host principal in the >>>> client's keytab and uses that as the credential for lease >>>> management operations like SETCLIENTID, without any regard to >>>> whether sec=sys or sec=krb5-yada is used on the mount command. >>>> The client ID string is not supposed to change between those >>>> cases. >>>> >>>> The server associates the client ID string with the Kerberos >>>> principal the client used to perform the SETCLIENTID. >>> >>> I haven't checked the spec but is this required? >> >> Yes, it is required. That's what the NFS4ERR_CLID_INUSE status >> code is for. >> >> RFC 7530 p. 291: >> >> For any confirmed >> record with the same id string x, if the recorded principal does >> not match that of the SETCLIENTID call, then the server returns an >> NFS4ERR_CLID_INUSE error. >> >> >>>> If a different Kerberos principal is used with a SETCLIENTID >>>> that bears the same client ID string as a client whose lease >>>> is still active, the server is supposed to reject that >>>> SETCLIENTID with NFS4ERR_CLID_INUSE. >>> >>> I have tried (against the linux server), do a mount with krb5 and one >>> without that used the clientaddr of the client with krb5 mount and I >>> could get into the same lease revocation behavior. Which makes me >>> question if indeed the servers do associate Kerberos principal in the >>> SETCLIENTID handling. >> >> That sounds like a bad server bug to me. >> >> Input validation on a client can't possibly be a reliable fix >> for this issue. > > But for auth_sys I believe it is helpful. With AUTH_UNIX, the best you can do is try to ensure that all your clients have unique cl_nodenames. The question of whether the provided callback address is valid is a different matter. "Is this user-provided address one of my local interfaces or _ANY?" and then either warn or fail the mount if not. >> Preventing lease tampering is exactly why the >> Linux client uses krb5i with the host principal for lease >> management whenever it can. >> >> >>>>> I have also tried to do a mount with and without Kerberos and the >>>>> clientid string is that same has NFSv4.0 client ip/server ip. >>>> >>>> A quick way to disable the use of Kerberos for lease management >>>> is to >>>> >>>> sudo mv /etc/krb5.keytab /etc/krb5.keytab.bak >>>> >>>> and then restart rpc.gssd. >>>> >>>> If the clients are using AUTH_UNIX credentials for SETCLIENTID, >>>> client A and client B would have to have the same cl_nodename >>>> to be able to futz with each others leases. Is that the case? >>> >>> That is correct. Auth_unix mount can do it. But so it turns out to be >>> with Kerberos/auth_unix mix. I haven't tried Kerberos/Kerberos but it >>> makes me thing that it will also be a problem (since mix is a >>> problem). >> >> If an AUTH_UNIX client can tamper with a lease established >> by an AUTH_GSS client, that's a pretty serious server bug. >> >> Which server implementation is this? > > This is linux 4.16-rc1. Would it be easy for you confirm if two AUTH_GSS clients are appropriately protected from each other? It would be good to file a bug on bugzilla.linux-nfs.org to document the full extent of the badness. >>>> There used to be a way to get the client to include a uniquifier >>>> in the client ID string. Has that logic been removed? >>> >>> I'm unaware of such logic. I wonder what that uniquer string used to >>> be , a MAC address? The spec talks about how difficult it is to come >>> up with a reboot persistent unique identifier. >> >> Search for nfs4_client_id_uniquifier . >> >> It's meant to be a UUID, but it can be any random string. >> This can be set as a kernel boot parameter so it can be >> stored on a network boot server. > > I see, ok thanks. >> >> >> -- >> Chuck Lever -- Chuck Lever -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
