On Jun 3, 2013, at 9:13 PM, Chuck Lever <chuck.lever@xxxxxxxxxx> wrote: > > On Jun 3, 2013, at 7:30 PM, NeilBrown <neilb@xxxxxxx> wrote: > >> On Mon, 3 Jun 2013 00:32:54 -0400 Chuck Lever <chuck.lever@xxxxxxxxxx> wrote: >> >>> >>> On Jun 2, 2013, at 11:01 PM, NeilBrown <neilb@xxxxxxx> wrote: >>> >>>> On Sun, 2 Jun 2013 22:45:16 -0400 Chuck Lever <chuck.lever@xxxxxxxxxx> wrote: >>>> >>>>> >>>>> On Jun 2, 2013, at 10:23 PM, NeilBrown <neilb@xxxxxxx> wrote: >>>>> >>>>>> On Sun, 2 Jun 2013 22:01:50 -0400 Chuck Lever <chuck.lever@xxxxxxxxxx> wrote: >>>>>> >>>>>>> >>>>>>> On Jun 2, 2013, at 9:00 PM, Neil Brown <neilb@xxxxxxx> wrote: >>>>>>> >>>>>>>> As you probably know, since 3.7 (I think) Linux NFS has explicitly >>>>>>>> asked for machine credentials for certain requests rather than asking >>>>>>>> for root credentials as is previously did. >>>>>>>> This causes a regression for people who don't have any machine >>>>>>>> credentials configured and use "gssd -n". >>>>>>>> >>>>>>>> I gather this was discussed on the mailing list earlier this year but >>>>>>>> not resolved. >>>>>>> >>>>>>> It's resolved in 3.10-rc. >>>>>>> >>>>>>> The kernel will attempt to use krb5i for lease management operations. If that fails because there is no keytab available, it falls back to using AUTH_SYS. >>>>>> >>>>>> And if the server refuses to accept AUTH_SYS? >>>>>> >>>>>> I guess this is commit 79d852bf5e7691dc7 ?? >>>>> >>>>> That's one of the subsequent bug fixes. The initial change is commit 4edaa308. >>>>> >>>>>> It seems to say that the server should always accept AUTH_SYS ... is that right? >>>>> >>>>> If we ever find a server implementation that does not support either Kerberos or AUTH_SYS, we can add another step to the negotiation. >>>>> >>>>> So far, despite RFC 3530 not requiring AUTH_SYS support on NFSv4 servers, I haven't found an implementation that does not support AUTH_SYS. We have found one (FreeBSD) that does not support AUTH_NONE. We do know that some servers allow administrators to control what security flavors are allowed for lease management. >>>>> >>>>>> That commit isn't tagged for -stable. >>>>>> So do we still need to make it work for 3.7,3.8,3.9 users? >>>>> >>>>> There are several commits that would need to be back-ported, starting with commit 4edaa308. I am not certain they would apply cleanly to 3.[789], but a backport should not be difficult. >>>>> >>>>> This change also requires that now gssd must be running on the client. Otherwise without gssd a sec=sys mount hangs for a bit waiting for the upcall to time out (since the client will attempt to use krb5i for lease management operations). Trond and Bruce have been discussing a change to address that. >>>> >>>> Thanks for the explanation. That all looks rather painful to back-port >>>> though, especially as some of it isn't even written yet :-) >>>> I think I'll stick with my "-N" option for openSUSE for now. >>>> >>>> Do you think that supporting -N (or similar) so that the admin can ask for >>>> root credentials to be used for SETCLIENTID requests is reasonable? i.e. what >>>> do you think of my patch going in to nfs-utils anyway? >>> >>> So, let me confirm my understanding of your proposed changes: a user logs in as root, then kinit's with her own user principal. That establishes a Kerberos credential for root to use, and "-N" makes gssd return this credential when the kernel requests "service=*". >> >> Correct. >> >>> >>> Lease management is shared among all NFS users on a client. In particular, machine credentials give us two important features that cannot be matched by user credentials: >>> >>> o Machine credentials never expire even when users log out. On a >>> multi-user client, you don't want credential expiry or a user >>> logout to cause lease management to stop working for other users. >> >> ... while on a single-user client, this isn't really an issue. >> >> And I can easily imagine a server admin not wanting to hand out credentials >> that never expire, but still requiring strong credentials for lease >> management. > > Why, exactly, would strong credentials be required for lease management, but an admin would choose not to hand out keytabs? > >> >>> >>> o Machine credentials are always the same no matter who is logged >>> in. The client has to use the same credential every time for >>> lease management, or else servers return CLID_INUSE and prevent a >>> fresh lease from being established (at least until the existing >>> lease for that clientid expires on the server). >> >> Again, not really an issue for a single-user client. And single-user >> machines are certainly a common use-case these days. >> >>> >>> Using root's credentials for lease management makes it likely one of these two bullets will end up in your foot. Specifically for the use case where one user is always using the same client (say, a student's laptop), "-N" might work fine. For other use cases, like a shared build machine that doesn't have a keytab, "-N" would not work reliably. >> >> Completely agree. So -N should certainly not be the default. >> >>> >>> With 3.10, if we can't use a Kerberos host principal for lease management, we'll use a less secure equivalent. The point of backing off the security flavor for lease management is that AUTH_SYS with UID 0, or even AUTH_NONE, are essentially machine credentials. They don't expire, and the client will present a consistent credential for lease management, no matter what user logs in or leaves the machine. >>> >>> So, yes, machine credentials are a pain in the ass, and have been for years. I'm sorry this took 3 kernel releases to figure out, but I'm hoping that the path we're on with these kernel changes is towards more transparent support for a wider array of use cases. NFSv4.1 also offers some relief in this area (see SP4_MACH_CRED, which I think we do not support quite yet). >> >> All sounds very good, but doesn't seem to address the issue. > >> For the openSUSE user who raised this, he simply does not have a machine >> credential. I don't know why, maybe the server admin doesn't want to hand >> them out. > > Handing out keytabs for every NFS client that wants to use Kerberized NFS is onerous for some administrators. > >> But this hasn't been a problem because "gssd -n" meant that a >> machine credential would never be used. > > No, it meant that root's user credential was used instead of a service credential for lease management. As I explained above, that solution has some compromises for some important use cases. Correction: rpc.gssd(8) on Fedora 18 says: > With the -n option, "machine credentials" will not be used for accesses by UID 0. In other words, normally root doesn't have a Kerberos ticket. The kernel uses the machine's credentials for UID 0. "-n" means only that root has to kinit. >> But now, with 3.[789], "gssd -n" still wants the machine credential that >> doesn't exist. > > The change that introduced the requirement for a machine credential is commit 68c97153 "SUNRPC: Clean up the RPCSEC_GSS service ticket requests", Tue Jan 3 13:22:46 2012, which was merged long before 3.7. > > The Uniform Client String changes in 3.7 merely made the machine credential requirement more obtrusive. The new broken "-n" behavior is an unintended side effect. I was wondering what exactly about 3.7's UCS changes exposed the requirement for a machine credential, so I built a 3.6 kernel and tried it. The key difference between 3.6 and 3.7 is that 3.6 performs the SETCLIENTID much later, after a user has logged in and kinit'd, and is actually doing an OPEN. 3.6 requests a machine credential, but if that fails, it asks for the OPEN user's credential for SETCLIENTID. The retried upcall request is for "service=<null> UID=nnnn". If root kinit's for the mount, but the user does not kinit before accessing the mount, the SETCLIENTID still fails! Root's ticket and "gssd -n" has nothing to do with SETCLIENTID's security flavor, as far as I can tell. With 3.7, the SETCLIENTID is done right at mount time. There is no user with credentials trying to do an OPEN, so the only credential the kernel tries is the machine's credential. Perhaps the kernel could retry the upcall with "server=<null> UID=0". >> It is possible that with 3.10, he might get away with doing lease management >> with AUTH_SYS, but that seems unlikely. Given that an integrity protecting >> security flavour is now recommended for SETCLIENTID, his server admin >> could easily make a case that AUTH_SYS won't be accepted. In that case he >> would still have a problem. > > His problem then would be with his server admin, not with us. > > If a server admin wants to protect his server by denying the use of AUTH_SYS for lease management, he has every right to do that. But he must also therefore understand the position that puts his users in. > >> Given that "gssd -n" has meant "no machine credential needed" in the past, > > It never meant that. It meant "use root's credential as the machine credential". I think we should be clear about exactly what "-n" is. Right, I should heed my own words. > I also believe "-n" (or "-N") is a workaround. Logging in as root and authenticating is a pain for users, who have already logged in once. This is still true, however. >> I think it is important to preserve the possibility of running secure NFS with >> no machine credential. I don't see how your changes (valuable though they are) >> achieve that. > > Please explain how allowing lease management to occur using AUTH_SYS is somehow a major new exposure. A mixture of sec=sys and sec=krb5 mounts today means there's a good chance that a sec=sys mount might occur first, and thus lease management uses AUTH_SYS anyway. > > Can you share your attack model? Why do you believe that using AUTH_SYS for lease management on sec=krb5 mounts will be unacceptable to users? I suspect that most will not notice or care. I'm still interested in knowing if we have a palpable hole in the current solution in 3.10. >> So I think we either need to change "gssd -n" to mean "never use machine >> credentials" or add something like "gssd -N", so the two different possible >> uses of machine credentials can be separately controlled. > > I'm not convinced that "-N" is the right way to fix this. Another knob to turn means more testing for us, more documentation to write and maintain, and so on. It doesn't get us any farther forward. The difference between "-n" (which no longer works) and "-N" is subtle and confusing. Explaining when to use "-N" versus "-n" is not a job I would like to do, and I'm not interested in taking the support calls either. :-p Let me restate this without all the cleverness. Trond's recommendation was to tell gssd exactly where to find the machine credential if it's not in a keytab: basically an option that says "look in this file for that user credential". That was our first try, but it was clumsy. You want "gssd -N" to mean specifically "use root's ticket as the machine credential." Then "-n -N" would mean "force root to kinit, and use root's ticket as the machine credential." (Perhaps "-N" doesn't make sense unless "-n" is also specified...?) But that's still not the same as 3.6's behavior. And, it would still be a regression, as I understand it, because a user space change would be needed after a kernel upgrade. For that reason, for 3.[789] we probably want to find a kernel-only solution. Command line options on gssd are really poor from a usability standpoint. IMO in the long run we want "mount sec=krb5 without a client keytab" just to work. No monkeying with gssd. -- Chuck Lever chuck[dot]lever[at]oracle[dot]com -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
