On Thu, 2012-09-27 at 09:59 -0700, Linus Torvalds wrote:
> On Thu, Sep 27, 2012 at 9:16 AM, Myklebust, Trond
> <Trond.Myklebust@xxxxxxxxxx> wrote:
> >
> > I cannot see how that BUG_ON can be triggered in the current code, given
> > that the only place where idmap->idmap_key_cons is set to a non-NULL
> > value is covered by a mutex, and that it is always cleared before we
> > release said mutex.
>
> Quite frankly, the "I cannot see" thing is *never* an excuse for a BUG_ON().
>
> We don't do kernel-killing asserts in Linux. Never.
>
> The only excuse for a BUG_ON() is "I cannot possibly continue, I don't
> even have an error path I can take".
>
> If it's a fundamentally impossible situation, the BUG_ON() should
> never have been there in the first place!
>
> And if it's a "I don't see how it could happen", then it should have
> been something like
>
> if (WARN_ON_ONCE(condition))
> goto cleanup;
>
> rather than a BUG_ON().
>
> We have too many f*cking BUG_ON's in the kernel, and the fact that one
> triggers and it has taken a month and a half without it even being
> resolved is a problem.
>
> Get rid of the thing, already, dammit. If you cannot figure out how it
> can happen, then the *last* thing you want to do is then kill the
> machine so that it's impossible to debug it sanely.
>
> Besides, as far as I can tell, idmap_key_cons locking is suspect
> anyway. Stuff like this:
>
> cons = ACCESS_ONCE(idmap->idmap_key_cons);
> idmap->idmap_key_cons = NULL;
>
> is an almost certain example of "the code is racy, and we did it
> wrong". The above is basically *never* correct.
>
> If the access is properly locked, then the ACCESS_ONCE() is a bug.
>
> And if the access *isn't* properly locked, then setting things to NULL
> afterwards is in no way safe.
>
> IOW, either way, it's broken. And there's at least two of those
> clearly buggy code-sequences involving that field.
>
> So get rid of the BUG_ON() (possibly replacing it with the
> WARN_ON_ONCE), and please look at those ACCESS_ONCE() sequences and
> fix them. Either they happen under a lock, or they don't. None of this
> crazy racy crap, please.
The ACCESS_ONCE is a bug. It isn't needed, since the whole upcall is
covered by the idmap_mutex. I'll remove it and the BUG_ON() come the
merge window (or I can send a patch sooner if you care).
The downcall is ordered w.r.t. idmap_pipe_destroy_msg() (which clears
idmap->idmap_key_cons only on upcall failure). The downcall is also
protected against collisions with idmap_release_pipe() by means of the
inode->i_mutex. Finally, the call to request_key() (during which the
idmap_mutex is held) will always do an uninterruptible wait for one of
those 3 functions to complete.
IOW: all the functions that manipulate idmap->idmap_key_cons are
theoretically ordered w.r.t. each other. That's what I mean when I say
that I really don't understand why this is happening.
So the first thing to do is to try a _vanilla_ 3.6-rc7, and see if the
problem is reproducible without Joerg's extra patches.
--
Trond Myklebust
Linux NFS client maintainer
NetApp
Trond.Myklebust@xxxxxxxxxx
www.netapp.com
��.n��������+%������w��{.n�����{��w��� jg��� �����ݢj����G���� ���j:+v���w�m������w�������h�����٥
