On 2021-06-08 16:23:45 [+0200], Peter Zijlstra wrote: > There's more futex users than glibc, and some of them are really hurting > because of the NUMA issue. Oracle used to (I've no idea what they do or > do not do these days) use sysvsem because the futex hash table was a > massive bottleneck for them. > > And as Nick said, other vendors are having the same problems. I just wanted to do a brief summary of last events. The implementation tglx did with the cookie resulting in a quick lookup did not have any downsides except that the user-API had to change glibc couldn't. So if we are back to square one why not start with that. > And if you don't extend the futex to store the nid you put the waiter in > (see all the problems above) you will have to do wakeups on all nodes, > which is both slower than it is today, and scales possibly even worse. > > The whole numa-aware qspinlock saga is in part because of futex. sure. > That said; if we're going to do the whole futex-vector thing, we really > do need a new interface, because the futex multiplex monster is about to > crumble (see the fun wrt timeouts for example). This might have been a series of unfortunate events leading to this. The sad part is that glibc has a comment that the kernel does not support this and nobody bother to change it (until recently). > And if we're going to do a new interface, we ought to make one that can > solve all these problems. Now, ideally glibc will bring forth some > opinions, but if they don't want to play, we'll go back to the good old > days of non-standard locking libraries.. we're halfway there already due > to glibc not wanting to break with POSIX were we know POSIX was just > dead wrong broken. > > See: https://github.com/dvhart/librtpi I'm aware of that, I hacked on it, too :) This was the unfortunate result of a ~8y old bug which was not fixed instead and part of the code was rewritten and a bit-spinlock was added in user-land. You may remember the discussion regarding spins in userland… That said, REQUEUE_PI is no longer used by glibc. Sebastian
