On Wed, Apr 26, 2023 at 11:42:01PM +0000, Kernel.org Bugbot wrote: > phemmer+kernel writes via Kernel.org Bugzilla: > > (In reply to Bugbot from comment #1) > > Matthew Wilcox <willy@xxxxxxxxxxxxx> writes: > > > > On Wed, Apr 26, 2023 at 05:58:06PM +0000, Kernel.org Bugbot wrote: > > > When running a threaded program, and opening a file descriptor that > > > is a power of 2 (starting at 64), the call takes a very long time to > > > complete. Normally such a call takes less than 2us. However with this > > > issue, I've seen the call take up to around 50ms. Additionally this only > > > happens the first time, and not subsequent times that file descriptor is > > > used. I'm guessing there might be some expansion of some internal data > > > structures going on. But I cannot see why this process would take so long. > > > > Because we allocate a new block of memory and then memcpy() the old > > block of memory into it. This isn't surprising behaviour to me. > > I don't think there's much we can do to change it (Allocating a > > segmented array of file descriptors has previously been vetoed by > > people who have programs with a million file descriptors). Is it > > causing you problems? > > Yes. I'm using using sockets for IPC. Specifically haproxy with its SPOE protocol. Low latency is important. Normally a call (including optional connect if a new connection is needed) will easily complete in under 100us. So I want to set a timeout of 1ms to avoid blocking traffic. However because this issue effectively randomly pops up, that 1ms timeout is too low, and the issue can actually impact multiple in-flight requests because haproxy tries to share that one IPC connection for them all. But if I raise the timeout (and I'd have to raise it to something like 100ms, as I've seen delays up to 47ms in just light testing), then I run the risk of significantly impacting traffic if there is a legitimate slowdown. While a low timeout and the occasional failure is probably the better of the two options, I'd prefer not to fail at all. I wonder if you could use io_uring for this. The problem sounds a lot like it could be solved by using the fixed file descriptor feature. The async and linking operation nature of it might be rather valuable for this as well...
