On 2/2/24 1:20 PM, Olivier Langlois wrote: > On Wed, 2024-01-31 at 13:52 -0700, Jens Axboe wrote: >> On 1/31/24 12:56 PM, Olivier Langlois wrote: >>> On Wed, 2024-01-31 at 12:59 -0500, Olivier Langlois wrote: >>>> On Wed, 2024-01-31 at 10:32 -0700, Jens Axboe wrote: >>>>> >>>>> Thanks for testing! >>>>> >>>>> Any chance that you could run some tests with and without NAPI >>>>> that >>>>> help >>>>> validate that it actually works? That part is what I'm most >>>>> interested >>>>> in, not too worried about the stability of it as I have >>>>> scrutinized >>>>> it >>>>> pretty close already. >>>>> >>>> >>>> There is maybe a test that I can perform. The data that I receive >>>> is >>>> timestamped. I have a small test program that checks the age of >>>> the >>>> updates on their reception... >>>> >>>> I would expect that it should be possible to perceive the busy >>>> polling >>>> effect by comparing the average update age with and without the >>>> feature >>>> enabled... >>>> >>>> A word of warning... The service that my client is connecting to >>>> has >>>> relocated recently. I used to have an RTT of about 8mSec with it >>>> to >>>> about 400-500 mSec today... >>>> >>>> because of the huge RTT, I am unsure that the test is going to be >>>> conclusive at all... >>>> >>>> However, I am also in the process of relocating my client closer >>>> to >>>> the >>>> service. If you can wait a week or so, I should able to do that >>>> test >>>> with a RTT < 1 mSec... >>>> >>>> Beside that, I could redo the same test that Stefan did with the >>>> ping >>>> client/server setup but would that test add any value to the >>>> current >>>> collective knowledge? >>>> >>>> I'll do the update age test when I restart my client and I'll >>>> report >>>> back the result but my expectations aren't very high that it is >>>> going >>>> to be conclusive due to the huge RTT. >>>> >>>> >>> As I expected, the busy polling difference in the update age test >>> is so >>> small compared to the RTT that the result is inconclusive, IMHO... >>> >>> The number of collected updates to build the stats is 500. >>> >>> System clocks are assumed to be synchronized and the RTT is the >>> difference between the local time and the update timestamp. >>> Actually, it may be more accurate to say that the displayed RTT >>> values >>> are in fact TT... >>> >>> latency NO napi busy poll: >>> [2024-01-31 11:28:34] INFO Main/processCollectedData rtt >>> min/avg/max/mdev = 74.509/76.752/115.969/3.110 ms >>> >>> latency napi busy poll: >>> [2024-01-31 11:33:05] INFO Main/processCollectedData rtt >>> min/avg/max/mdev = 75.347/76.740/134.588/1.648 ms >>> >>> I'll redo the test once my RTT is closer to 1mSec. The relative >>> gain >>> should be more impressive... >> >> Also happy to try and run it here, if you can share it? If not I have >> some other stuff I can try as well, with netbench. >> > I have redone my test with a fixed liburing lib that actually enable > io_uring NAPI busy polling correctly and I have slightly more > convincing result: > > latency NO napi busy poll (kernel v7.2.3): > [2024-02-02 11:42:41] INFO Main/processCollectedData rtt > min/avg/max/mdev = 73.089/75.142/107.169/2.954 ms > > latency napi busy poll (kernel v7.2.3): > [2024-02-02 11:48:18] INFO Main/processCollectedData rtt > min/avg/max/mdev = 72.862/73.878/124.536/1.288 ms > > FYI, I said that I could redo the test once I relocate my client to > have a RTT < 1ms... > > I might not be able to do that. I might settle for an AWS VPS instead > of a bare metal setup and when you are running the kernel on a VPS, > AFAIK, the virtual Ethernet driver does not have NAPI... I'm going to try some local 10g testing here, I don't think the above says a whole lot as we're dealing with tens of msecs here. But if significant and stable, does look like an improvement. If NAPI works how it should, then sub msec ping/pong replies should show a significant improvement. I'll report back once I get to it... -- Jens Axboe
