I've had some feedback from a hardware supplier. The Linuxcnc RT ethernet driver has a pin that reports ethernet packet error count. This increments when the ethernet response exceeds our real time servo thread period (typically 1 ms) We should be able to monitor this pin and stop the trace if it is > 0 without too much effort. So let me know the next steps and I'll try and implement a trace test. - Rod Rod Webster VMN® www.vmn.com.au Ph: 1300 896 832 Mob: +61 435 765 611 On Sat, 29 Apr 2023 at 07:37, Rod Webster <rod@xxxxxxxxxx> wrote: > > > Okay. Then it could be related to softirq rework which started in > > v5.0.19-rt11 and updated in v5.9-rc6-rt9. > v5.9 could also be significant as the R8125 driver support was added > in that version > I upgraded from Debian Buster to Bullseye (v5.10) on one PC for this reason. > > > Are you able to recompile a kernel? > Yes, > I have attached a text file containing my 6.3 build steps. > Note we had to git pull the kernel source as it was not available via > http at that time > I have not tried with your 6.3-rc11 patch yet. > > > However as per you Debian bug report v4.19-RT is fine, v5.10-RT isn't > > and v6.1-RT is better or is it fine? > I built v6.1-rc7-rt5 before Debian advanced to v6.1 > It was reported to be an improvement and it solved the Debian latency > issue for some users. > v6.3 was reported as being as good as the 4.x kernes by our hardware vendor > > > Where is the Intel ethernet coming from? You were saying it is the > > realtek nic. > Sorry, you mentioned coalesce but that's an Intel feature and does not > exist with Realtek. > > > Is the vendor driver better than in the tree? This is not obvious from > > what you are writing. > I have been unable to compile the R8168 driver from source on the > Realtek website. > They are behind with supported kernels and the build process is > invariably broken for me. > On the Hardkernel Odroid forum, it has been reported that compilng the > R8125 from Realtek source offers 15-35% improvement over the in-tree > driver. > I have one of their PC's (H2+). It's currently running v5.10 and the > in-tree R8168 driver (which must be the Debian default). > It is affected and I plan to upgrade to v6.3 with the R8125-dkms driver > > > Two things you could do: > > - enabling tracing to see what is causing the delays/ latency spike. You > > will need a trigger to notice the latency and then stop the trace at > > this point. If so, I could provide additional steps unless you can do > > it yourself. > Please advise additional steps to do this. > We might be able to modify our hardware device driver to stop the > trace as it reports latency incursions if they occur. > > > - You could try to isolate the realtek driver on one CPU and moving > > everything else to another. > This sounds interesting. Please advise how. > On a 4 core machine we typically add isolcpus=2,3 for best RT performance. > > Thanks for your support. > > -end > -Rod Webster
