On Monday, April 17, 2023 11:32 PM, Kurt Kanzenbach <kurt.kanzenbach@xxxxxxxxxxxxx> wrote: >On Mon Apr 17 2023, Jesper Dangaard Brouer wrote: >> To correlate the hardware RX timestamp with something, add tracking of >> two software timestamps both clock source CLOCK_TAI (see description >> in man clock_gettime(2)). >> >> XDP metadata is extended with xdp_timestamp for capturing when XDP >> received the packet. Populated with BPF helper bpf_ktime_get_tai_ns(). >> I could not find a BPF helper for getting CLOCK_REALTIME, which would >> have been preferred. In userspace when AF_XDP sees the packet another >> software timestamp is recorded via clock_gettime() also clock source >> CLOCK_TAI. >> >> Example output shortly after loading igc driver: >> >> poll: 1 (0) skip=1 fail=0 redir=2 >> xsk_ring_cons__peek: 1 >> 0x12557a8: rx_desc[1]->addr=100000000009000 addr=9100 comp_addr=9000 >> rx_hash: 0x82A96531 with RSS type:0x1 >> rx_timestamp: 1681740540304898909 (sec:1681740540.3049) >> XDP RX-time: 1681740577304958316 (sec:1681740577.3050) delta >sec:37.0001 (37000059.407 usec) >> AF_XDP time: 1681740577305051315 (sec:1681740577.3051) delta >sec:0.0001 (92.999 usec) >> 0x12557a8: complete idx=9 addr=9000 >> >> The first observation is that the 37 sec difference between RX HW vs >> XDP timestamps, which indicate hardware is likely clock source >> CLOCK_REALTIME, because (as of this writing) CLOCK_TAI is initialised >> with a 37 sec offset. > >Maybe I'm missing something here, but in order to compare the hardware with >software timestamps (e.g., by using bpf_ktime_get_tai_ns()) the time sources >have to be synchronized by using something like phc2sys. That should make them >comparable within reasonable range (nanoseconds). > >Thanks, >Kurt Tested-by: Song Yoong Siang <yoong.siang.song@xxxxxxxxx> I tested this patchset by using I226-LM (rev 04) NIC on Tiger Lake Platform. I use testptp selftest tool to make sure PHC is almost same as system clock. Below are the detail of test steps and result. 1. Run xdp_hw_metadata tool. @DUT: sudo ./xdp_hw_metadata eth0 2. Enable Rx HWTS for all incoming packets. Note: This step is not needed if https://lore.kernel.org/all/20230414154902.2950535-1-yoong.siang.song@xxxxxxxxx/ bug fix patch is applied to the igc driver. @DUT: sudo hwstamp_ctl -i eth0 -r 1 3. Set the ptp clock time from the system time using testptp tool. @DUT: sudo ./testptp -d /dev/ptp0 -s 4. Send UDP packet with 9091 port from link partner immediately after step 3. @LinkPartner: echo -n xdp | nc -u -q1 <Destination IPv4 addr> 9091 Result: poll: 1 (0) skip=1 fail=0 redir=1 xsk_ring_cons__peek: 1 0x5626248d16d0: rx_desc[0]->addr=100000000008000 addr=8100 comp_addr=8000 rx_hash: 0x35E1B60E with RSS type:0x1 rx_timestamp: 1677762195217129600 (sec:1677762195.2171) XDP RX-time: 1677762195217202099 (sec:1677762195.2172) delta sec:0.0001 (72.499 usec) AF_XDP time: 1677762195217231775 (sec:1677762195.2172) delta sec:0.0000 (29.676 usec) 0x5626248d16d0: complete idx=8 addr=8000
