Re: RFC: using NIC DMA ring as tracing buffer?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]



On Wed, Apr 17, 2024 at 6:51 PM Alexander Aring <aahringo@xxxxxxxxxx> wrote:
> Hi Mathieu,
> On Tue, Apr 16, 2024 at 10:49 AM Mathieu Desnoyers
> <mathieu.desnoyers@xxxxxxxxxxxx> wrote:
> >
> > On 2024-04-16 10:41, Alexander Aring wrote:
> > > Hi all,
> > >
> > > I have an idea about using the DMA rings of a networking card as a
> > > tracing buffer.
> >
> > The first thing I'd be worried about is overhead caused by
> > false-sharing if you trace a large SMP/many-core system all
> > into the same global NIC buffer. The cache lines will bounce
> > left and right which is bad performance-wise.
> >
> modern NICs have several tx and rx dma rings. There is an ethtool
> command to query the number:
> sudo ethtool -g $IFACE
> so it depends on the hardware if you can pin one or more rings on a
> per-cpu buffer basis.
> I hope that somehow helps to make this idea less worrisome.

What I can think about that makes it worrisome is NUMA. In case of an
E-PCI connected NIC, the NICs buffer associated with a core needs to
be in the same NUMA node. You need to set up your system right, even
with 2 nics or more, to make it work "better".

- Alex

[Index of Archives]     [Linux USB Development]     [Linux USB Development]     [Linux Audio Users]     [Yosemite Hiking]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux