On Mon, Oct 26, 2020 at 11:35:52AM -0700, Jakub Kicinski wrote: > On Tue, 27 Oct 2020 01:38:04 +0800 Xu Yilun wrote: > > > > The line/host side Ether Group is not the terminal of the network data stream. > > > > Eth1 will not paticipate in the network data exchange to host. > > > > > > > > The main purposes for eth1 are: > > > > 1. For users to monitor the network statistics on Line Side, and by comparing the > > > > statistics between eth0 & eth1, users could get some knowledge of how the User > > > > logic is taking function. > > > > > > > > 2. Get the link state of the front panel. The XL710 is now connected to > > > > Host Side of the FPGA and the its link state would be always on. So to > > > > check the link state of the front panel, we need to query eth1. > > > > > > This is very non-intuitive. We try to avoid this in the kernel and the > > > API to userspace. Ethernet switches are always modelled as > > > accelerators for what the Linux network stack can already do. You > > > configure an Ethernet switch port in just the same way configure any > > > other netdev. You add an IP address to the switch port, you get the > > > Ethernet statistics from the switch port, routing protocols use the > > > switch port. > > > > > > You design needs to be the same. All configuration needs to happen via > > > eth1. > > > > > > Please look at the DSA architecture. What you have here is very > > > similar to a two port DSA switch. In DSA terminology, we would call > > > eth0 the master interface. It needs to be up, but otherwise the user > > > does not configure it. eth1 is the slave interface. It is the user > > > facing interface of the switch. All configuration happens on this > > > interface. Linux can also send/receive packets on this netdev. The > > > slave TX function forwards the frame to the master interface netdev, > > > via a DSA tagger. Frames which eth0 receive are passed through the > > > tagger and then passed to the slave interface. > > > > > > All the infrastructure you need is already in place. Please use > > > it. I'm not saying you need to write a DSA driver, but you should make > > > use of the same ideas and low level hooks in the network stack which > > > DSA uses. > > > > I did some investigation about the DSA, and actually I wrote a > > experimental DSA driver. It works and almost meets my need, I can make > > configuration, run pktgen on slave inf. > > > > A main concern for dsa is the wiring from slave inf to master inf depends > > on the user logic. If FPGA users want to make their own user logic, they > > may need a new driver. But our original design for the FPGA is, kernel > > drivers support the fundamental parts - FPGA FIU (where Ether Group is in) > > & other peripherals on board, and userspace direct I/O access for User > > logic. Then FPGA user don't have to write & compile a driver for their > > user logic change. > > It seems not that case for netdev. The user logic is a part of the whole > > functionality of the netdev, we cannot split part of the hardware > > component to userspace and the rest in kernel. I really need to > > reconsider this. > > This is obviously on purpose. Your design as it stands will not fly > upstream, sorry. > > >From netdev perspective the user should not care how many hardware > blocks are in the pipeline, and on which piece of silicon. You have > a 2 port (modulo port splitting) card, there should be 2 netdevs, and > the link config and forwarding should be configured through those. > > Please let folks at Intel know that we don't like the "SDK in user > space with reuse [/abuse] of parts of netdev infra" architecture. > This is a second of those we see in a short time. Kernel is not a > library for your SDK to use. I get your point. I'll share the information internally and reconsider the design. Thanks, Yilun
