On IOFS, there are many methods to access the AFU for virtualization and non-virtualization usage. 1. Legacy Model. This is used for N3000 and N5000 card. In this model the entire AFU region is a unit of PR, and there is a Port device connected to this AFU. In this model, the follow of exposing to VM is the same we have in N3000 and N5000 card. The major follow is: * release the port device. * configure the SRIOV * assign the VF to VM * put the port device back to PF after finished access in VM 2. Micro-Personas in AFU. IOFS intruding new model for PR and AFU access. Micro-Personas allow the RTL developer to designate their own AFU-defined PR regions. In this model, each PR slot has a dedicate Port device, the follow of exposing to VM is the same we have in N3000 and N5000 card. 3. Multiple VFs per PR slot. In this method, we can instance multiple VFs over SRIOV for one PR slot, and access the AFU resource by different VFs in virtualization usage. In this case, the Port device would not connected to AFU/PR slot, so we don't need to release the Port device. before we assign the VF to VM. Which model will deploy in IOFS product depends on the RTL developer. > > > + > > + +-----------------+ +--------------+ +-------------+ +------------+ > > + | FPGA Managerment| | VirtIO | | User App | | Virtual | > > s/Managerment/Management/ Thanks, I will fix it on next version. > > > + | App | | App | | | | Machine | > > + +--------+--------+ +------+-------+ +------+------+ +-----+------+ > > + | | | | > > + | | | | > > + +--------+--------+ +------+-------+ +------+------+ | > > + | DFL Driver | |VirtIO driver | | VFIO | | > > + +--------+--------+ +------+-------+ +------+------+ | > > + | | | | > > + | | | | > > + +--------+--------+ +------+-------+ +------+------+ +----+------+ > > + | PF0 | | PF1 | | PF0_VF0 | | PF0_VF1 | > > + +-----------------+ +--------------+ +-------------+ +-----------+ > > + > > +On IOFS, it introducing some enhancements compared with original DFL > design. > > +1. It introducing Port Gasket in PF0 which is responsible for FPGA > > +management, like FME and Port management. The Port Gasket contains > > +all the PR specific > > modules > > So in IOFS, in PF0, we always have FME and PORT DFH, is my understanding > correct? Yes. > Then why we need patch #3? This is for " Multiple VFs per PR slot" model, in this model, the Port device would not connected to AFU (the BarID of Port device should be set to invalid), so we just can access PR slot/AFU resource via VFs. > > Another question is in IOFS, do we need to support multiple PR regions/Ports? > If that is the case, how should we know which VFs belongs to PORT1 or PORT2? > > > +and logic, e.g., PR slot reset/freeze control, user clock, remote STP etc. > > +Architecturally, a Port Gasket can have multiple PR slots where user > > +workload > > can > > +be programmed into. > > +2. To expend the scalable of FPGA, it can support multiple FPs in > > +static region > > s/FPs/PFs/ I will fix on next version. > > > +which contain some static functions like VirtIO, diagnostic test, and > > +access over VFIO or assigned to VMs easily. Those PFs will not have a > > +Port Unit which > > without > > +PR region (AFU) connected to those PFs, and the end-user cannot > > +partial > > reconfigurate > > s/reconfigurate/reconfigure/ I will fix on next version. > > > +those PFs. > > +3. In our previous DFL design, it can only create one VF based in an > > +AFU. To > > raise > > +the efficiency usage of AFU, it can create more than one VFs in an > > +AFU via PCIe SRIOV, so those VFs share the PR region and resource. > > + > > +There is one reference architecture design for IOFS as illustrated below: > > + > > + +----------------------+ > > + | PF/VF mux/demux | > > + +--+--+-----+------+-+-+ > > + | | | | | > > + +------------------------+ | | | | > > + PF0 | +---------+ +-+ | | > > + +---+---+ | +---+----+ | | > > + | DFH | | | DFH | | | > > + +-------+ +-----+----+ +--------+ | | > > + | FME | | VirtIO | | Test | | | > > + +-------+ +----------+ +--------+ | | > > + | Port | PF1 PF2 | | > > + +---+---+ | | > > + | +----------+ | > > + | | ++ > > + | | | > > + | | PF0_VF0 | PF0_VF1 > > + | +-----------------+-----------+------------+ > > + | | +-----+-----------+--------+ | > > + | | | | | | | > > + | | +------+ | +--+ -+ +--+---+ | | > > + | | | CSR | | | DFH | | DFH | | | > > + +-----------+ +------+ | +-----+ +------+ | | > > + | | | DEV | | DEV | | | > > + | | +-----+ +------+ | | > > + | | PR Slot | | > > + | +--------------------------+ | > > + | Port Gasket | > > + +------------------------------------------+ > > + > > +Here are the major changes about DFL structures on IOFS > > +implementation > > design: > > +1. The Port Gasket connects to FIU Port in DFL, but the Next_AFU > > +pointer in FIU feature header can point to NULL so that it is no AFU > > +connects to a FIU Port. > > +2. The VF which include in PR region can start with AFU feature > > +header without a FIU Port feature header. > > What about PF2 in static region? Which type of DFH will be used? An IP designer may choose to add more than one PF for interfacing with IP on the FPGA. If at least one PF implements a DFL with management features such as an FME or PR, then the device can be managed using the IOFS software stack. For example, a design may include FME and PR on PF0 and the actual workload interfaces on PF1. If a workload implements virtio-net backend and interface, the IOFS software stack will only bind to and communicate with IOFS features/interfaces found in the DFL on PF0. The second PF, PF1, will bind with virtio-net driver presenting itself as a network interface to the OS. So the IOFS providing the diversity for IP designer. > > Thanks > Hao > > > > > Performance Counters > > ==================== > > -- > > 2.17.1
