On 07-04-23, 13:31, Fenghua Yu wrote: > Applications can send 64B descriptors to the DSA device via CPU > instructions MOVDIR64B or ENQCMD. The application can choose to have > the device write back a completion record (CR) in system memory to > indicate the status of the descriptor submitted on completion. > > With the DSA hardware, the device is able to do on demand paging through > the hardware by faulting in the user pages that do not have physical memory > page backing with assistance from IOMMU. In the spec this was designated as > the block on fault feature. While this hardware feature made operation > simpler, it also stalls the device engines while the memory pages are being > faulted in through Page Request Service (PRS). For applications sharing the > same workqueue (wq) or wqs in the same group, operations are stalled if > there are no free engines. To avoid slowing the performance of all other > running applications sharing the same device engine(s), PRS can to be > disabled and software can deal with partial completion. > > The block on fault feature on DSA 1.0 can be disabled for the wq. However, > PRS is not completely disabled for the whole path. It is not disabled for > CRs or batch list for a batch operation. > > The other issue is the DSA 1.0 error reporting mechanism, SWERROR register. > The SWERROR register can only report a single error at a time until the > driver reads and acknowledges the error. The follow on errors cannot be > reported until the current error is "cleared" by the software by writing > a bit to the SWERR register. If a large number of faults arrive and the > software cannot clear them fast enough, overflowed errors will be dropped > by the device. > > A CR is the optional 32 bytes (DSA) or 64 bytes (IAA) status that is > written back for a submitted descriptor. If the address for the CR faults, > the error is reported to the SWERROR register instead. > > With DSA 2.0 hardware [1], the event log feature is added. All errors are > reported as an entry in a circular buffer reside in the system memory. > The system admin is responsible to configure the size of the circular > buffer large enough per device to handle the potential errors that may be > reported. If the buffer is full and another error needs to be reported, > the device engine will block until there's a free slot in the buffer. > An event log entry for a faulted CR will contain the error information, > the CR address that faulted, and the expected CR content the device had > originally intended to write. > > DSA 2.0 also introduces per wq PRS disable knob. This will disable all PRS > operations for the specific wq. The device will still have Address > Translation Service (ATS) on. When ATS fails on a memory address for a CR, > an eventlog entry will be written by the hardware into the event log > ring buffer. The driver software is expected to parse the event log entry, > fault in the address of the CR, and the write the content of the CR to > the memory address. > > This patch series will implement the DSA 2 event log support. The support > for the handling of the faulted user CR is added. The driver is also > adding the same support for batch operation descriptors. With a batch > operation the handling of the event log entry is a bit more complex. > The faulting CR could be for the batch descriptor or any of the operation > descriptors within the batch. The hardware generates a batch identifier > that is used by the driver software to correlate the event log entries for > the relevant descriptors of that batch. > > The faulting of source and destination addresses for the operation is not > handled by the driver. That is left to be handled by the user application > by faulting in the memory and re-submit the remaining operation. Applied, thanks -- ~Vinod
