On Tue, Feb 23, 2021 at 10:30:14AM -0800, Dan Williams wrote: > On Tue, Feb 23, 2021 at 10:11 AM Jason Gunthorpe <jgg@xxxxxxxxxx> wrote: > > > > On Tue, Feb 23, 2021 at 10:05:58AM -0800, Dan Williams wrote: > > > On Tue, Feb 23, 2021 at 9:09 AM Jason Gunthorpe <jgg@xxxxxxxxxx> wrote: > > > > > > > > On Tue, Feb 23, 2021 at 08:27:46AM -0700, Dave Jiang wrote: > > > > > > > > > > On 2/23/2021 5:59 AM, Jason Gunthorpe wrote: > > > > > > On Thu, Feb 18, 2021 at 02:31:54PM -0700, Dave Jiang wrote: > > > > > > > Remove devm_* allocation of memory of 'struct device' objects. > > > > > > > The devm_* lifetime is incompatible with device->release() lifetime. > > > > > > > Address issues flagged by CONFIG_DEBUG_KOBJECT_RELEASE. Add release > > > > > > > functions for each component in order to free the allocated memory at > > > > > > > the appropriate time. Each component such as wq, engine, and group now > > > > > > > needs to be allocated individually in order to setup the lifetime properly. > > > > > > I really don't understand why idxd has so many struct device objects. > > > > > > > > > > > > Typically I expect a simple driver to have exactly one, usually > > > > > > provided by its subsystem. > > > > > > > > > > > > What is the purpose? > > > > > > > > > > When we initially designed this, Dan suggested to tie the device and > > > > > workqueue enabling to the Linux device model so that the enabling/disabling > > > > > can be done via bind/unbind of the sub drivers. So that's how we ended up > > > > > with all the 'struct device' under idxd and one for each configurable > > > > > component of the hardware device. > > > > > > > > IDXD created its own little bus just for that?? :\ > > > > > > Yes, for the dynamic configurability of the queues and engines it was > > > either a pile of ioctls, configfs, or a dynamic sysfs organization. I > > > did dynamic sysfs for libnvdimm and suggested idxd could use the same > > > model. > > > > > > > It is really weird that something called a workqueue would show up in > > > > the driver model at all. > > > > > > It's a partition of the hardware functionality. > > > > But to what end? What else are you going to do with a slice of the > > IDXD device other than assign it to the IDXD driver? > > idxd, unlike other dmaengines, has a dynamic relationship between > backend hardware engines and frontend submission queues. The > assignment of resources to queues is managed via sysfs. I think this > would be clearer if there were some more upstream usage examples > beyond dmaengine. However, consider one exploratory usage of > offloading memory copies in the pmem driver. Each pmem device could be > given a submission queue even if all pmem devices shared an engine on > the backend. But there are no other idxd_device_driver's in the tree, so this is all some future imaginings? Jason
