On Wed, Sep 09, 2020 at 05:29:20PM +0200, Cornelia Huck wrote: > On Wed, 9 Sep 2020 14:40:33 +0200 > Erik Skultety <eskultet@xxxxxxxxxx> wrote: > > > On Wed, Sep 09, 2020 at 08:23:39AM +0200, Bjoern Walk wrote: > > > Erik Skultety <eskultet@xxxxxxxxxx> [2020-09-08, 06:01PM +0200]: > > > > On Mon, Aug 24, 2020 at 01:59:15PM +0200, Bjoern Walk wrote: > > > > > From: Boris Fiuczynski <fiuczy@xxxxxxxxxxxxx> > > > > > + case VIR_NODE_DEV_CAP_SYSTEM: > > > > > + case VIR_NODE_DEV_CAP_USB_DEV: > > > > > + case VIR_NODE_DEV_CAP_USB_INTERFACE: > > > > > + case VIR_NODE_DEV_CAP_NET: > > > > > + case VIR_NODE_DEV_CAP_SCSI_HOST: > > > > > + case VIR_NODE_DEV_CAP_SCSI_TARGET: > > > > > + case VIR_NODE_DEV_CAP_SCSI: > > > > > + case VIR_NODE_DEV_CAP_STORAGE: > > > > > + case VIR_NODE_DEV_CAP_FC_HOST: > > > > > + case VIR_NODE_DEV_CAP_VPORTS: > > > > > + case VIR_NODE_DEV_CAP_SCSI_GENERIC: > > > > > + case VIR_NODE_DEV_CAP_DRM: > > > > > + case VIR_NODE_DEV_CAP_MDEV_TYPES: > > > > > + case VIR_NODE_DEV_CAP_MDEV: > > > > > + case VIR_NODE_DEV_CAP_CCW_DEV: > > > > > > > > What about ccw mdevs? I would think that it's CCW end point device that > > > > we would want to assign concurrently, but we're only "slicing" the root channel > > > > subsystem device. If you have and CSS mdev, are the CCW subchannels separate > > > > for each VM or does it behave kind of like NPIV with its vHBAs? OR I'm > > > > completely off here? > > > > > > You are correct. This is an unfortunate architectural decision on our > > > platform. For vfio-ccw, we instatiate the mdevs on the CSS layer, which > > > has a 1-to-1 relation to the underlying CCW device. That's why we need > > > the information about the CSS devices in libvirt, to give the user a > > > chance to easily find this relation when he wants to do passthrough with > > > a DASD for example. It's a bit unintuitive and even silly for the > > > end user, but it's architecturally correct and hence it was implemented > > > like this in the vfio-ccw kernel driver. We don't even get the benefits > > > of something like NPIV because for every subchannel there is only one > > > CCW device (at least to my knowledge). > > > > Having read both Connie's blog on channels and the I/O chapter in the free > > architectural publication you linked in the other reply, I now have a very > > vague understating of the architecture. One thing that still > > puzzles me though is that if there's a 1:1 subchannel mapping between CSS and an > > CCW device which then communicates via a control unit (which there may be more > > than 1) with CSS over up to 8 channel paths (this at least to me resembles > > NPIV), what does the mdev really solve, can't you assign one of the channel > > paths of one of the several control units the device is connected to to > > plug into the VM? Well, clearly not, I'm just being curious and trying to > > understand the use case here. > > I think it's easier to leave control units mostly out of the picture, > as they are not really relevant for the modelling here. And yes, there > is always exactly one subchannel for an I/O device. > > Subchannels (mostly a concept describing how to interact with a device) > and channel paths (mostly corresponding to the way data flows to/from > the device) are probably best understood as different ways to describe > how devices are accessed. You don't really want to pass channel paths > (as they are shared between many devices); the subchannel is a more > natural unit to pass. The operating system also has way more control > over subchannels than over individual channel paths. Gotcha. > > Regarding mdev, the idea here is less to be able to slice up a device > as you do with GPUs, but more to be able to add a vendor driver that > does the actual interesting work (channel program translations etc.) > > As to why we use the subchannel instead of the device, that's mostly > following from how the low-level channel I/O layer in the Linux kernel > is organized: The default I/O subchannel driver does a lot of things > that a normal device driver should not need to deal with (path > grouping, verification, status accumulation, etc.), but which would get > into the way if you wanted to assign the device via vfio. Therefore, > vfio-ccw implements an alternate I/O subchannel driver. Okay, now it's much clearer what the mdev use case is. Thanks Connie. Erik
