On Tue, Dec 18, 2018 at 06:25:27PM +0100, David Hildenbrand wrote: > On 18.12.18 18:13, Cornelia Huck wrote: > > On Mon, 17 Dec 2018 14:56:38 +0000 > > Stefan Hajnoczi <stefanha@xxxxxxxxxx> wrote: > > > >> On Mon, Dec 17, 2018 at 11:53:46AM +0100, David Hildenbrand wrote: > >>> On 14.12.18 14:44, Stefan Hajnoczi wrote: > >>>> On Thu, Dec 13, 2018 at 01:38:23PM +0100, Cornelia Huck wrote: > >>>>> On Thu, 13 Dec 2018 13:24:31 +0100 > >>>>> David Hildenbrand <david@xxxxxxxxxx> wrote: > > > >>>>>> As s390x does not have the concept of memory mapped io (RAM is RAM, > >>>>>> nothing else), this is not architectured. vitio-ccw can therefore not > >>>>>> define anything similar like that. However, in virtual environments we > >>>>>> can do whatever we want on top of the pure transport (e.g. on the virtio > >>>>>> layer). > >>>>>> > >>>>>> Conny can correct me if I am wrong. > >>>>> > >>>>> I don't think you're wrong, but I haven't read the code yet and I'm > >>>>> therefore not aware of the purpose of this BAR. > >>>>> > >>>>> Generally, if there is a memory location shared between host and guest, > >>>>> we need a way to communicate its location, which will likely differ > >>>>> between transports. For ccw, I could imagine a new channel command > >>>>> dedicated to exchanging configuration information (similar to what > >>>>> exists today to communicate the locations of virtqueues), but I'd > >>>>> rather not go down this path. > >>>>> > >>>>> Without reading the code/design further, can we use one of the > >>>>> following instead of a BAR: > >>>>> - a virtqueue; > >>>>> - something in config space? > >>>>> That would be implementable by any virtio transport. > >>>> > >>>> The way I think about this is that we wish to extend the VIRTIO device > >>>> model with the concept of shared memory. virtio-fs, virtio-gpu, and > >>>> virtio-vhost-user all have requirements for shared memory. > >>>> > >>>> This seems like a transport-level issue to me. PCI supports > >>>> memory-mapped I/O and that's the right place to do it. If you try to > >>>> put it into config space or the virtqueue, you'll end up with something > >>>> that cannot be realized as a PCI device because it bypasses PCI bus > >>>> address translation. > >>>> > >>>> If CCW needs a side-channel, that's fine. But that side-channel is a > >>>> CCW-specific mechanism and probably doesn't apply to all other > >>>> transports. > >>>> > >>>> Stefan > >>>> > >>> > >>> I think the problem is more fundamental. There is no iommu. Whatever > >>> shared region you want to indicate, you want it to be assigned a memory > >>> region in guest physical memory. Like a DIMM/NVDIMM. And this should be > >>> different to the concept of a BAR. Or am I missing something? > >> > >> If you implement a physical virtio PCI adapter then there is bus > >> addressing and an IOMMU and VIRTIO has support for that. I'm not sure I > >> understand what you mean by "there is no iommu"? > > > > For ccw, there is no iommu; channel-program translation is doing > > similar things. (I hope that is what David meant :) > > > >> > >>> I am ok with using whatever other channel to transport such information. > >>> But I believe this is different to a typical BAR. (I wish I knew more > >>> about PCI internals ;) ). > >>> > >>> I would also like to know how shared memory works as of now for e.g. > >>> virtio-gpu. > >> > >> virtio-gpu currently does not use shared memory, it needs it for future > >> features. > > > > OK, that all sounds like we need to define a generic, per transport, > > device agnostic way to specify shared memory. > > > > Where is that memory situated? Is it something in guest memory (like > > virtqueues)? If it is something provided by the device, things will get > > tricky for ccw (remember that there's no mmio on s390; pci on s390 uses > > special instructions for that.) > > > > I am just very very confused right now. What I am struggling with right > now (Stefan, hope you can clarify it for me): > > We need some place where this shared memory is located in the guest > physical memory. On x86 - if I am not wrong - this BAR is placed into > the reserved memory area between 3 and 4 GB. Right, the shared memory is provided by the device and does not live in guest RAM. > There is no such thing on > s390x. Because we don't have IO via memory (yet). All we have is one or > two KVM memory slots filled with all memory. > > So what we will need on s390x is on the QEMU side such a reserved memory > region where devices like virtio-fs can reserve a region for shared memory. > > So it is something like a dimm/nvdimm except that it is smaller and not > visible to the user directly (via memory backends). I see. That makes sense. Stefan
Attachment:
signature.asc
Description: PGP signature
