On 11/26/2018 01:59 PM, Stefan Hajnoczi wrote:
On Fri, Nov 23, 2018 at 10:09:53PM +0000, Felipe Franciosi wrote:
Hi Alex,
I'm also terribly sorry for the delay in responding to this. I'm only now having the time resources to come back to this topic and figure out a way forward with my proposal. Please see my notes below (quoting only the relevant sections).
On Nov 6, 2018, at 9:32 PM, Alex Williamson <alex.williamson@xxxxxxxxxx> wrote:
There were mostly two main threads of discussion, the first was Filipe's
discussion of a socket interface making use of the VFIO ABI to
implement a userspace device model. For a VM use case, QEMU would be
modified to use a socket for the VFIO ABI, including shared memory for
DMA. Ideally this would be transparent for much of the QEMU vfio code
outside of setup. Effectively this becomes a userspace implementation
of mdev as this could already be done using mdev, but it requires a
vendor driver to expose the userspace interface and likely a longer
round trip through the kernel and back to userspace. Either path
allows device models in userspace that can be open or proprietary, but
this new proposal eliminates the vendor kernel driver component. Of
course non-upstream kernel mdev vendor drivers do taint the host
kernel, so they at least leave a breadcrumb.
Spot on. This can be done today via mdev, but it would unnecessarily involve the kernel in the setup of devices. Also, since vfio is the only existing bus driver for mdev, it would make little sense to implement an mdev "physical" driver (which wouldn't have a physical backing device) just to talk a vfio-like abi back to userspace.
This "vfio-user" proposal is a perfect fit for Qemu. I think the correct design is to slightly rearchitect the vfio implementation in Qemu to allow for userspace backends through unix sockets, very much like vhost-user does it for virtio devices today.
As a matter of fact, with something like this in place, I think the Qemu code could potentially be simplified by moving some of the existing vhost-user offloading code underneath it.
I believe Filipe
mentioned a conversation after the BoF with Alex Graf who had a related
concern about using the socket approach to use this as a non-GPL
backdoor for device models in QEMU and a suggested approach was a GPL
handshake via the socket interface.
Not exactly. The GPL-violating concerns came from Paolo and Stefan (cc'd). Alex Graf (cc'd) came up with a solution for the concern which involves adding a copyrighted "poem" to the protocol handshake. Qemu would then grant the copyright for GPL applications, hence limiting who can use the protocol.
Vendors can obviously bypass this by using a GPL proxy application that talks to Qemu, but then uses a separate mechanism (possibly another unix socket) to talk to other non-GPL applications. Vendors which distribute Qemu can possibly also bypass this by modifying the protocol not to include the copyrighted messages. Using either legal loophole approach, however, can lead to a lot of public point and shame.
The point I would like to discuss more widely (before going forwards with code) is whether vendors can actually make use of a GPL-only protocol. I understand the community desire of gathering efforts into GPL software, Qemu code and Virtio. A non-GPL implementation means that maybe some vendors cannot use this.
I envision this protocol being used by userspace applications which _need_ to emulate a device outside Qemu for performance reasons, not license reasons (eg. a separate process can efficiently poll VQs from multiple VMs from a single core). Applications that already do that today, albeit open-source, are not GPL (eg. OVS, DPDK, SPDK).
This is why I'd like to hear more from Paolo and Stefan on why making this non-GPL is so bad. It is already possible today (via mdev), and similar approaches already exist for a large number of device implementations via vhost-user. Additionally, I would also like to hear from willing vendors that feel they could _only_ benefit from this if it does not enforce GPL.
Finally, I want to clarify I am not opposing making this GPL-only. I just want to make sure the effort is justifiable (ie. there is reason for this to enforce GPL peer applications given the existing vhost-user and mdev alternatives) and not in vain (ie. there are GPL use cases for this out there).
Device emulation is the core of what GPL QEMU does. Mechanisms for
out-of-tree device emulation get rejected because they are likely to be
misused by parties who do not wish to contribute to QEMU.
Are you sure? Maybe people just want to use out of tree device emulation
because it allows for sophisticated use cases:
* RTL simulation (see the Oracle talk on this from KVM Forum a few
years back)
* Security isolation (eg an SPDK based target, you don't want to give
all of QEMU access to the SPDK device)
* Reuse other OSS projects (share device emulation between different
emulators)
* Share emulation with a big broker entity (which could then poll
every VM on the entire system on 1 core, giving possibly lower latencies)
* Different programming language (say I want to implement a device in
rust, because it's more secure, or easier to prototype with)
I can see the FUD around closed source and terrible quality of out of
tree devices, but that doesn't mean they are the only reason people want
this. And I also don't think it's a good reason to hold back
compartmented device emulation.
That would result in the QEMU community maintaining infrastructure for
third parties who ship code of questionable quality and profit from it.
You mean like we do today for the Ceph block module? In-tree vs
out-of-tree doesn't say too much about code quality. And I surely hope
people understand that a separate host process from a separate entity
means that the overall product can only be as good as its components.
That's not healthy for the QEMU community and there's no incentive to
allow it.
I disagree - I think it's quite healthy, because it allows us to grow
into use cases we currently neglect or have random downstream projects
for at this point. There are numerous forks of QEMU to allow RTL
simulation of external devices for example.
A bit more about specifics you mentioned:
vhost-user is an exception for vhost devices. They are based on the
open VIRTIO specification. QEMU has in-tree implementations of these
devices so users can fall back to them if they need to request support
(otherwise it's impossible to support users with third-party devices).
Generic out-of-tree PCI device emulation is way too big a loophole. The
QEMU community cannot support users who run arbitrary third-party PCI
devices.
The fact that there are ways to get around this by writing software mdev
devices doesn't mean it makes sense for QEMU to open a loophole itself.
IMHO it would be much more sensible to give people a path we can support
rather than encourage hacks, where hacks are either:
* downstream patches to do out of tree emulation
* mdev based implementations of the same
So how about we give people a reasonable interface that allows them to
support their sophisticated things without us getting penalized for it?
What really are the problems you're seeing? From all discussions I know
so far, it's mostly been:
1) Code Quality
We can always have a tainted flag. IIRC Gerd already worked on bits for
unmaintained code inside of QEMU itself. There's little reason to not
extend this to external modules. We should treat "someone uses random
emulated device nobody maintains" and "someone uses external code" the
same way.
2) GPL
One idea I had here was to enforce licensing via legal methods. Linux
for example has an explicit tag that allows kernel modules to expose
their license information to Linux. Based on that, it can expose
interfaces or refuse to do so. We could do the same - and for starters
only expose PCIe device emulation to GPL peers.
Alex
