Hi Anthony, Anthony Liguori wrote: > Gregory Haskins wrote: >> Gregory Haskins wrote: >> >>> Ingo Molnar wrote: >>> >>>> We all love faster code and better management interfaces and tons of >>>> your prior patches got accepted by Avi. This time you didnt even >>>> _try_ to improve virtio. >>>> >>> Im sorry, but you are mistaken: >>> >>> http://lkml.indiana.edu/hypermail/linux/kernel/0904.2/02443.html >>> >>> >> >> BTW: One point that I forgot to point out in this most recent thread >> that I am particularly proud of here is the design of the vbus >> shared-memory model. Despite some claims to the contrary; not only is >> it possible to improve virtio with vbus (as evident by the patch >> referenced above)...I specifically designed vbus with virtio >> considerations in mind from the start! In fact, the design is conducive >> to accelerating a variety of other models as well. Read on for details. >> >> Vbus was designed it to be _agnostic_ to the shm algorithm in general. >> This allows you to, of course, run ring algorithms (such as virtqueues, >> or IOQs), but really any other designs as well, such as shared-tables, >> etc. >> >> A guest driver sees the following interface: >> >> struct vbus_device_proxy_ops { >> int (*open)(struct vbus_device_proxy *dev, int version, int flags); >> int (*close)(struct vbus_device_proxy *dev, int flags); >> int (*shm)(struct vbus_device_proxy *dev, int id, int prio, >> void *ptr, size_t len, >> struct shm_signal_desc *sigdesc, struct shm_signal **signal, >> int flags); >> int (*call)(struct vbus_device_proxy *dev, u32 func, >> void *data, size_t len, int flags); >> void (*release)(struct vbus_device_proxy *dev); >> }; >> >> note the ops->shm() method. This allows the driver to register some >> arbitrary pointer (ptr, len) with the host, optionally embedding a >> shm_signal_desc object in the memory. If "sigdesc" is non-null, the >> connector will allocate and return a fully formed shm_signal object in >> **signal. >> > > Fundamentally, how is this different than the virtio->add_buf concept? From my POV, they are at different levels. Calling vbus->shm() is for establishing a shared-memory region including routing the memory and signal-path contexts. You do this once at device init time, and then run some algorithm on top (such as a virtqueue design). virtio->add_buf() OTOH, is a run-time function. You do this to modify the shared-memory region that is already established at init time by something like vbus->shm(). You would do this to queue a network packet, for instance. That said, shm-signal's closest analogy to virtio would be vq->kick(), vq->callback(), vq->enable_cb(), and vq->disable_cb(). The difference is that the notification mechanism isn't associated with a particular type of shared-memory construct (such as a virt-queue), but instead can be used with any shared-mem algorithm (at least, if I designed it properly). The closest analogy for vbus->shm() to virtio would be vdev->config->find_vqs(). Again, the difference is that the algorithm (ring, etc) is not dictated by the call. You then overlay something like virtqueue on top. > > virtio provides a mechanism to register scatter/gather lists, associate > a handle with them, and provides a mechanism for retrieving notification > that the buffer has been processed. Yes, and I agree this is very useful for many/most algorithms...but not all. Sometimes you don't want ring-like semantics, but instead want something like an idempotent table. (Think of things like interrupt controllers, timers, etc). Rings, of course, have a trait that all updates are retained in fifo order. For many things (e.g. network, block io, etc), this is exactly what you want. If I say "send packet X" now, and "send packet Y" later, I want the system to do both (and perhaps in that order), so a ring scheme works well. However, sometimes you may want to say "time is now X", and later "time is now Y". The update value of 'X' is technically superseded by Y and is stale. But a ring may allow both to exist in-flight within the shm simultaneously if the recipient (guest or host) is lagging, and the X may be processed even though its data is now irrelevant. What we really want is the transform of X->Y to invalidate anything else in flight so that only Y is visible. So in a case like this, we may want a different algorithm. Something like a table which always contains the current/valid value, and a way to signal in both directions when something interesting happens to that data. If you think about it, a ring is a superset of this construct...the ring meta-data is the "shared-table" (e.g. HEAD ptr, TAIL ptr, COUNT, etc). So we start by introducing the basic shm concept, and allow the next layer (virtio/virtqueue) in the stack to refine it for its needs. > > vbus provides a mechanism to register a single buffer with an integer > handle, priority, and a signaling mechanism. Again, I think we are talking about two different layers. You would never put entries into a virtio-ring of different priority. This doesn't make sense, as they would just get linearized by the fifo. What you *would* do is possibly make multiple virtqueues, each with a different priority (for instance, say 8-rx queues for virtio-net). > > So virtio provides builtin support for scatter/gathers whereas vbus > models priority. But fundamentally, they seem like almost identical > concepts. I would say that virtqueue and IOQ are a much closer analogy in terms of comparison at the scatter-gather level. The virtio device model itself is similar to a vbus device-model except its oriented towards the virtqueue ring design. In addition, a big part of vbus is also what happens _behind_ the device model. > > If we added priority to virtio->add_buf, would it be equivalent in your > mind functionally speaking? As indicated above, this wouldn't be sane. A better design (IMO) is to use a ring per priority. > > What does one do with priority, btw? There are, of course, many answers to that question. One particularly trivial example is 802.1p networking. So, for instance, you can classify and prioritize network traffic so that things like control/timing packets are higher priority than best-effort HTTP. Doing this "right" means you have end-to-end priority within the system (e.g. your switch/fabric, nics, interrupt controllers, etc). Today, virt is fairly far removed from being fully integrated in this sense, but the vbus project is addressing this short-coming. HTH, Kind Regards, -Greg
Attachment:
signature.asc
Description: OpenPGP digital signature
