On Wed, Nov 30, 2011 at 6:24 PM, Ohad Ben-Cohen <ohad@xxxxxxxxxx> wrote: > On Wed, Nov 30, 2011 at 6:15 PM, Michael S. Tsirkin <mst@xxxxxxxxxx> wrote: >> How are the rings mapped? normal memory, right? > > No, device memory. Ok, I have more info. Originally remoteproc was mapping the rings using ioremap, and that meant ARM Device memory. Recently, though, we moved to CMA (allocating memory for the rings via dma_alloc_coherent), and that isn't Device memory anymore: it's uncacheable Normal memory (on ARM v6+). We still require mandatory barriers though: one very reproducible problem I personally face is that the avail index doesn't get updated before the kick. As a result, the remote processor misses a buffer that was just added (the kick wakes it up only to find that the avail index wasn't changed yet). In this case, it probably happens because the mailbox, used to kick the remote processor, is mapped as Device memory, and therefore the kick can be reordered before the updates to the ring can be observed. I did get two additional reports about reordering issues, on different setups than my own, and which I can't personally reproduce: the one I've described earlier (avail index gets updated before the avail array) and one in the receive path (reading a used buffer which we already read). I couldn't personally verify those, but both issues were reported to be gone when mandatory barriers were used. I expect those reports only to increase: the diversity of platforms that are now looking into adopting virtio for this kind of inter-process communication is quite huge, with several different architectures and even more hardware implementations on the way (not only ARM). Thanks, Ohad. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
