On 08/29/2012 07:57 AM, Sylvain Munaut wrote:
Hi,
It might be obvious for people knowing the API but somehow I can't
figure it out:
How and when will the call back specified in a rbd_completion_t be called ?
Imagine I do a rbd_aio_write and then do while (1); ... I don't see
how the library could call my callback unless there is threads
involved (and then my cb would be called in another thread context
which is not great ...)
It's called from another thread. There are several threads for
messaging started when you connect with a rados_cluster_t. When a
request is complete, it's put into a queue inside librados, and a
'finisher' thread reads from the queue and calls callbacks as
appropriate.
In the case of rbd, a single rbd_aio_write may translate to many rados
requests, but when the last is complete, the callback you provided is
called by the librados finisher thread. At this point the data is on
disk on all replicas.
(Of course the 'while (1)' here is just a stub, in reality it's a call
to a library beyond my control)
My problem here is that I need to integrate inside the main loop of
another software and I can't block, but I still need to be able to
'notify' when the requests are over. I can ask the scheduler of that
main loop to call me on some even on a file descriptor (but I don't
see any fd here), or to call me periodically (but I don't see any call
to test for completion without waiting for it).
The way this is done in the qemu block driver (since qemu I/O
must be completed from the original qemu thread context) is to have a
pipe that the qemu thread reads from. When an rbd I/O completes, the
callback (called by a librados thread) tells the qemu thread that it's
done via the pipe. Here's the original discussion about this:
http://lists.gnu.org/archive/html/qemu-devel/2010-10/msg00420.html
and the code:
http://git.qemu.org/?p=qemu.git;a=blob;f=block/rbd.c;h=5a0f79fc8f38e90c5c635a86b60db4c0f2f20c2f;hb=HEAD
Cheers,
Sylvain
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
