On Thu, Jan 12, 2017 at 2:14 PM, Matt Benjamin <mbenjamin@xxxxxxxxxx> wrote: > Hi, > > ----- Original Message ----- >> From: "Yehuda Sadeh-Weinraub" <ysadehwe@xxxxxxxxxx> >> To: "Sage Weil" <sweil@xxxxxxxxxx> >> Cc: "Gregory Farnum" <gfarnum@xxxxxxxxxx>, "Jason Dillaman" <dillaman@xxxxxxxxxx>, "Piotr Dałek" >> <piotr.dalek@xxxxxxxxxxxx>, "ceph-devel" <ceph-devel@xxxxxxxxxxxxxxx>, "ceph-users" <ceph-users@xxxxxxxxxxxxxx> >> Sent: Thursday, January 12, 2017 3:22:06 PM >> Subject: Re: [ceph-users] Any librados C API users out there? >> >> On Thu, Jan 12, 2017 at 12:08 PM, Sage Weil <sweil@xxxxxxxxxx> wrote: >> > On Thu, 12 Jan 2017, Gregory Farnum wrote: >> >> On Thu, Jan 12, 2017 at 5:54 AM, Jason Dillaman <jdillama@xxxxxxxxxx> >> >> wrote: >> >> > There is option (3) which is to have a new (or modified) >> >> > "buffer::create_static" take an optional callback to invoke when the >> >> > buffer::raw object is destructed. The raw pointer would be destructed >> >> > when the last buffer::ptr / buffer::list containing it is destructed, >> >> > so you know it's no longer being referenced. >> >> > >> >> > You could then have the new C API methods that wrap the C buffer in a >> >> > bufferlist and set a new flag in the librados::AioCompletion to delay >> >> > its completion until after it's both completed and the memory is >> >> > released. When the buffer is freed, the callback would unblock the >> >> > librados::AioCompltion completion callback. >> >> >> >> I much prefer an approach like this: it's zero-copy; it's not a lot of >> >> user overhead; but it requires them to explicitly pass memory off to >> >> Ceph and keep it immutable until Ceph is done (at which point they are >> >> told so explicitly). >> > >> > Yeah, this is simpler. I still feel like we should provide a way to >> > revoke buffers, though, because otherwise it's possible for calls to block >> > semi-indefinitey if, say, an old MOSDOp is quueed for another OSD and that >> > OSD is not reading data off the socket but has not failed (e.g., due to >> > it's rx throttling). >> > >> >> We need to provide some way to cancel requests (at least from the >> client's aspect), that would guarantee that buffers are not going to >> be used (and no completion callback is going to be called). > > is the client/consumer cancellation async wrt completion? a cancellation in that case could ensure that, if it succeeds, those guarantees are met, or else fails (because the callback and completion have raced cancellation)? > A cancellation succeeding would mean that no completion callback is triggering, nor will it trigger. This means that in order for the cancellation to complete, any ongoing completions will need to finish. So either it waits on the completion to finish or abort cancellation. Yehuda -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
