On Wed, Oct 20, 2010 at 9:53 AM, Sage Weil <sage@xxxxxxxxxxxx> wrote: > On Wed, 20 Oct 2010, Colin McCabe wrote: >> > SimpleMessenger will change no matter what, because the core problem that >> > we're trying to solve is that it allocates a new buffer when a message is >> > being read off the wire. See SimpleMessenger::read_message(). Somehow >> > that needs to discover and use the user provided buffer instead. That >> > will invariably involve a callback of some sort to find the user buffer. >> > And some means of yanking the buffer away to deal with the corner issue >> > above. >> >> I can see now that we're always allocating a new bufferlist in >> SimpleMessenger. I can see why we want this feature. >> >> I wish we didn't have to have a callback just to get the buffer we're >> trying to write into. It feels like that should somehow be grouped >> with the rest of the transaction data. Why can't we add it to >> Objector::Op's constructor, for example. There must be some obvious >> reason but I don't see it yet. > > Well, we could push it into the Objecter. We could also try to push it > all the way down into the messenger, my maintaining a map of <type,tid> to > buffer or something. Mainly I want to avoid changing too much of the > intervening interfaces if we can avoid it. That may be possible, though. > If the Objecter::read() is given a non-empty bufferlist* to read into, it > could advertise that buffer to the messenger and the messenger could > handle it all. I kind of like the idea of always having the Objecter to pass in a bufferlist. If the user wants to allocate a new buffer each time, he knows how to call bufferlist::new(). > But the same problem comes up: we need to also have an interface to take > the buffer back, which will mean some interesting interaction with the > reader threads. Yeah, we would need to know when it's safe for the client to read / reuse the buffer they passed in. Colin > The kernel messenger avoids this, btw, because it is event driven: we wake > up when there is socket activity and do non-blocking reads/writes, so we > can lock the buffers while doing that IO. The SimpleMessenger > implementation uses threads, and IIRC the pthread locking > and select/poll interfaces don't play well together. > > sage > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
