On Wed, 20 Oct 2010, Colin McCabe wrote: > > SimpleMessenger will change no matter what, because the core problem that > > we're trying to solve is that it allocates a new buffer when a message is > > being read off the wire. See SimpleMessenger::read_message(). Somehow > > that needs to discover and use the user provided buffer instead. That > > will invariably involve a callback of some sort to find the user buffer. > > And some means of yanking the buffer away to deal with the corner issue > > above. > > I can see now that we're always allocating a new bufferlist in > SimpleMessenger. I can see why we want this feature. > > I wish we didn't have to have a callback just to get the buffer we're > trying to write into. It feels like that should somehow be grouped > with the rest of the transaction data. Why can't we add it to > Objector::Op's constructor, for example. There must be some obvious > reason but I don't see it yet. Well, we could push it into the Objecter. We could also try to push it all the way down into the messenger, my maintaining a map of <type,tid> to buffer or something. Mainly I want to avoid changing too much of the intervening interfaces if we can avoid it. That may be possible, though. If the Objecter::read() is given a non-empty bufferlist* to read into, it could advertise that buffer to the messenger and the messenger could handle it all. But the same problem comes up: we need to also have an interface to take the buffer back, which will mean some interesting interaction with the reader threads. The kernel messenger avoids this, btw, because it is event driven: we wake up when there is socket activity and do non-blocking reads/writes, so we can lock the buffers while doing that IO. The SimpleMessenger implementation uses threads, and IIRC the pthread locking and select/poll interfaces don't play well together. sage
