On Tue, Aug 30, 2011 at 2:14 PM, Jeff Moyer <jmoyer@xxxxxxxxxx> wrote: > Daniel Ehrenberg <dehrenberg@xxxxxxxxxx> writes: > >> On Tuesday, August 30, 2011, Jeff Moyer <jmoyer@xxxxxxxxxx> wrote: >>> Dan Ehrenberg <dehrenberg@xxxxxxxxxx> writes: >>> >>>> When a single thread is reading from a libaio io_context_t object >>>> in a non-blocking polling manner (that is, with the minimum number >>>> of events to return being 0), then it is possible to safely read >>>> events directly from user-space, taking advantage of the fact that >>>> the io_context_t object is a pointer to memory with a certain layout. >>>> This patch adds an option, userspace_libaio_reap, which allows >>>> reading events in this manner when the libaio engine is used. >>> >>> I haven't yet tried to poke holes in your code, but I'm pretty sure I >>> can find some. I have patches for the kernel and libaio which allow >>> user-space reaping of events. Why don't I dust those off and post them, >>> and then fio won't have to change at all? That seems like to proper >>> approach to solving the problem. >>> >>> Cheers, >>> Jeff >>> >> >> Ken Chen posted some patches which accomplish this in 2007. However, I was a >> little concerned about his lock-free queue structure--it seems like an >> integer overflow might cause events to be lost; on the other hand this is >> very unlikely. Are you talking about Ken's patchset or another one? > > I'm talking about another one, since I completely forgot about Ken's > patches. Thanks for reminding me! > > I'm not sure your approach will work on all architectures without kernel > modification. I think at least a flush_dcache_page may be required. > More importantly, though, I'd like to discourage peeking into the > internals outside of the libaio library. Doing this makes it really > hard to change things moving forward. Are you saying flush_dcache_page from the kernel or from user-space? What kind of architecture will have problems? What kind of failures could result? About ABI dependence: I agree, it would be better to have things in libaio rather than here. But to do reaping like this, with certain restrictions about what context it's used in, I think we'd have to make another function rather than just changing io_getevents. At first, I was looking into changing io_getevents, but then I realized that the application I'm working on optimizing, like FIO, is only calling io_getevents in a certain pattern, making all of the synchronization unnecessary. I don't think these two things are the only users of io_getevents in this pattern. But maybe proper synchronization can be made cheap enough that there's not a big penalty for doing it properly all the time. > > Cheers, > Jeff > -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
