On Wed, Jan 21, 2009 at 03:25:20AM +0000, Jamie Lokier wrote: > Nick Piggin wrote: > > > For database writes, you typically write a bunch of stuff in various > > > regions of a big file (or multiple files), then ideally fdatasync > > > some/all of the written ranges - with writes committed to disk in the > > > best order determined by the OS and I/O scheduler. > > > > Do you know which databases do this? It will be nice to ask their > > input and see whether it helps them (I presume it is an OSS database > > because the "big" ones just use direct IO and manage their own > > buffers, right?) > > I just found this: > > http://markmail.org/message/injyo7coein7o3xz > (Postgresql) > > Tom Lane writes (on org.postgreql.pgsql-hackets): > >Greg Stark <gsst...@xxxxxxx> writes: > >> Come to think of it I wonder whether there's anything to be gained by > >> using smaller files for tables. Instead of 1G files maybe 256M files > >> or something like that to reduce the hit of fsyncing a file. > >> > >> Actually probably not. The weak part of our current approach is that > >> we tell the kernel "sync this file", then "sync that file", etc, in a > >> more or less random order. This leads to a probably non-optimal > >> sequence of disk accesses to complete a checkpoint. What we would > >> really like is a way to tell the kernel "sync all these files, and let > >> me know when you're done" --- then the kernel and hardware have some > >> shot at scheduling all the writes in an intelligent fashion. > >> > >> sync_file_range() is not that exactly, but since it lets you request > >> syncing and then go back and wait for the syncs later, we could get > >> the desired effect with two passes over the file list. (If the file > >> list is longer than our allowed number of open files, though, the > >> extra opens/closes could hurt.) > >> > >> Smaller files would make the I/O scheduling problem worse not better. Interesting. > So if you can make > commit-to-multiple-files-in-optimal-I/O-scheduling-order work, that > would be even better ;-) fsyncv? Send multiple inode,range tuples to the kernel to sync. -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
