> I write scientific number-crunching codes which deal with large > input and output files (Gb, tens of Gb, as much as hundreds of > Gygabytes on occasions). Now there are these multi-core setups > like Intel Core 2 Duo becoming available at a low cost. Disk I/O > is one of the biggest bottlenecks. I would very much like to put > to use the multiple processors to read or write in parallel > (probably using OMP directives in a C program). How do you think more processors is going to help? Disk I/O doesn't take much processor. It's hard to imagine a realistic disk I/O application that was somehow limited by available CPU. > My question is -- can the filesystem actually read/write to files > truly in parallel Yes. > (more processors -- faster read/write), No. Parallel I/O does not require more processors. > or if I > write such a code the I/O commands from the CPUs will just queue > after each other and it would be the same thing as using a single > CPU? Why do you think the I/O commands juse queue after each other on a single CPU? That is a very mistaken view. As further progress is capable to make on each operaiton, a single CPU will make the progress. But that doesn't mean it works one operation from start to finish before it gets to the next one. > If parallel I/O is possible, can it be accomplished entirely > transparently, or using special libraries, or only in special > circumstances, like reading in parallel with N CPUs from N > different physical disks? Or only on some types of hardware? Is > there a max nr of threads/processes that can write to disk in > parallel? If ext3 does not do this, which (stable) Linux > filesystem does it? You're asking the wrong questions because you asking about CPUs. One CPU can issue multiple I/O requests without waiting for each one to finish before issuing the next one. It is a legitimate question how you perform I/O asynchronously. There are a few ways such as threads and POSIX asynchronous I/O. DS _______________________________________________ Ext3-users mailing list Ext3-users@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/ext3-users
