On Sun, 20 Mar 2011 18:55:32 -0700 Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> wrote: > > > The pagecache trick potentially gives zero-copy access, whereas > > > the proposed code is single-copy. Although the expected benefits > > > of that may not be so great due to TLB manipulation overheads. > > > > > > I worry that one day someone will come along and implement the > > > pagecache trick, then we're stuck with obsolete code which we > > > have to maintain for ever. > > > > Perhaps I don't understand what you're saying correctly but I think > > that one problem with the zero copy page flipping approach is that > > there is no guarantee with the data that the MPI apps want to send > > resides in a page or pages all by itself. > > Well. The applications could of course be changed. But if the > applications are changeable then they could be changed to use > MAP_SHARED memory sharing and we wouldn't be having this discussion, > yes? > > (Why can't the applications be changed to use existing shared memory > capabilities, btw?) An MPI application commonly doesn't know in advance when allocating memory if the data it will eventually be sending will be to a local node or remote node process. It will depend on the configuration of the cluster that you run the application on and parameters when you start it up (eg how many processes per node to start etc), and exactly how the program ends up executing. So short of allocating everything to be shared memory just in case you want intranode communication we can't use shared memory cooperatively like that to reduce copies. Shared memory *is* often used for intranode communication, but in a copy-in to shared memory on the sender and copy-out on the receiver side. We did originally do some early hacking on hpcc where we did allocate everything from a shared memory pool just to see what sort of theoretical gain we could have from a single-copy model, but its not a solution we can use in general. Regards, Chris -- cyeoh@xxxxxxxxxx -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxxx For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>