On 01/15/2013 12:36 PM, Gregory Farnum wrote: > On Tue, Jan 15, 2013 at 10:33 AM, Dimitri Maziuk <dmaziuk@xxxxxxxxxxxxx> wrote: >> At the start of the batch #cores-in-the-cluster processes try to mmap >> the same 2GB and start reading it from SEEK_SET at the same time. I >> won't know until I try but I suspect it won't like that. > > Well, it'll be #servers-in-cluster serving up 4MB chunks out of cache. > It's possible you could overwhelm their networking but my bet is > they'll just get spread out slightly on the first block and then not > contend in the future. In the future the application spreads out the reads as well: running instances go through the data at different speed, and when one's finished, the next one starts on the same core & it mmap's the first chunk again. > Just as long as you're thinking of it as a test system that would make > us very happy. :) Well, IRL this is throw-away data generated at the start of a batch, and we're good if one batch a month runs to completion. So if it doesn't crash all the time every time, that actually should be good enough for me. However, not all of the nodes have spare disk slots, so I couldn't do a full-scale deployment anyway, not without rebuilding half the nodes. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu
Attachment:
signature.asc
Description: OpenPGP digital signature
