> A data point regarding reiserfs/ext3: > > We are in the process of moving from reiserfs to ext3 (with dir_index). > > ext3 seems to do substantially better than reiserfs for us, especially for > read heavy loads (squatter runs at least twice as fast as it used do). Are you comparing an "old" reiserfs partition with a "new" ext3 one where you've just copied the email over to? If so, that's not a fair comparison. What we found was that when we first copied the data from reiserfs -> ext3, everything seemed nice, but that's only because when you copy mailboxes over, you're effectively defragmenting all the files and directories in your filesystem. We've actually been thinking of setting up a regular process to just randomly move users around, because the act of moving them effectively defragments them regardless of the filesystem you're using! Give it a month or two of active use though (delivering new emails, deleting old ones, etc), and everything starts getting fragmented again. Then ext3 really started going to crap on us. Machines that had been absolutely fine under reiserfs, the load just blew out to unuseable under ext3. > data=ordered in both cases. data=journal didn't seem to make any > difference with ext3. data=journal with reiserfs caused amusing kernel > memory leaks, which it looks like Fastmail also hit recently. An dedicated > journal device would probably make a big difference with data=journal. Talking with Chris Mason about this, data=journal is faster in certain scenarios with lots of small files + fsyncs from different processes, exactly the type of workload cyrus generates! As it turns out, the memory leaks weren't critical, because the the pages do seem to be reclaimed when needed, though it was annoying not knowing exactly how much memory was really free/used. The biggest problem was that with cyrus you have millions of small files, and with a 32bit linux kernel the inode cache has to be in low memory, severely limiting how many files the OS will cache. See this blog post for the gory details, and why a 64-bit kernel was a nice win for us. http://blog.fastmail.fm/2007/09/21/reiserfs-bugs-32-bit-vs-64-bit-kernels-cache-vs-inode-memory/ Rob ---- Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
