Re: LARGE single-system Cyrus installs?

[Ian G Batten <ian.batten@xxxxxxxxxxxxxx>]

On 15 Nov 07, at 1504, Michael Bacon wrote:

Interesting thought.  We haven't gone to ZFS yet, although I like the idea 

a lot.  My hunch is it's an enormous win for the mailbox partitions, but 

perhaps it's not a good thing for the meta partition.  I'll have to let 

someone else who knows more about ZFS and write speeds vs. read speeds 

chime in here.

We're finding it a real win for the meta-partition.  We're handing ~1000 users on a 2-way stripe by two-way mirror on the internal disks in a T2000 for the meta-data, with the message data coming in over NFS.    We do see a few spikes of write operations (this is one instance from zpool isotat -v 1):

                 capacity     operations    bandwidth

pool           used  avail   read  write   read  write

------------  -----  -----  -----  -----  -----  -----

pool1         52.1G  25.9G      4    657  3.96K  3.71M

  mirror      26.0G  13.0G      4    354  3.96K  1.42M

    c0t0d0s4      -      -      0    135      0  1.42M

    c0t1d0s4      -      -      0    126  63.4K  1.42M

  mirror      26.0G  13.0G      0    302      0  2.29M

    c0t2d0s4      -      -      0    112      0  2.29M

    c0t3d0s4      -      -      0    109      0  2.29M

------------  -----  -----  -----  -----  -----  -----

but it's showing no signs at all of being IO bound on the metadata.   The spikes are really just spikes for a second: the typical level is about 10 ops / disk / sec.  

ian

----
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html