Hi,
I've done some testing on old hardware (5 nodes with 1 GHz Athlon, 512 MB
RAM, 40 GB IDE-disks, 100 Mbit/s ethernet). Both NFS and GlusterFS fill
the ethernet when using large blocksizes. For small blocksizes however
GlusterFS is running out of CPU... Is there any good way to minimize the
CPU usage on the GlusterFS client and to increase small blocksize
performance? I'm trying different combinations with the translators to see
if I get slightly better configuration (but after removing all of them
bonnie++ take ages to complete.) :)
bonnie++ 1.03:
Seq Output----- Seq Input-----
Per Chr CPU Per Chr CPU
local disk: 17324 K/s 94% 18154 K/s 89%
GlusterFS: 4305 K/s 21% 7195 K/s 36%
NFS: 6197 K/s 29% 11253 K/s 56%
Well, NFS seem to be more efficient on this old hardware than GlusterFS...
One of the reasons that I find GlusterFS interesting is the possibility to
increase the number of IO per second when using many drives in parallell.
That has been the major performance bottleneck for me when running
mail/webhosting. More drives help, and adding more servers instad of
buying large RAIDs would be neat. In a way, yes it seem to scale, but as
far as I understand at the cost that every node will spend more CPU
accessing the file system. Perhaps not a problem on modern hardware?
I also tried some bittorrent-seeding from GlusterFS. For
bittorrent-clients not using much read-ahead cache this is normally
limited by the disk seek-time (IO/s). I was hoping that utilizing remote
disks with a total of five times the IO/s than the local disk would
increase the seeding rate, but instead GlusterFS and the bittorrent is
just competing for CPU. Well well, for bittorrent the problem seem to be
solved with readahead cache in the modern clients anyway, increasing the
RAM cache hit-rate a lot.
All bonnie output here:
http://www.update.uu.se/~jerker/tmp/glusterfs.bonnie.txt
Regards,
Jerker Nyberg.
On Mon, 15 Oct 2007, Steffen Grunewald wrote:
What's not so beautiful is that the first dd (always nfs) does
include staging of the file from the input media into buffer
cache (/dev/zero means: filling memory with zero bytes, which
certainly is faster than reading from a physical disk).
I would have repeated the write tests to see whether ordering
is important:
- nfs write
- glusterfs write
- nfs write again
- glusterfs write again
Buffers are often able to fool the benchmarker.
Also some information about your machine is missing - but I suppose 1GB
would easily fit into main memory. What about *several* GBs to effectively
trash the page cache?
Cheers,
Steffen, always doubtful when it comes to benchmarks
--
Steffen Grunewald * MPI Grav.Phys.(AEI) * Am Mühlenberg 1, D-14476 Potsdam
Cluster Admin * http://pandora.aei.mpg.de/merlin/ * http://www.aei.mpg.de/
* e-mail: steffen.grunewald(*)aei.mpg.de * +49-331-567-{fon:7233,fax:7298}
No Word/PPT mails - http://www.gnu.org/philosophy/no-word-attachments.html
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxx
http://lists.nongnu.org/mailman/listinfo/gluster-devel
