On 9/29/11 9:36 AM, Andrey wrote: > Let me to share some testing RAID 5 results with bonnie++: What kernel version was this tested on? Thanks, -Eric > ext3 (defaults,noatime): > > Version 1.96 ------Sequential Output------ --Sequential Input- --Random- > Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP > debian 2G 242 96 22458 10 8826 2 1854 98 120985 11 317.1 3 > Latency 211ms 896ms 720ms 22258us 18733us 622ms > Version 1.96 ------Sequential Create------ --------Random Create-------- > debian -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > 16 12857 33 +++++ +++ 15377 34 13585 33 +++++ +++ 15404 35 > Latency 12284us 992us 1029us 432us 140us 76us > > ext3 (-T small,defaults,noatime): > > Version 1.96 ------Sequential Output------ --Sequential Input- --Random- > Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP > debian 2G 229 98 4989 5 3862 1 1762 97 91111 9 266.6 6 > Latency 79046us 22858ms 2577ms 19253us 12120us 767ms > Version 1.96 ------Sequential Create------ --------Random Create-------- > debian -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > 16 6422 16 +++++ +++ 10319 25 8934 21 +++++ +++ 10347 26 > Latency 9968us 977us 964us 482us 144us 178us > > ext3 (-T news,defaults,noatime): > Version 1.96 ------Sequential Output------ --Sequential Input- --Random- > Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP > debian 2G 237 95 22807 11 8807 2 1897 99 121893 11 324.6 5 > Latency 223ms 808ms 523ms 13765us 11049us 831ms > Version 1.96 ------Sequential Create------ --------Random Create-------- > debian -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > 16 12826 33 +++++ +++ 15900 35 14548 36 +++++ +++ 15460 35 > Latency 417us 984us 1024us 430us 140us 175us > > ext4 (defaults,noatime): > > Version 1.96 ------Sequential Output------ --Sequential Input- --Random- > Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP > debian 2G 256 98 21495 6 9896 2 1771 99 125775 11 349.7 5 > Latency 37738us 992ms 3490ms 10811us 12045us 495ms > Version 1.96 ------Sequential Create------ --------Random Create-------- > debian -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > 16 14766 43 +++++ +++ 18026 46 16094 46 +++++ +++ 17428 45 > Latency 424us 982us 1023us 367us 139us 174us > > xfs(defaults,noatime,logbufs=8,logbsize=131072): > > Version 1.96 ------Sequential Output------ --Sequential Input- --Random- > Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP > debian 2G 476 96 35129 9 12524 3 1417 99 124716 12 445.9 9 > Latency 19798us 420ms 721ms 14122us 9394us 131ms > Version 1.96 ------Sequential Create------ --------Random Create-------- > debian -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > 16 1552 8 +++++ +++ 1705 11 1675 9 +++++ +++ 1346 8 > Latency 104ms 291us 48604us 109ms 45us 227ms > > It seems that latency is big in whole results, best is for XFS. It is tempting me to think that there are some RAID 5 issues here. It's really strange that block writing for SCSI server disks in RAID5 is no more than 30MB/sec(XFS). I guess I should consider XFS file system or different RAID configuration. May be someone can comment this strange benchmark result? Will very appreciate that. > > With regards, Andrey. > > 23.09.2011 11:31, Janne Pikkarainen пишет: >> Hello, >> >> On 09/23/2011 08:51 AM, Andrey wrote: >>> Hello, >>> >>> I have a production mail server with maildir++ structure and about >>> 250GB (~10 millions) of files on the ext3 partition on RAID5. It's >>> mounted with noatime option. These mail server is responsible to local >>> delivery and storing mail messages. >>> >>> System has Debian Squeeze installed and Exim as MDA + Dovecot as >>> IMAP+POP3 server. >>> >>> Bonnie results are terrible. Sequential output for Block and Rewrite >>> are 10722ms and 9232ms. So if there is a 1000 messages in the mail >>> queue load is extremely high, delivery time is very big and server can >>> hang. I did not see such problems with UFS on FreeBSD server. >>> >>> As I understand ext3 file system is really bad for such configurations >>> with Maildir++ (many smaill files)? Is there a way to decrease disk >>> latency on ext3 or speed up it? >>> >>> With regards, Andrey >>> >>> ___ >> > > _______________________________________________ > Ext3-users mailing list > Ext3-users@xxxxxxxxxx > https://www.redhat.com/mailman/listinfo/ext3-users _______________________________________________ Ext3-users mailing list Ext3-users@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/ext3-users
