I tried rolling back the kernal and have the same issue. Here is an example of the dstat output when writing with bonnie++ on RAID 1. As soon as the write buffer fills up the wait climbs as it is waiting to write to the disk. The output looks the same on both systems. usr sys idl wai hiq siq| read writ| recv send| in out | int csw 0 0 100 0 0 0| 0 0 | 586B 1682B| 0 0 |1015 110 0 0 100 0 0 0| 0 0 | 64B 412B| 0 0 |1022 96 4 1 96 1 0 0| 40k 0 | 238B 664B| 0 0 |1011 124 43 6 50 1 0 0|4096B 0 | 375B 428B| 0 0 |1026 90 43 7 50 0 0 0| 0 0 | 64B 412B| 0 0 |1005 60 43 8 50 0 0 0| 0 0 | 64B 412B| 0 0 |1023 91 43 6 50 0 0 0|4096B 0 | 64B 412B| 0 0 |1006 77 40 14 44 0 0 1| 0 62M| 158B 396B| 0 0 |1194 160 40 10 0 46 0 3| 0 145M| 158B 522B| 0 0 |1297 128 38 8 0 52 0 3|4096B 127M| 64B 412B| 0 0 |1276 147 41 9 1 48 0 3|4096B 120M| 174B 366B| 0 0 |1252 129 43 8 3 45 0 0| 0 16k| 158B 412B| 0 0 |1012 113 40 16 6 36 0 1|4096B 41M| 64B 318B| 0 0 |1142 188 42 11 0 45 0 2| 0 130M| 64B 675B| 0 0 |1327 276 43 9 0 44 0 4| 0 138M| 64B 412B| 0 0 |1280 130 34 9 16 38 0 2|4096B 107M| 64B 412B| 0 0 |1229 120 44 9 4 44 0 0| 0 8192B| 64B 412B| 0 0 |1024 175 41 17 0 41 0 0| 0 33M| 192B 366B| 0 0 |1096 193 37 9 1 51 0 3|4096B 126M| 64B 428B| 0 0 |1288 173 44 8 0 44 0 3| 0 142M| 64B 412B| 0 0 |1289 164 Here is the dstat output with the same bonnie command on the RAID 5 volume. This machine has two VMware guests running so the system wasn't idle when grabbing the output. usr sys idl wai hiq siq| read writ| recv send| in out | int csw 1 14 83 2 0 0| 120k 537k| 0 0 | 0 0.2 |3084 10k 18 11 68 1 0 0| 40k 8192B|9180B 1192B| 0 0 |3074 12k 36 18 43 2 0 0| 272k 24M| 123k 4865B| 0 0 |3858 14k 30 26 42 1 1 2| 808k 67M| 21k 1418B| 0 0 |5253 18k 39 19 43 0 0 0|4096B 0 |4600B 692B| 0 0 |3079 11k 36 19 29 17 0 0| 116k 1104k|3024B 2464B| 0 0 |3221 10k 40 17 14 30 0 1| 136k 400k| 86k 5828B| 0 0 |3189 10k 37 21 17 23 0 0| 380k 35M| 30k 1708B| 0 0 |4223 16k 30 29 37 2 2 2|1160k 115M| 390B 550B| 0 0 |6647 24k 31 29 37 1 1 2|1112k 127M| 664B 314B| 0 0 |6745 24k 33 26 28 11 0 1| 728k 71M|3074B 526B| 0 0 |4608 16k 37 24 2 37 0 0| 0 16k|1616B 14k| 0 0 |3086 10k 34 21 11 33 1 1| 388k 33M| 26k 1280B| 0 0 |3939 13k 30 32 36 1 1 1|1304k 111M| 60B 420B| 0 0 |5083 19k 31 35 30 2 1 2|1296k 125M| 19k 2051B| 0 0 |5987 20k 38 22 19 22 0 1| 692k 28M|3084B 2480B| 0 0 |3744 11k 41 17 38 3 0 0| 736k 2184k| 120B 298B| 0 0 |3785 11k 34 30 21 14 0 0| 360k 35M| 48k 2862B| 0 0 |4178 12k 37 26 35 1 1 1|1056k 136M| 13k 1394B| 0 0 |4331 11k 34 28 33 2 0 1|1228k 134M| 30k 1658B| 0 0 |4132 11k 36 21 28 14 0 0| 332k 23M| 151k 5798B| 0 0 |3368 9166 37 18 18 28 0 0| 16k 88k| 13k 990B| 0 0 |3092 8403 38 23 23 16 1 0| 316k 39M| 30k 1920B| 0 0 |3635 9723 32 33 33 2 0 1|1180k 132M| 295B 404B| 0 0 |3907 9935 31 31 35 2 1 1|1120k 123M| 43k 2424B| 0 0 |4746 14k 32 29 37 2 1 1|1380k 71M|3084B 2440B| 0 0 |5341 19k 37 24 36 1 0 0| 700k 53M| 459B 496B| 0 0 |4402 20k 35 20 29 14 1 1|1808k 61M|4596B 500B| 0 0 |5551 19k 30 30 35 2 1 3|1076k 107M| 246B 620B| 0 0 |6769 24k 36 25 30 7 1 2|1088k 66M| 165k 10k| 0 0 |5093 17k On Fri, Mar 13, 2009 at 5:17 AM, Alain Williams <addw@xxxxxxxxxxxx> wrote: > On Thu, Mar 12, 2009 at 10:21:28PM -0500, Ryan Wagoner wrote: >> I'm glad I'm not the only one experiencing the issue. Luckily the >> issues on both my systems aren't as bad. I don't have any errors >> showing in /var/log/messages on either system. I've been trying to >> track down this issue for about a year now. I just recently my the >> connection with RAID 1 and mdadm when copying data on the second >> system. > > Did you have the problem straight from install, or perhaps when a new > kernel started being used ? > > My system worked well for some months, there was no kernel update and > it started to go wrong a couple of weeks ago. I also see errors > when I run 'badblocks' -- which makes it smell of a hardware issue, > but the disks were tested, on return, by the hardware supplier and > they did not find any problem with them. > > Given that you are not seeing anything in /var/log/messages make me > think that I do have some other problem -- perhaps in addition to > what you have. > > -- > Alain Williams > Linux/GNU Consultant - Mail systems, Web sites, Networking, Programmer, IT Lecturer. > +44 (0) 787 668 0256 http://www.phcomp.co.uk/ > Parliament Hill Computers Ltd. Registration Information: http://www.phcomp.co.uk/contact.php > Past chairman of UKUUG: http://www.ukuug.org/ > #include <std_disclaimer.h> > -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
