Re: Load spikes when new email arrives

Blake Hudson <blake@xxxxxxxx> · Fri, 25 Jan 2013 08:26:34 -0600



    francis picabia wrote the following on
      1/25/2013 7:55 AM:

    
      On Thu, Jan 24, 2013 at 12:22 PM, Blake
        Hudson <blake@xxxxxxxx>
        wrote:

        
          There are a couple suggestions I'd like to put forth. First,
          improper

          partition alignment is generally masked by the controller
          cache. I

          strongly encourage you to check that your RAID array is making
          use of

          this cache by enabling the WriteBack caching option on this
          array,

          especially if your PERC card has a BBU (I think this was
          optional on

          perc 5). You can install the MegaCLI tool from LSI to verify
          this (can

          also be checked from OpenManage or reboot into the controller
          BIOS).

          
          Thanks for this tip.  It put me on to what is wrong.

          
          Jan 18 07:25:39 myserv Server Administrator: Storage Service
          EventID: 2335  Controller event log: BBU disabled; changing WB
          virtual disks to WT:  Controller 0 (PERC 5/i Integrated) 

          
          Bingo!  We had write back all along, and the performance
          tanked when it

          fell back to write through.  I was wondering why my policy
          change attempts

          were flipping back when I tried testing WB this morning!

          
          This explains everything we've been seeing.  Wow.  Gotta call
          Dell.

          
          Thanks everyone for the assistance.  I didn't think a battery
          which shows OK

          status in omreport could wound us!

          
    The PERC cards will disable write-back caching while the BBU is
    charging/exercising. However, within a few hours the BBU should
    return to normal status. In rare instances, people on the Dell
    mailing list have reported that their caching status never returns
    to write-back - even after attempting to force write-back caching on
    the array. Attempts and power cycling or firmware flashing are
    tried, but seem to be futile in most cases. Often, replacement of
    the card is necessary. I'm unsure if it's the battery, the card, or
    some software setting, but I would definitely follow up with Dell.

    
    On the next server (or array) you configure, I would attempt to
    align your partitions as you've investigated. Sector 2048 seems to
    be a good starting position for most RAID levels. I have no
    conclusive evidence that a different file system or alignment
    improves my performance, because I've never done a fair side by side
    test with controlled inputs. However, we use ext4 and do align our
    partitions using RAID10 on 15k SAS drives for all our Cyrus
    installs. I have found some issues with the newer systems that I
    attribute to the move from ext3 to ext4 which can result in MySQL
    replication problems on power loss/freeze, but these issues are vary
    rare and usually easy to recover from in our environment. I also
    notice that new systems always perform better than the old systems,
    even with identical hardware - I've often attributed this to
    fragmentation.

    
    --Blake

  
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus