Hello,
Does anyone have experience using a 3ware 9650SE series raid controller on CentOS 6.0?
I am getting very sporadic throughput with moderately sized files (0.5-2GB) on ext3. I have tried most of the mount time tuning options:
* noatime
* trying different journal types
* setting commit=120 - helped a little
Even after these optimizations it doesn't seem like the raid array is working as it should. After a few 1GB writes (dd from /dev/zero to the raid) kjournald runs for an hour and later writes are really slow. Not only that, using tw_cli is very slow ... whereas tw_cli is superfast if kjournald is not churning away.
So it goes something like this
* write one 1GB file (486 MB/s)
* writes another (223 MB/s)
* any writes before the 120s commit kicks in, is ~200MB/s
* commit kicks in and kjournald starts churning
* writes are all over the map - 6-85MB/s
Maybe this is just the way it is, but it did not seem to be the case this same hardware was running Fedora (Core) 9 and I have a similar machine where this does not seem to be the case (I can't experiment on it at the moment).
The one thing I do not know, since I did not create the RAID or ext3 filesystem is whether the stride and stripe-width were properly selected to match the 64k chunk size of the raid array. I don't know how to tell from tune2fs ... output below ... its either not there or by another name.
Any help or suggestions are appreciated.
Austin
Useful Information
==============
============== /proc/mounts =====================
/dev/sda1 /tonga_raid ext3 rw,noatime,errors=continue,nouser_xattr,noacl,commit=120,barrier=1,data="" 0 0
========= tw_cli /c2 show diag =====================
### Time Stamp: 12:34:18 01-Sep-2011
### Host Name: tonga
### Host Architecture: x86_64 (64 bit)
### OS Version: Linux 2.6.32-71.29.1.el6.x86_64
### Model: 9650SE-8LPML
### Serial #: L326025A8221043
### Controller ID: 2
### CLI Version: 2.00.11.016
### API Version: 2.08.00.017
### Driver Version: 2.26.02.014RH
### Firmware Version: FE9X 3.08.00.016
### BIOS Version: BE9X 3.08.00.004
### Available Memory: 224MB
==========================================================================
Diagnostic Information on Controller //.../c2/...
--------------------------------------------------------------------------
Event Trigger and Log Information:
Triggered Event(s) =
ctlreset (controller soft reset)
fwassert (firmware assert)
driveerr (drive error)
Diagnostic log save mode = -
Parameter table does not exist
========== tune2fs -l /dev/sda1 =================
tune2fs 1.41.12 (17-May-2010)
Filesystem volume name: /data
Last mounted on: <not available>
Filesystem UUID: 3c5f6dbb-d5dc-4f85-bc70-9b761c89c86e
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file
Filesystem flags: signed_directory_hash
Default mount options: user_xattr acl
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 427245568
Block count: 1708965879
Reserved block count: 85448293
Free blocks: 809305423
Free inodes: 426287552
First block: 0
Block size: 4096
Fragment size: 4096
Reserved GDT blocks: 616
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 8192
Inode blocks per group: 512
Filesystem created: Tue Sep 9 09:57:44 2008
Last mount time: Thu Sep 1 12:40:01 2011
Last write time: Thu Sep 1 12:40:01 2011
Mount count: 11
Maximum mount count: -1
Last checked: Tue Aug 30 23:12:49 2011
Check interval: 0 (<none>)
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 256
Journal inode: 8
Default directory hash: tea
Directory Hash Seed: 0b324311-93a9-4c23-bf15-40965792029b
Journal backup: inode blocks
===========================================================
=================== tw_cli info c2 ============================
Unit UnitType Status %RCmpl %V/I/M Stripe Size(GB) Cache AVrfy
------------------------------------------------------------------------------
u0 RAID-5 OK - - 64K 6519.19 OFF OFF
Port Status Unit Size Blocks Serial
---------------------------------------------------------------
p0 OK u0 931.51 GB 1953525168 WD-WCASJ1631953
p1 OK u0 931.51 GB 1953525168 WD-WCASJ1622428
p2 OK u0 931.51 GB 1953525168 WD-WCASJ1639721
p3 OK u0 931.51 GB 1953525168 WD-WCASJ1636054
p4 OK u0 931.51 GB 1953525168 WD-WCASJ1621694
p5 OK u0 931.51 GB 1953525168 WD-WCASJ1636292
p6 OK u0 931.51 GB 1953525168 WD-WCASJ1637586
p7 OK u0 931.51 GB 1953525168 WD-WCASJ1637516
===========================================================
_______________________________________________ CentOS mailing list CentOS@xxxxxxxxxx http://lists.centos.org/mailman/listinfo/centos
