Re: Slow disks.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, Dec 22, 2010 at 5:43 AM, Rogier Wolff <R.E.Wolff@xxxxxxxxxxxx> wrote:
>
> Unquoted text below is from either me or from my friend.
>
>
> Someone suggested we try an older kernel as if kernel 2.6.32 would not
> have this problem. We do NOT think it suddenly started with a certain
> kernel version. I was just hoping to have you kernel-guys help with
> prodding the kernel into revealing which component was screwing things
> up....
>
>
> On Mon, Dec 20, 2010 at 01:32:44PM -0500, Greg Freemyer wrote:
>> On Mon, Dec 20, 2010 at 1:06 PM, Bruno Prémont
>> <bonbons@xxxxxxxxxxxxxxxxx> wrote:
>> > Hi,
>> >
>> > [ccing linux-ide]
>> >
>> > Please provide the part of kernel log showing initialization of your
>> > disk controller(s) as well as detection of all the discs.
>
>
> sata_sil 0000:03:01.0: version 2.4
> sata_sil 0000:03:01.0: PCI INT A -> GSI 24 (level, low) -> IRQ 24
> sata_sil 0000:03:01.0: Applying R_ERR on DMA activate FIS errata fix
> scsi2 : sata_sil
> scsi3 : sata_sil
> scsi4 : sata_sil
> scsi5 : sata_sil
> ata3: SATA max UDMA/100 mmio m1024@0xed200000 tf 0xed200080 irq 24
> ata4: SATA max UDMA/100 mmio m1024@0xed200000 tf 0xed2000c0 irq 24
> ata5: SATA max UDMA/100 mmio m1024@0xed200000 tf 0xed200280 irq 24
> ata6: SATA max UDMA/100 mmio m1024@0xed200000 tf 0xed2002c0 irq 24
> ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
> ata3.00: ATA-8: WDC WD10EARS-00Y5B1, 80.00A80, max UDMA/133
> ata3.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 0/32)
> ata3.00: configured for UDMA/100
> scsi 2:0:0:0: Direct-Access     ATA      WDC WD10EARS-00Y 80.0 PQ: 0 ANSI: 5
> usb 2-2: new low speed USB device using uhci_hcd and address 2
> ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
> ata4.00: ATA-7: SAMSUNG HD103SI, 1AG01118, max UDMA7
> ata4.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 0/32)
> ata4.00: configured for UDMA/100
> scsi 3:0:0:0: Direct-Access     ATA      SAMSUNG HD103SI  1AG0 PQ: 0 ANSI: 5
> ata5: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
> ata5.00: ATA-8: WDC WD10EARS-00Y5B1, 80.00A80, max UDMA/133
> ata5.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 0/32)
> ata5.00: configured for UDMA/100
> scsi 4:0:0:0: Direct-Access     ATA      WDC WD10EARS-00Y 80.0 PQ: 0 ANSI: 5
> ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
> ata6.00: ATA-8: WDC WD10EARS-00Y5B1, 80.00A80, max UDMA/133
> ata6.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 0/32)
> ata6.00: configured for UDMA/100
> scsi 5:0:0:0: Direct-Access     ATA      WDC WD10EARS-00Y 80.0 PQ: 0 ANSI: 5
> sd 2:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
> sd 2:0:0:0: [sda] Write Protect is off
> sd 2:0:0:0: [sda] Mode Sense: 00 3a 00 00
> sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't
> support DPO or FUA
> sd 3:0:0:0: [sdb] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
> sd 3:0:0:0: [sdb] Write Protect is off
> sd 3:0:0:0: [sdb] Mode Sense: 00 3a 00 00
> sd 3:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't
> support DPO or FUA
> sd 4:0:0:0: [sdc] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
> sd 4:0:0:0: [sdc] Write Protect is off
> sd 4:0:0:0: [sdc] Mode Sense: 00 3a 00 00
> sd 4:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't
> support DPO or FUA
> sd 5:0:0:0: [sdd] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
> sd 5:0:0:0: [sdd] Write Protect is off
> sd 5:0:0:0: [sdd] Mode Sense: 00 3a 00 00
> sd 5:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't
> support DPO or FUA
> sd 5:0:0:0: [sdd] Write Protect is off
> sd 5:0:0:0: [sdd] Mode Sense: 00 3a 00 00
> sd 5:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't
> support DPO or FUA
>  sdb: sdb1 sdb2 sdb3 sdb4
> sd 3:0:0:0: [sdb] Attached SCSI disk
>  sda: sda1 sda2 sda3 sda4
> sd 2:0:0:0: [sda] Attached SCSI disk
>  sdc: sdc1 sdc2 sdc3 sdc4
> sd 4:0:0:0: [sdc] Attached SCSI disk
>  sdd: sdd1 sdd2 sdd3 sdd4
> sd 5:0:0:0: [sdd] Attached SCSI disk
>
>
>
>> > Verbose lspci output for the disc controller and $(smartctl -i -A $disk)
>> > output might be useful as well.
>
>
> 03:01.0 Mass storage controller: Silicon Image, Inc. SiI 3114
> [SATALink/SATARaid] Serial ATA Controller (rev 02)
>        Subsystem: Silicon Image, Inc. SiI 3114 SATALink Controller
>        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
> Stepping- SERR+ FastB2B- DisINTx-
>        Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
> <TAbort- <MAbort- >SERR- <PERR- INTx-
>        Latency: 32, Cache Line Size: 32 bytes
>        Interrupt: pin A routed to IRQ 24
>        Region 0: I/O ports at 4020 [size=8]
>        Region 1: I/O ports at 4014 [size=4]
>        Region 2: I/O ports at 4018 [size=8]
>        Region 3: I/O ports at 4010 [size=4]
>        Region 4: I/O ports at 4000 [size=16]
>        Region 5: Memory at ed200000 (32-bit, non-prefetchable) [size=1K]
>        [virtual] Expansion ROM at e8000000 [disabled] [size=512K]
>        Capabilities: [60] Power Management version 2
>                Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA
>                PME(D0-,D1-,D2-,D3hot-,D3cold-)
>                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=2 PME-
>        Kernel driver in use: sata_sil
>        Kernel modules: sata_sil
>
>
> But also tried onboard card:
>
> 00:1f.1 IDE interface: Intel Corporation 82801G (ICH7 Family) IDE
> Controller (rev 01) (prog-if 8a [Master SecP PriP])
>        Subsystem: Super Micro Computer Inc Device 7980
>        Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
> Stepping- SERR- FastB2B- DisINTx-
>        Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
> <TAbort- <MAbort- >SERR- <PERR- INTx-
>        Latency: 0
>        Interrupt: pin A routed to IRQ 18
>        Region 0: I/O ports at 01f0 [size=8]
>        Region 1: I/O ports at 03f4 [size=1]
>        Region 2: I/O ports at 0170 [size=8]
>        Region 3: I/O ports at 0374 [size=1]
>        Region 4: I/O ports at 30a0 [size=16]
>        Kernel driver in use: ata_piix
>        Kernel modules: ata_generic, pata_acpi, ata_piix, ide-pci-generic,
>        piix
>
> smartctl output:
>        Kernel modules: ata_generic, pata_acpi, ata_piix, ide-pci-generic,
>        piix
>
> smartctl output:
>
> smartctl 5.40 2010-10-16 r3189 [x86_64-unknown-linux-gnu] (local build)
> Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net
>
> === START OF INFORMATION SECTION ===
> Model Family:     Western Digital Caviar Green (Adv. Format) family
> Device Model:     WDC WD10EARS-00Y5B1
> Serial Number:    WD-WCAV55759454
> Firmware Version: 80.00A80
> User Capacity:    1,000,204,886,016 bytes
> Device is:        In smartctl database [for details use: -P show]
> ATA Version is:   8
> ATA Standard is:  Exact ATA specification draft version not indicated
> Local Time is:    Tue Dec 21 20:06:00 2010 CET
> SMART support is: Available - device has SMART capability.
> SMART support is: Enabled
>
> === START OF READ SMART DATA SECTION ===
> SMART Attributes Data Structure revision number: 16
> Vendor Specific SMART Attributes with Thresholds:
> ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE
> UPDATED  WHEN_FAILED RAW_VALUE
>  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail
> Always       -       0
>  3 Spin_Up_Time            0x0027   132   119   021    Pre-fail
> Always       -       6391
>  4 Start_Stop_Count        0x0032   100   100   000    Old_age
> Always       -       56
>  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail
> Always       -       0
>  7 Seek_Error_Rate         0x002e   200   200   000    Old_age
> Always       -       0
>  9 Power_On_Hours          0x0032   091   091   000    Old_age
> Always       -       7189
>  10 Spin_Retry_Count        0x0032   100   253   000    Old_age
> Always       -       0
>  11 Calibration_Retry_Count 0x0032   100   253   000    Old_age
> Always       -       0
>  12 Power_Cycle_Count       0x0032   100   100   000    Old_age
> Always       -       54
> 192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always
>      -       39
> 193 Load_Cycle_Count        0x0032   164   164   000    Old_age   Always
>      -       109955
> 194 Temperature_Celsius     0x0022   109   107   000    Old_age   Always
>      -       38
> 196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always
>      -       0
> 197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always
>      -       0
> 198 Offline_Uncorrectable   0x0030   200   200   000    Old_age
> Offline      -       0
> 199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always
>      -       0
> 200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age
> Offline      -       0
>      -       0
> 200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age
> Offline      -       0
>
> smartctl 5.40 2010-10-16 r3189 [x86_64-unknown-linux-gnu] (local build)
> Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net
>
> === START OF INFORMATION SECTION ===
> Model Family:     Western Digital Caviar Green (Adv. Format) family
> Device Model:     WDC WD10EARS-00Y5B1
> Serial Number:    WD-WCAV55759454
> Firmware Version: 80.00A80
> User Capacity:    1,000,204,886,016 bytes
> Device is:        In smartctl database [for details use: -P show]
> ATA Version is:   8
> ATA Standard is:  Exact ATA specification draft version not indicated
> Local Time is:    Tue Dec 21 20:06:00 2010 CET
> SMART support is: Available - device has SMART capability.
> SMART support is: Enabled
>
> === START OF READ SMART DATA SECTION ===
> SMART Attributes Data Structure revision number: 16
> Vendor Specific SMART Attributes with Thresholds:
> ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE
> UPDATED  WHEN_FAILED RAW_VALUE
>  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail
> Always       -       0
>  3 Spin_Up_Time            0x0027   132   119   021    Pre-fail
> Always       -       6391
>  4 Start_Stop_Count        0x0032   100   100   000    Old_age
> Always       -       56
>  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail
> Always       -       0
>  7 Seek_Error_Rate         0x002e   200   200   000    Old_age
> Always       -       0
>  9 Power_On_Hours          0x0032   091   091   000    Old_age
> Always       -       7189
>  10 Spin_Retry_Count        0x0032   100   253   000    Old_age
> Always       -       0
>  11 Calibration_Retry_Count 0x0032   100   253   000    Old_age
> Always       -       0
>  12 Power_Cycle_Count       0x0032   100   100   000    Old_age
> Always       -       54
> 192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always
>      -       39
> 193 Load_Cycle_Count        0x0032   164   164   000    Old_age   Always
>      -       109955
> 194 Temperature_Celsius     0x0022   109   107   000    Old_age   Always
>      -       38
> 196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always
>      -       0
> 197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always
>      -       0
> 198 Offline_Uncorrectable   0x0030   200   200   000    Old_age
> Offline      -       0
> 199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always
>      -       0
> 200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age
> Offline      -       0
>
> smartctl 5.40 2010-10-16 r3189 [x86_64-unknown-linux-gnu] (local build)
> 200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age
> Offline      -       0
>
>
> The others are very similar....
>
>
>> >
>> > Did you try the individual discs on a completely different system (e.g.
>> > plain desktop system) and what revision of SATA are both components
>> > supporting?
>
> Yes I did. The disks were installed in a MSI/Core2DUO based desktop
> system. No problems at all. Transfer rates up to 200MB/s.
>
>
> The SIL 3114 chip is 1.5Gbps SATA. .
>
>
> Searching for information on the WD drives I stumbled across:
>
> http://community.wdc.com/t5/Other-Internal-Drives/1-TB-WD10EARS-desynch-issues-in-RAID/m-p/11559
>
> Where it seems that WD simply says not to use these drives in a RAID.
> I have experience with "Raid Edition" drives: They go bad at a MUCH
> too high rate. If we can't use the non-raid for a RAID application, then
> there is just ONE possible option: STAY AWAY FROM WESTERN DIGITAL:
>
> Western digital claims it has the right to mess things up if you put a
> non-raid drive in a raid configuration. Well fine. Then they can also
> mess things up in normal situations because when Linux does software
> raid there isn't any difference from RAID accesses.
>
> (if you click through and read their entry in the knowledge base,
> you'd notice that it should be more or less the other way
> around. Linux will drop the RAID-enabled drive from the RAID within
> seven seconds and reporting error on a sector, whereas the desktop
> drive would remain operational until Linux times out (30 seconds?))
>
>
>
> More hardware info:
>
> System: Supermicro PDSMi, 4xDDR2 1GB, disks and controllers as above.
> Current kernel version: 2.6.36.2
> Problem was also present in kernel 2.6.33 (sorry cannot downgrade again.
> This is a production system...)
>
> uname -a:
> Linux jcz.nl 2.6.36-ARCH #1 SMP PREEMPT Fri Dec 10 20:32:37 CET 2010
> x86_64 Intel(R) Pentium(R) D CPU 3.20GHz GenuineIntel GNU/Linux
>
> Disklayout:
>
> major minor  #blocks  name
>
>   8        0  976762584 sda
>   8        1     240943 sda1
>   8        2   19535040 sda2
>   8        3    1951897 sda3
>   8        4  955032120 sda4
>   8       16  976762584 sdb
>   8       17     240943 sdb1
>   8       18   19535040 sdb2
>   8       19    1951897 sdb3
>   8       20  955032120 sdb4
>   8       32  976762584 sdc
>   8       33     240943 sdc1
>   8       34   19535040 sdc2
>   8       35    1951897 sdc3
>   8       36  955032120 sdc4
>   8       48  976762584 sdd
>   8       49     240943 sdd1
>   8       50   19535040 sdd2
>   8       51    1951897 sdd3
>   8       52  955032120 sdd4
>   9      127     240832 md127
>   9        1   39067648 md1
>   9      126 1910063104 md126
>   9      125    3903488 md125
>
> MDstat:
>
> Personalities : [raid1] [raid6] [raid5] [raid4]
> md125 : active raid5 sdd3[5](S) sdb3[4] sda3[0] sdc3[3]
>      3903488 blocks super 1.1 level 5, 512k chunk, algorithm 2 [3/3] [UUU]
>
> md126 : active raid5 sda4[0] sdd4[3] sdc4[5](S) sdb4[4]
>      1910063104 blocks super 1.1 level 5, 512k chunk, algorithm 2
> [3/3] [UUU]
>
> md1 : active raid5 sda2[0] sdd2[3](S) sdb2[1] sdc2[4]
>      39067648 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3]
> [3/3] [UUU]
>
> md1 : active raid5 sda2[0] sdd2[3](S) sdb2[1] sdc2[4]
>      39067648 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3]
> [UUU]
>
> md127 : active raid1 sdd1[3](S) sda1[0] sdb1[1] sdc1[2]
>      240832 blocks [3/3] [UUU]
>
> unused devices: <none>
> rootfs / rootfs rw 0 0
> proc /proc proc rw,relatime 0 0
> sys /sys sysfs rw,relatime 0 0
> udev /dev devtmpfs
> rw,nosuid,relatime,size=10240k,nr_inodes=506317,mode=755 0 0
> /dev/disk/by-label/rootfs / ext4
> rw,relatime,barrier=1,stripe=256,data=ordered 0 0
> devpts /dev/pts devpts rw,relatime,mode=600,ptmxmode=000 0 0
> shm /dev/shm tmpfs rw,nosuid,nodev,relatime 0 0
> /dev/md127 /boot ext3
> rw,relatime,errors=continue,barrier=0,data=writeback 0 0
> /dev/md126 /data ext4 rw,relatime,barrier=1,data=ordered 0 0
>
>
> Because of the severity of the problems (which remain after trying
> another sata card), I have already bought a new Supermicro server. Let's
> hope that helps.


The load_cycle_counts are very high and that means your drive heads
are parking all the time.  Possibly multiple times a minute.

I don't know if its your problem, but I'd say something is wrong and
I've seen excessive head parking cause disk write failures in Windows.
 In linux I think it just wears out your drive way prematurely.  And
of course and i/o's are delayed if the heads are parked when the
commands hit the drive.

There is a linux package specifically targeting drives that have this
issue.  Hopefully it can at least keep your heads from parking
continuously.  storage-fixup.

1) Be sure you have the userspace package storage-fixup installed.

2) Look in /etc/storage-fixup.conf and see if your drives are in the list.

If not, try to work with the storage-fixup maintainer (Tejun Heo?) to
get your drives added.

And while testing, watch Load_cycle_count and ensure it is not
increasing too fast.  ie. Several times an hour is fine.  Several
times per minute is too much.

Greg
--
To unsubscribe from this list: send the line "unsubscribe linux-ide" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Filesystems]     [Linux SCSI]     [Linux RAID]     [Git]     [Kernel Newbies]     [Linux Newbie]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Samba]     [Device Mapper]

  Powered by Linux