Following the advice in https://raid.wiki.kernel.org/index.php/RAID_Recovery I am writing to ask for help fixing a busted Linux RAID array. On an older computer running Linux 2.6.26.6-49.fc8 i686 I had what I thought was a RAID-6 array: 4 x 3TB drives. One of the drives was showing a lot of bad blocks, so I decided to proactively replace it. I marked the drive as failed (mdadm --manage /dev/md1 --fail /dev/sdb1) and removed it (mdadm --manage /dev/md1 --remove /dev/sdb1), then shutdown the computer and swapped the drive. Alas, when I restarted, one of the remaining drives was not added back to the array, leaving me with just 2 drives (mdadm --detail in exhibit 1 below). Fortunately, it was a RAID-6 array, so two is enough. I added the missing drive back (sudo mdadm --manage /dev/md1 --re-add /dev/sdd1), but it was marked as spare rebuilding rather than active. But it started to recover (see /proc/mdstat in exhibit 2). I added the new drive as well and it was marked as a spare (see exhibit 3 for the output of mdadm --examine). However, during the recovery period one of the two “good” drives threw an error and the array stopped. Looking more carefully at the smartctl output, I am wondered if I replaced one of the good drives by mistake and left the failing one. Anyway. I still had the “bad” drive I’d removed, so I took out the new drive and put the old one back in. I think I forgot to fail and remove the new one, now that I am writing this. When I rebooted, the array didn’t start by itself. From the output of mdadm --examine (exhibit 4) tt seemed that I did have two up to date disks, so I did a force assemble, which seemed to work. $ mdadm --assemble --force /dev/md1 /dev/sd[abcd]1 mdadm: forcing event count in /dev/sdc1(3) from 34914 upto 34922 mdadm: clearing FAULTY flag for device 2 in /dev/md1 for /dev/sdc1 mdadm: /dev/md1 has been started with 2 drives (out of 4) and 1 spare. But it didn’t add sdb1, which was the one I’d failed and removed. So, I re-added it: $ mdadm --re-add /dev/md1 /dev/sdb1 mdadm: re-added /dev/sdb1 That seemed to work, except sdb was also marked as a spare (see exhibit 5). The system started recovering sdd again. But before the recovery finished, there was another disk error. And this time sdc was booted from the array (see exhibit 6, the dmesg output from this morning). Yet sdc is apparently there and with the expected header (see exhibit 7). Without sdc there is nothing to recover from. I tried again to assemble the drive, but got a new error: $ mdadm --stop /dev/md1 mdadm: stopped /dev/md1 $ mdadm --assemble /dev/md1 --force /dev/sd[abcd]1 mdadm: failed to RUN_ARRAY /dev/md1: Input/output error Trying to just re-add sdc doesn’t work either: $ mdadm --re-add /dev/md1 /dev/sdc1 mdadm: add new device failed for /dev/sdc1 as 5: Invalid argument Since I have two or maybe three out of four devices from a RAID-6 array, this should be recoverable, but I am stuck. Reading other peoples’ online recounts, I believe it might be possible to recreate the array rather than reassembling. But following the advice in https://raid.wiki.kernel.org/index.php/RAID_Recovery I am asking for help before going any farther. Exhibits: 1) mdadm detail right after the replacement sudo mdadm --detail /dev/md1 /dev/md1: Version : 01.02.03 Creation Time : Fri Oct 24 18:40:59 2014 Raid Level : raid6 Array Size : 5860532736 (5589.04 GiB 6001.19 GB) Used Dev Size : 5860532736 Raid Devices : 4 Total Devices : 2 Preferred Minor : 1 Persistence : Superblock is persistent Update Time : Sat Mar 14 16:01:50 2020 State : clean, degraded Active Devices : 2 Working Devices : 2 Failed Devices : 0 Spare Devices : 0 Chunk Size : 64K Name : 1 UUID : 27527034:7e381d1e:61ee4796:be882bf6 Events : 34812 Number Major Minor RaidDevice State 0 8 1 0 active sync /dev/sda1 1 0 0 1 removed 2 0 0 2 removed 5 8 33 3 active sync /dev/sdc1 2) /proc/mdstat right after replacement more /proc/mdstat Personalities : [raid1] [raid6] [raid5] [raid4] md0 : active raid1 sdf2[2] sde2[3] 240973976 blocks super 1.2 [2/2] [UU] md1 : active raid6 sdb1[4](S) sdd1[1] sda1[0] sdc1[5] 5860532736 blocks super 1.2 level 6, 64k chunk, algorithm 2 [4/2] [U__U] [>....................] recovery = 0.0% (1818752/2930266368) finish=5080.5min speed=9603K/sec 3) mdadm examine right after the replacement (with one new drive) sudo mdadm --examine /dev/sd[abcd]1 /dev/sda1: Magic : a92b4efc Version : 1.2 Feature Map : 0x0 Array UUID : 27527034:7e381d1e:61ee4796:be882bf6 Name : 1 Creation Time : Fri Oct 24 18:40:59 2014 Raid Level : raid6 Raid Devices : 4 Used Dev Size : 5860532829 (2794.52 GiB 3000.59 GB) Array Size : 11721065472 (5589.04 GiB 6001.19 GB) Used Size : 5860532736 (2794.52 GiB 3000.59 GB) Data Offset : 272 sectors Super Offset : 8 sectors State : clean Device UUID : 75d38a37:9fbf43a3:a45bcf82:266661ab Update Time : Sat Mar 14 16:01:50 2020 Checksum : fe8682fb - correct Events : 34812 Chunk Size : 64K Array Slot : 0 (0, failed, failed, failed, failed, 3) Array State : U__u 4 failed mdadm: No md superblock detected on /dev/sdb1. /dev/sdc1: Magic : a92b4efc Version : 1.2 Feature Map : 0x0 Array UUID : 27527034:7e381d1e:61ee4796:be882bf6 Name : 1 Creation Time : Fri Oct 24 18:40:59 2014 Raid Level : raid6 Raid Devices : 4 Used Dev Size : 5860532741 (2794.52 GiB 3000.59 GB) Array Size : 11721065472 (5589.04 GiB 6001.19 GB) Used Size : 5860532736 (2794.52 GiB 3000.59 GB) Data Offset : 360 sectors Super Offset : 8 sectors State : clean Device UUID : 52d606cb:4fc78e74:0891c48d:d2191050 Update Time : Sat Mar 14 16:01:50 2020 Checksum : 12f1769e - correct Events : 34812 Chunk Size : 64K Array Slot : 5 (0, failed, failed, failed, failed, 3) Array State : u__U 4 failed /dev/sdd1: Magic : a92b4efc Version : 1.2 Feature Map : 0x0 Array UUID : 27527034:7e381d1e:61ee4796:be882bf6 Name : 1 Creation Time : Fri Oct 24 18:40:59 2014 Raid Level : raid6 Raid Devices : 4 Used Dev Size : 5860532829 (2794.52 GiB 3000.59 GB) Array Size : 11721065472 (5589.04 GiB 6001.19 GB) Used Size : 5860532736 (2794.52 GiB 3000.59 GB) Data Offset : 272 sectors Super Offset : 8 sectors State : clean Device UUID : 10e44c8f:e72a746b:ed9d6ce6:55e7deb0 Update Time : Sat Mar 14 15:15:53 2020 Checksum : 8796b72e - correct Events : 34712 Chunk Size : 64K Array Slot : 1 (0, 1, failed, failed, failed, 3) Array State : uU_u 3 failed 4) mdadm --examine after putting back the drive /dev/sda1: Magic : a92b4efc Version : 1.2 Feature Map : 0x0 Array UUID : 27527034:7e381d1e:61ee4796:be882bf6 Name : 1 Creation Time : Fri Oct 24 18:40:59 2014 Raid Level : raid6 Raid Devices : 4 Used Dev Size : 5860532829 (2794.52 GiB 3000.59 GB) Array Size : 11721065472 (5589.04 GiB 6001.19 GB) Used Size : 5860532736 (2794.52 GiB 3000.59 GB) Data Offset : 272 sectors Super Offset : 8 sectors State : clean Device UUID : 75d38a37:9fbf43a3:a45bcf82:266661ab Update Time : Sun Mar 15 06:02:06 2020 Checksum : fe83485b - correct Events : 34922 Chunk Size : 64K Array Slot : 0 (0, empty, failed, failed, empty, failed) Array State : U___ 3 failed /dev/sdb1: Magic : a92b4efc Version : 1.2 Feature Map : 0x0 Array UUID : 27527034:7e381d1e:61ee4796:be882bf6 Name : 1 Creation Time : Fri Oct 24 18:40:59 2014 Raid Level : raid6 Raid Devices : 4 Used Dev Size : 5860532741 (2794.52 GiB 3000.59 GB) Array Size : 11721065472 (5589.04 GiB 6001.19 GB) Used Size : 5860532736 (2794.52 GiB 3000.59 GB) Data Offset : 360 sectors Super Offset : 8 sectors State : clean Device UUID : 1e35de21:8800bfd9:dbe82508:69752cdb Update Time : Sat Mar 14 13:21:22 2020 Checksum : d4789dec - correct Events : 34678 Chunk Size : 64K Array Slot : 4 (0, 1, failed, failed, 2, 3) Array State : uuUu 2 failed /dev/sdc1: Magic : a92b4efc Version : 1.2 Feature Map : 0x0 Array UUID : 27527034:7e381d1e:61ee4796:be882bf6 Name : 1 Creation Time : Fri Oct 24 18:40:59 2014 Raid Level : raid6 Raid Devices : 4 Used Dev Size : 5860532741 (2794.52 GiB 3000.59 GB) Array Size : 11721065472 (5589.04 GiB 6001.19 GB) Used Size : 5860532736 (2794.52 GiB 3000.59 GB) Data Offset : 360 sectors Super Offset : 8 sectors State : clean Device UUID : 52d606cb:4fc78e74:0891c48d:d2191050 Update Time : Sat Mar 14 23:12:37 2020 Checksum : 12f2dbfc - correct Events : 34914 Chunk Size : 64K Array Slot : 5 (0, empty, failed, failed, empty, 3) Array State : u__U 2 failed /dev/sdd1: Magic : a92b4efc Version : 1.2 Feature Map : 0x0 Array UUID : 27527034:7e381d1e:61ee4796:be882bf6 Name : 1 Creation Time : Fri Oct 24 18:40:59 2014 Raid Level : raid6 Raid Devices : 4 Used Dev Size : 5860532829 (2794.52 GiB 3000.59 GB) Array Size : 11721065472 (5589.04 GiB 6001.19 GB) Used Size : 5860532736 (2794.52 GiB 3000.59 GB) Data Offset : 272 sectors Super Offset : 8 sectors State : clean Device UUID : 10e44c8f:e72a746b:ed9d6ce6:55e7deb0 Update Time : Sun Mar 15 06:02:06 2020 Checksum : 879087b8 - correct Events : 34922 Chunk Size : 64K Array Slot : 1 (0, empty, failed, failed, empty, failed) Array State : u___ 3 failed 5) mdadm --describe after re-adding removed drive mdadm -D /dev/md1 /dev/md1: Version : 1.02 Creation Time : Fri Oct 24 18:40:59 2014 Raid Level : raid6 Array Size : 5860532736 (5589.04 GiB 6001.19 GB) Used Dev Size : 2930266368 (2794.52 GiB 3000.59 GB) Raid Devices : 4 Total Devices : 4 Preferred Minor : 1 Persistence : Superblock is persistent Update Time : Sat Mar 28 13:14:31 2020 State : clean, degraded, recovering Active Devices : 2 Working Devices : 4 Failed Devices : 0 Spare Devices : 2 Chunk Size : 64K Rebuild Status : 5% complete Name : 1 UUID : 27527034:7e381d1e:61ee4796:be882bf6 Events : 34932 Number Major Minor RaidDevice State 0 8 1 0 active sync /dev/sda1 1 8 49 1 spare rebuilding /dev/sdd1 2 0 0 2 removed 5 8 33 3 active sync /dev/sdc1 4 8 17 - spare /dev/sdb1 6) dmesg | grep “md" md: raid1 personality registered for level 1 md: raid6 personality registered for level 6 md: raid5 personality registered for level 5 md: raid4 personality registered for level 4 pata_amd 0000:00:08.0: version 0.3.10 scsi6 : pata_amd scsi7 : pata_amd ata7: PATA max UDMA/133 cmd 0x1f0 ctl 0x3f6 bmdma 0xf000 irq 14 ata8: PATA max UDMA/133 cmd 0x170 ctl 0x376 bmdma 0xf008 irq 15 ata9: PATA max UDMA/100 cmd 0xbc00 ctl 0xb800 bmdma 0xac00 irq 16 ata10: PATA max UDMA/100 cmd 0xb400 ctl 0xb000 bmdma 0xac08 irq 16 md: md1 stopped. md: md0 stopped. md: bind<sde2> md: bind<sdf2> raid1: raid set md0 active with 2 out of 2 mirrors md: md1 stopped. md: bind<sdd1> md: bind<sdc1> md: bind<sdb1> md: bind<sda1> md: kicking non-fresh sdc1 from array! md: unbind<sdc1> md: export_rdev(sdc1) raid5: not enough operational devices for md1 (3/4 failed) raid5: failed to run raid set md1 md: pers->run() failed ... md: md1 stopped. md: unbind<sda1> md: export_rdev(sda1) md: unbind<sdb1> md: export_rdev(sdb1) md: unbind<sdd1> md: export_rdev(sdd1) md: md1 stopped. md: bind<sdd1> md: bind<sdc1> md: bind<sdb1> md: bind<sda1> md: kicking non-fresh sdc1 from array! md: unbind<sdc1> md: export_rdev(sdc1) raid5: not enough operational devices for md1 (3/4 failed) raid5: failed to run raid set md1 md: pers->run() failed ... md: md1 stopped. md: unbind<sda1> md: export_rdev(sda1) md: unbind<sdb1> md: export_rdev(sdb1) md: unbind<sdd1> md: export_rdev(sdd1) md: md1 stopped. md: bind<sdd1> md: bind<sdc1> md: bind<sdb1> md: bind<sda1> md: kicking non-fresh sdc1 from array! md: unbind<sdc1> md: export_rdev(sdc1) raid5: not enough operational devices for md1 (3/4 failed) raid5: failed to run raid set md1 md: pers->run() failed ... md1: ADD_NEW_DISK not supported md1: ADD_NEW_DISK not supported md1: ADD_NEW_DISK not supported md1: ADD_NEW_DISK not supported 7) mdadm --examine after sdc removed from array mdadm --examine /dev/sd[abcd]1 /dev/sda1: Magic : a92b4efc Version : 1.2 Feature Map : 0x0 Array UUID : 27527034:7e381d1e:61ee4796:be882bf6 Name : 1 Creation Time : Fri Oct 24 18:40:59 2014 Raid Level : raid6 Raid Devices : 4 Avail Dev Size : 5860532829 (2794.52 GiB 3000.59 GB) Array Size : 11721065472 (5589.04 GiB 6001.19 GB) Used Dev Size : 5860532736 (2794.52 GiB 3000.59 GB) Data Offset : 272 sectors Super Offset : 8 sectors State : clean Device UUID : 75d38a37:9fbf43a3:a45bcf82:266661ab Update Time : Sat Mar 28 17:24:51 2020 Checksum : fe970bf7 - correct Events : 34946 Chunk Size : 64K Array Slot : 0 (0, 1, failed, failed, empty, failed) Array State : Uu__ 3 failed /dev/sdb1: Magic : a92b4efc Version : 1.2 Feature Map : 0x0 Array UUID : 27527034:7e381d1e:61ee4796:be882bf6 Name : 1 Creation Time : Fri Oct 24 18:40:59 2014 Raid Level : raid6 Raid Devices : 4 Avail Dev Size : 5860532741 (2794.52 GiB 3000.59 GB) Array Size : 11721065472 (5589.04 GiB 6001.19 GB) Used Dev Size : 5860532736 (2794.52 GiB 3000.59 GB) Data Offset : 360 sectors Super Offset : 8 sectors State : clean Device UUID : 1e35de21:8800bfd9:dbe82508:69752cdb Update Time : Sat Mar 28 17:24:51 2020 Checksum : d4874d07 - correct Events : 34946 Chunk Size : 64K Array Slot : 4 (0, 1, failed, failed, empty, failed) Array State : uu__ 3 failed /dev/sdc1: Magic : a92b4efc Version : 1.2 Feature Map : 0x0 Array UUID : 27527034:7e381d1e:61ee4796:be882bf6 Name : 1 Creation Time : Fri Oct 24 18:40:59 2014 Raid Level : raid6 Raid Devices : 4 Avail Dev Size : 5860532741 (2794.52 GiB 3000.59 GB) Array Size : 11721065472 (5589.04 GiB 6001.19 GB) Used Dev Size : 5860532736 (2794.52 GiB 3000.59 GB) Data Offset : 360 sectors Super Offset : 8 sectors State : clean Device UUID : 52d606cb:4fc78e74:0891c48d:d2191050 Update Time : Sat Mar 28 16:21:33 2020 Checksum : 1304f0be - correct Events : 34940 Chunk Size : 64K Array Slot : 5 (0, empty, failed, failed, empty, 3) Array State : u__U 2 failed /dev/sdd1: Magic : a92b4efc Version : 1.2 Feature Map : 0x2 Array UUID : 27527034:7e381d1e:61ee4796:be882bf6 Name : 1 Creation Time : Fri Oct 24 18:40:59 2014 Raid Level : raid6 Raid Devices : 4 Avail Dev Size : 5860532829 (2794.52 GiB 3000.59 GB) Array Size : 11721065472 (5589.04 GiB 6001.19 GB) Used Dev Size : 5860532736 (2794.52 GiB 3000.59 GB) Data Offset : 272 sectors Super Offset : 8 sectors Recovery Offset : 4140298240 sectors State : clean Device UUID : 10e44c8f:e72a746b:ed9d6ce6:55e7deb0 Update Time : Sat Mar 28 17:24:51 2020 Checksum : 7e6c3b57 - correct Events : 34946 Chunk Size : 64K Array Slot : 1 (0, 1, failed, failed, empty, failed) Array State : uU__ 3 failed Other potentially useful info: $ more /proc/mdstat Personalities : [raid1] [raid6] [raid5] [raid4] md0 : active raid1 sdf2[2] sde2[3] 240973976 blocks super 1.2 [2/2] [UU] md1 : inactive sda1[0] sdb1[4](S) sdd1[1] 8790799104 blocks super 1.2 unused devices: <none> $ ls -F /sys/block/md1/md array_state component_size dev-sdb1/ layout metadata_version raid_disks resync_start chunk_size dev-sda1/ dev-sdd1/ level new_dev reshape_position safe_mode_delay $ more /sys/block/md1/md/dev-sd*/state :::::::::::::: /sys/block/md1/md/dev-sda1/state :::::::::::::: in_sync :::::::::::::: /sys/block/md1/md/dev-sdb1/state :::::::::::::: spare :::::::::::::: /sys/block/md1/md/dev-sdd1/state :::::::::::::: spare $ foreach i ( /dev/sd[abcd] ) foreach? /usr/sbin/smartctl -a $i foreach? end smartctl version 5.38 [i386-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen Home page is http://smartmontools.sourceforge.net/ === START OF INFORMATION SECTION === Device Model: ST3000DM001-1CH166 Serial Number: Z1F50Y21 Firmware Version: CC29 User Capacity: 3,000,592,982,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 9 ATA Standard is: Not recognized. Minor revision code: 0x1f Local Time is: Sun Mar 29 13:40:29 2020 EDT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x00) Offline data collection activity was never started. Auto Offline Data Collection: Disabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 600) seconds. Offline data collection capabilities: (0x73) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. No Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x3085) SCT Status supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 119 099 006 Pre-fail Always - 230220376 3 Spin_Up_Time 0x0003 093 093 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 73 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 084 060 030 Pre-fail Always - 258429136 9 Power_On_Hours 0x0032 046 046 000 Old_age Always - 48061 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 73 183 Unknown_Attribute 0x0032 100 100 000 Old_age Always - 0 184 Unknown_Attribute 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Unknown_Attribute 0x0032 100 100 000 Old_age Always - 0 189 High_Fly_Writes 0x003a 057 057 000 Old_age Always - 43 190 Airflow_Temperature_Cel 0x0022 070 049 045 Old_age Always - 30 (Lifetime Min/Max 14/31) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 49 193 Load_Cycle_Count 0x0032 049 049 000 Old_age Always - 103886 194 Temperature_Celsius 0x0022 030 051 000 Old_age Always - 30 (0 12 0 0) 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 199767518916884 241 Unknown_Attribute 0x0000 100 253 000 Old_age Offline - 29356695670 242 Unknown_Attribute 0x0000 100 253 000 Old_age Offline - 42508156065 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 45557 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. smartctl version 5.38 [i386-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen Home page is http://smartmontools.sourceforge.net/ === START OF INFORMATION SECTION === Device Model: ST3000DM001-1CH166 Serial Number: W1F42Z54 Firmware Version: CC27 User Capacity: 3,000,592,982,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 9 ATA Standard is: Not recognized. Minor revision code: 0x1f Local Time is: Sun Mar 29 13:40:29 2020 EDT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED See vendor-specific Attribute list for marginal Attributes. General SMART Values: Offline data collection status: (0x00) Offline data collection activity was never started. Auto Offline Data Collection: Disabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 584) seconds. Offline data collection capabilities: (0x73) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. No Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x3085) SCT Status supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 115 094 006 Pre-fail Always - 94431168 3 Spin_Up_Time 0x0003 093 093 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 85 5 Reallocated_Sector_Ct 0x0033 098 098 010 Pre-fail Always - 2040 7 Seek_Error_Rate 0x000f 071 060 030 Pre-fail Always - 99051200434 9 Power_On_Hours 0x0032 041 041 000 Old_age Always - 52413 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 85 183 Unknown_Attribute 0x0032 100 100 000 Old_age Always - 0 184 Unknown_Attribute 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 077 077 000 Old_age Always - 23 188 Unknown_Attribute 0x0032 100 099 000 Old_age Always - 1 189 High_Fly_Writes 0x003a 097 097 000 Old_age Always - 3 190 Airflow_Temperature_Cel 0x0022 068 043 045 Old_age Always In_the_past 32 (0 13 33 15) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 59 193 Load_Cycle_Count 0x0032 039 039 000 Old_age Always - 122262 194 Temperature_Celsius 0x0022 032 057 000 Old_age Always - 32 (0 12 0 0) 197 Current_Pending_Sector 0x0012 078 075 000 Old_age Always - 3632 198 Offline_Uncorrectable 0x0010 078 075 000 Old_age Offline - 3632 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 187586991669585 241 Unknown_Attribute 0x0000 100 253 000 Old_age Offline - 39144200179 242 Unknown_Attribute 0x0000 100 253 000 Old_age Offline - 33632585895 SMART Error Log Version: 1 ATA Error Count: 23 (device log contains only the most recent five errors) CR = Command Register [HEX] FR = Features Register [HEX] SC = Sector Count Register [HEX] SN = Sector Number Register [HEX] CL = Cylinder Low Register [HEX] CH = Cylinder High Register [HEX] DH = Device/Head Register [HEX] DC = Device Command Register [HEX] ER = Error register [HEX] ST = Status register [HEX] Powered_Up_Time is measured from power on, and printed as DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes, SS=sec, and sss=millisec. It "wraps" after 49.710 days. Error 23 occurred at disk power-on lifetime: 51758 hours (2156 days + 14 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 25 00 18 ff ff ff ef 00 47d+05:32:03.461 READ DMA EXT 27 00 00 00 00 00 e0 00 47d+05:32:03.460 READ NATIVE MAX ADDRESS EXT ec 00 00 00 00 00 a0 00 47d+05:32:03.460 IDENTIFY DEVICE ef 03 46 00 00 00 a0 00 47d+05:32:03.460 SET FEATURES [Set transfer mode] 27 00 00 00 00 00 e0 00 47d+05:32:03.459 READ NATIVE MAX ADDRESS EXT Error 22 occurred at disk power-on lifetime: 51758 hours (2156 days + 14 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 25 00 18 ff ff ff ef 00 47d+05:31:59.731 READ DMA EXT 27 00 00 00 00 00 e0 00 47d+05:31:59.730 READ NATIVE MAX ADDRESS EXT ec 00 00 00 00 00 a0 00 47d+05:31:59.730 IDENTIFY DEVICE ef 03 46 00 00 00 a0 00 47d+05:31:59.729 SET FEATURES [Set transfer mode] 27 00 00 00 00 00 e0 00 47d+05:31:59.729 READ NATIVE MAX ADDRESS EXT Error 21 occurred at disk power-on lifetime: 51758 hours (2156 days + 14 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 25 00 18 ff ff ff ef 00 47d+05:31:56.010 READ DMA EXT 27 00 00 00 00 00 e0 00 47d+05:31:56.009 READ NATIVE MAX ADDRESS EXT ec 00 00 00 00 00 a0 00 47d+05:31:56.008 IDENTIFY DEVICE ef 03 46 00 00 00 a0 00 47d+05:31:56.008 SET FEATURES [Set transfer mode] 27 00 00 00 00 00 e0 00 47d+05:31:55.983 READ NATIVE MAX ADDRESS EXT Error 20 occurred at disk power-on lifetime: 51758 hours (2156 days + 14 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 25 00 18 ff ff ff ef 00 47d+05:31:51.976 READ DMA EXT 35 00 08 ff ff ff ef 00 47d+05:31:51.973 WRITE DMA EXT 35 00 08 ff ff ff ef 00 47d+05:31:51.965 WRITE DMA EXT 35 00 08 ff ff ff ef 00 47d+05:31:51.940 WRITE DMA EXT 35 00 08 ff ff ff ef 00 47d+05:31:51.934 WRITE DMA EXT Error 19 occurred at disk power-on lifetime: 51184 hours (2132 days + 16 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 25 00 00 ff ff ff ef 00 23d+08:05:26.236 READ DMA EXT 27 00 00 00 00 00 e0 00 23d+08:05:26.235 READ NATIVE MAX ADDRESS EXT ec 00 00 00 00 00 a0 00 23d+08:05:26.234 IDENTIFY DEVICE ef 03 46 00 00 00 a0 00 23d+08:05:26.234 SET FEATURES [Set transfer mode] 27 00 00 00 00 00 e0 00 23d+08:05:26.234 READ NATIVE MAX ADDRESS EXT SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 49924 - # 2 Extended offline Completed without error 00% 47925 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. smartctl version 5.38 [i386-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen Home page is http://smartmontools.sourceforge.net/ === START OF INFORMATION SECTION === Device Model: ST3000DM001-1CH166 Serial Number: W1F42FW6 Firmware Version: CC27 User Capacity: 3,000,592,982,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 9 ATA Standard is: Not recognized. Minor revision code: 0x1f Local Time is: Sun Mar 29 13:40:33 2020 EDT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED See vendor-specific Attribute list for marginal Attributes. General SMART Values: Offline data collection status: (0x00) Offline data collection activity was never started. Auto Offline Data Collection: Disabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 584) seconds. Offline data collection capabilities: (0x73) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. No Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x3085) SCT Status supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 111 099 006 Pre-fail Always - 230196590 3 Spin_Up_Time 0x0003 093 093 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 90 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 062 059 030 Pre-fail Always - 786257890871 9 Power_On_Hours 0x0032 040 040 000 Old_age Always - 53248 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 89 183 Unknown_Attribute 0x0032 097 097 000 Old_age Always - 3 184 Unknown_Attribute 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 070 070 000 Old_age Always - 30 188 Unknown_Attribute 0x0032 100 099 000 Old_age Always - 4 189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0022 069 044 045 Old_age Always In_the_past 31 (0 11 31 14) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 61 193 Load_Cycle_Count 0x0032 038 038 000 Old_age Always - 125375 194 Temperature_Celsius 0x0022 031 056 000 Old_age Always - 31 (0 12 0 0) 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 120 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 120 199 UDMA_CRC_Error_Count 0x003e 200 198 000 Old_age Always - 19 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 251483220132801 241 Unknown_Attribute 0x0000 100 253 000 Old_age Offline - 35688407725 242 Unknown_Attribute 0x0000 100 253 000 Old_age Offline - 59345299664 SMART Error Log Version: 1 ATA Error Count: 30 (device log contains only the most recent five errors) CR = Command Register [HEX] FR = Features Register [HEX] SC = Sector Count Register [HEX] SN = Sector Number Register [HEX] CL = Cylinder Low Register [HEX] CH = Cylinder High Register [HEX] DH = Device/Head Register [HEX] DC = Device Command Register [HEX] ER = Error register [HEX] ST = Status register [HEX] Powered_Up_Time is measured from power on, and printed as DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes, SS=sec, and sss=millisec. It "wraps" after 49.710 days. Error 30 occurred at disk power-on lifetime: 53242 hours (2218 days + 10 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 25 00 98 ff ff ff ef 00 06:02:38.768 READ DMA EXT 27 00 00 00 00 00 e0 00 06:02:38.767 READ NATIVE MAX ADDRESS EXT ec 00 00 00 00 00 a0 00 06:02:38.767 IDENTIFY DEVICE ef 03 46 00 00 00 a0 00 06:02:38.766 SET FEATURES [Set transfer mode] 27 00 00 00 00 00 e0 00 06:02:38.766 READ NATIVE MAX ADDRESS EXT Error 29 occurred at disk power-on lifetime: 53242 hours (2218 days + 10 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 25 00 98 ff ff ff ef 00 06:02:33.929 READ DMA EXT 27 00 00 00 00 00 e0 00 06:02:33.928 READ NATIVE MAX ADDRESS EXT ec 00 00 00 00 00 a0 00 06:02:33.928 IDENTIFY DEVICE ef 03 46 00 00 00 a0 00 06:02:33.927 SET FEATURES [Set transfer mode] 27 00 00 00 00 00 e0 00 06:02:33.927 READ NATIVE MAX ADDRESS EXT Error 28 occurred at disk power-on lifetime: 53242 hours (2218 days + 10 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 25 00 98 ff ff ff ef 00 06:02:29.073 READ DMA EXT 27 00 00 00 00 00 e0 00 06:02:29.072 READ NATIVE MAX ADDRESS EXT ec 00 00 00 00 00 a0 00 06:02:29.072 IDENTIFY DEVICE ef 03 46 00 00 00 a0 00 06:02:29.072 SET FEATURES [Set transfer mode] 27 00 00 00 00 00 e0 00 06:02:29.072 READ NATIVE MAX ADDRESS EXT Error 27 occurred at disk power-on lifetime: 53242 hours (2218 days + 10 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 25 00 98 ff ff ff ef 00 06:02:24.108 READ DMA EXT 27 00 00 00 00 00 e0 00 06:02:24.107 READ NATIVE MAX ADDRESS EXT ec 00 00 00 00 00 a0 00 06:02:24.107 IDENTIFY DEVICE ef 03 46 00 00 00 a0 00 06:02:24.107 SET FEATURES [Set transfer mode] 27 00 00 00 00 00 e0 00 06:02:24.106 READ NATIVE MAX ADDRESS EXT Error 26 occurred at disk power-on lifetime: 53242 hours (2218 days + 10 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 25 00 98 ff ff ff ef 00 06:02:19.301 READ DMA EXT 27 00 00 00 00 00 e0 00 06:02:19.300 READ NATIVE MAX ADDRESS EXT ec 00 00 00 00 00 a0 00 06:02:19.300 IDENTIFY DEVICE ef 03 46 00 00 00 a0 00 06:02:19.300 SET FEATURES [Set transfer mode] 27 00 00 00 00 00 e0 00 06:02:19.299 READ NATIVE MAX ADDRESS EXT SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 50745 - # 2 Short offline Completed without error 00% 10763 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. smartctl version 5.38 [i386-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen Home page is http://smartmontools.sourceforge.net/ === START OF INFORMATION SECTION === Device Model: ST3000DM001-1CH166 Serial Number: Z1F5083S Firmware Version: CC29 User Capacity: 3,000,592,982,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 9 ATA Standard is: Not recognized. Minor revision code: 0x1f Local Time is: Sun Mar 29 13:40:33 2020 EDT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x00) Offline data collection activity was never started. Auto Offline Data Collection: Disabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 584) seconds. Offline data collection capabilities: (0x73) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. No Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x3085) SCT Status supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 117 099 006 Pre-fail Always - 153985440 3 Spin_Up_Time 0x0003 093 093 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 70 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 084 060 030 Pre-fail Always - 250274127 9 Power_On_Hours 0x0032 046 046 000 Old_age Always - 48024 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 70 183 Unknown_Attribute 0x0032 099 099 000 Old_age Always - 1 184 Unknown_Attribute 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Unknown_Attribute 0x0032 100 099 000 Old_age Always - 2 189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0022 070 052 045 Old_age Always - 30 (Lifetime Min/Max 15/31) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 48 193 Load_Cycle_Count 0x0032 048 048 000 Old_age Always - 104813 194 Temperature_Celsius 0x0022 030 048 000 Old_age Always - 30 (0 13 0 0) 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 116372138930355 241 Unknown_Attribute 0x0000 100 253 000 Old_age Offline - 36713708761 242 Unknown_Attribute 0x0000 100 253 000 Old_age Offline - 25220463092 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 45524 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. Kevin Crowston 206 Meadowbrook Dr. Syracuse, NY 13210 USA Phone: +1 (315) 464-0272 Fax: +1 (815) 550-2155
