Re: slowness with kernel 6.4.10 and software raid

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri Aug18'23 01:39:08PM, Roger Heflin wrote:
> From: Roger Heflin <rogerheflin@xxxxxxxxx>
> Date: Fri, 18 Aug 2023 13:39:08 -0500
> To: Community support for Fedora users <users@xxxxxxxxxxxxxxxxxxxxxxx>
> Reply-To: Community support for Fedora users <users@xxxxxxxxxxxxxxxxxxxxxxx>
> Subject: Re: slowness with kernel 6.4.10 and software raid
>
> The above makes it very clear what is happening.   What kind of disks
> are these?  And did you set the scterc timeout?  You can see it via
> smartctl -l scterc /dev/sda   and then repeat on the other disk.
>
> Setting the timeout as low as you can will improve this situation
> some, but it appears that sda has a number of bad sectors on it.
>
> a full output of "smartclt --xall /dev/sda" would be useful also to
> see how bad it is.
>
> Short answer is you probably need a new device for sda.
>

Thanks!

I tried:

# smartctl -l scterc /dev/sda
 smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.4.10-200.fc38.x86_64] (local build)
 Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

SCT Error Recovery Control command not supported

# smartctl --xall /dev/sda

  smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.4.10-200.fc38.x86_64] (local build)
  Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

  === START OF INFORMATION SECTION ===
  Model Family:     Seagate Barracuda 7200.14 (AF)
  Device Model:     ST2000DM001-1ER164
  Serial Number:    Z4Z5F3LE
  LU WWN Device Id: 5 000c50 091167f04
  Firmware Version: CC27
  User Capacity:    2,000,398,934,016 bytes [2.00 TB]
  Sector Sizes:     512 bytes logical, 4096 bytes physical
  Rotation Rate:    7200 rpm
  Form Factor:      3.5 inches
  Device is:        In smartctl database 7.3/5528
  ATA Version is:   ACS-2, ACS-3 T13/2161-D revision 3b
  SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
  Local Time is:    Fri Aug 18 14:01:28 2023 CDT
  SMART support is: Available - device has SMART capability.
  SMART support is: Enabled
  AAM feature is:   Unavailable
  APM level is:     128 (minimum power consumption without standby)
  Rd look-ahead is: Enabled
  Write cache is:   Enabled
  DSN feature is:   Unavailable
  ATA Security is:  Disabled, NOT FROZEN [SEC1]
  Wt Cache Reorder: Unavailable

  === START OF READ SMART DATA SECTION ===
  SMART overall-health self-assessment test result: PASSED

  General SMART Values:
  Offline data collection status:  (0x00)	Offline data collection activity
  					was never started.
  					Auto Offline Data Collection: Disabled.
  Self-test execution status:      (   0)	The previous self-test routine completed
  					without error or no self-test has ever
  					been run.
  Total time to complete Offline
  data collection: 		(   80) seconds.
  Offline data collection
  capabilities: 			 (0x73) SMART execute Offline immediate.
  					Auto Offline data collection on/off support.
  					Suspend Offline collection upon new
  					command.
  					No Offline surface scan supported.
  					Self-test supported.
  					Conveyance Self-test supported.
  					Selective Self-test supported.
  SMART capabilities:            (0x0003)	Saves SMART data before entering
  					power-saving mode.
  					Supports SMART auto save timer.
  Error logging capability:        (0x01)	Error logging supported.
  					General Purpose Logging supported.
  Short self-test routine
  recommended polling time: 	 (   1) minutes.
  Extended self-test routine
  recommended polling time: 	 ( 212) minutes.
  Conveyance self-test routine
  recommended polling time: 	 (   2) minutes.
  SCT capabilities: 	       (0x1085)	SCT Status supported.

  SMART Attributes Data Structure revision number: 10
  Vendor Specific SMART Attributes with Thresholds:
  ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
    1 Raw_Read_Error_Rate     POSR--   116   092   006    -    106200704
    3 Spin_Up_Time            PO----   096   096   000    -    0
    4 Start_Stop_Count        -O--CK   100   100   020    -    97
    5 Reallocated_Sector_Ct   PO--CK   097   097   010    -    3960
    7 Seek_Error_Rate         POSR--   084   060   030    -    333268033
    9 Power_On_Hours          -O--CK   062   062   000    -    34085
   10 Spin_Retry_Count        PO--C-   100   100   097    -    0
   12 Power_Cycle_Count       -O--CK   100   100   020    -    96
  183 Runtime_Bad_Block       -O--CK   100   100   000    -    0
  184 End-to-End_Error        -O--CK   100   100   099    -    0
  187 Reported_Uncorrect      -O--CK   001   001   000    -    384
  188 Command_Timeout         -O--CK   100   098   000    -    3 71 72
  189 High_Fly_Writes         -O-RCK   065   065   000    -    35
  190 Airflow_Temperature_Cel -O---K   063   055   045    -    37 (Min/Max 37/42)
  191 G-Sense_Error_Rate      -O--CK   100   100   000    -    0
  192 Power-Off_Retract_Count -O--CK   100   100   000    -    19
  193 Load_Cycle_Count        -O--CK   001   001   000    -    294513
  194 Temperature_Celsius     -O---K   037   045   000    -    37 (0 18 0 0 0)
  197 Current_Pending_Sector  -O--C-   094   080   000    -    1064
  198 Offline_Uncorrectable   ----C-   094   080   000    -    1064
  199 UDMA_CRC_Error_Count    -OSRCK   200   200   000    -    0
  240 Head_Flying_Hours       ------   100   253   000    -    31366h+32m+19.252s
  241 Total_LBAs_Written      ------   100   253   000    -    22394883074
  242 Total_LBAs_Read         ------   100   253   000    -    258335971674
                              ||||||_ K auto-keep
                              |||||__ C event count
                              ||||___ R error rate
                              |||____ S speed/performance
                              ||_____ O updated online
                              |______ P prefailure warning

  General Purpose Log Directory Version 1
  SMART           Log Directory Version 1 [multi-sector log support]
  Address    Access  R/W   Size  Description
  0x00       GPL,SL  R/O      1  Log Directory
  0x01           SL  R/O      1  Summary SMART error log
  0x02           SL  R/O      5  Comprehensive SMART error log
  0x03       GPL     R/O      5  Ext. Comprehensive SMART error log
  0x06           SL  R/O      1  SMART self-test log
  0x07       GPL     R/O      1  Extended self-test log
  0x09           SL  R/W      1  Selective self-test log
  0x10       GPL     R/O      1  NCQ Command Error log
  0x11       GPL     R/O      1  SATA Phy Event Counters log
  0x21       GPL     R/O      1  Write stream error log
  0x22       GPL     R/O      1  Read stream error log
  0x30       GPL,SL  R/O      9  IDENTIFY DEVICE data log
  0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
  0xa1       GPL,SL  VS      20  Device vendor specific log
  0xa2       GPL     VS    4496  Device vendor specific log
  0xa8       GPL,SL  VS     129  Device vendor specific log
  0xa9       GPL,SL  VS       1  Device vendor specific log
  0xab       GPL     VS       1  Device vendor specific log
  0xb0       GPL     VS    5176  Device vendor specific log
  0xbe-0xbf  GPL     VS   65535  Device vendor specific log
  0xc0       GPL,SL  VS       1  Device vendor specific log
  0xc1       GPL,SL  VS      10  Device vendor specific log
  0xc3       GPL,SL  VS       8  Device vendor specific log
  0xe0       GPL,SL  R/W      1  SCT Command/Status
  0xe1       GPL,SL  R/W      1  SCT Data Transfer

  SMART Extended Comprehensive Error Log Version: 1 (5 sectors)
  Device Error Count: 384 (device log contains only the most recent 20 errors)
  	CR     = Command Register
  	FEATR  = Features Register
  	COUNT  = Count (was: Sector Count) Register
  	LBA_48 = Upper bytes of LBA High/Mid/Low Registers ]  ATA-8
  	LH     = LBA High (was: Cylinder High) Register    ]   LBA
  	LM     = LBA Mid (was: Cylinder Low) Register      ] Register
  	LL     = LBA Low (was: Sector Number) Register     ]
  	DV     = Device (was: Device/Head) Register
  	DC     = Device Control Register
  	ER     = Error register
  	ST     = Status register
  Powered_Up_Time is measured from power on, and printed as
  DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
  SS=sec, and sss=millisec. It "wraps" after 49.710 days.

  Error 384 [3] occurred at disk power-on lifetime: 34042 hours (1418 days + 10 hours)
    When the command that caused the error occurred, the device was active or idle.

    After command completion occurred, registers were:
    ER -- ST COUNT  LBA_48  LH LM LL DV DC
    -- -- -- == -- == == == -- -- -- -- --
    40 -- 53 00 00 00 00 a3 12 b9 20 00 00  Error: UNC at LBA = 0xa312b920 = 2735913248

    Commands leading to the command that caused the error were:
    CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
    -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
    60 00 00 00 08 00 00 a3 12 b9 20 40 00 16d+06:35:59.162  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b9 18 40 00 16d+06:35:59.154  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b9 10 40 00 16d+06:35:59.154  READ FPDMA QUEUED
    61 00 00 00 08 00 00 a3 12 b9 10 40 00 16d+06:35:59.154  WRITE FPDMA QUEUED
    ef 00 10 00 02 00 00 00 00 00 00 a0 00 16d+06:35:59.154  SET FEATURES [Enable SATA feature]

  Error 383 [2] occurred at disk power-on lifetime: 34042 hours (1418 days + 10 hours)
    When the command that caused the error occurred, the device was active or idle.

    After command completion occurred, registers were:
    ER -- ST COUNT  LBA_48  LH LM LL DV DC
    -- -- -- == -- == == == -- -- -- -- --
    40 -- 53 00 00 00 00 a3 12 b9 10 00 00  Error: UNC at LBA = 0xa312b910 = 2735913232

    Commands leading to the command that caused the error were:
    CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
    -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
    60 00 00 00 08 00 00 a3 12 b9 10 40 00 16d+06:35:53.336  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b9 08 40 00 16d+06:35:53.335  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b9 00 40 00 16d+06:35:53.335  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 f8 40 00 16d+06:35:53.335  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 f0 40 00 16d+06:35:53.331  READ FPDMA QUEUED

  Error 382 [1] occurred at disk power-on lifetime: 34042 hours (1418 days + 10 hours)
    When the command that caused the error occurred, the device was active or idle.

    After command completion occurred, registers were:
    ER -- ST COUNT  LBA_48  LH LM LL DV DC
    -- -- -- == -- == == == -- -- -- -- --
    40 -- 53 00 00 00 00 a3 12 b8 e8 00 00  Error: UNC at LBA = 0xa312b8e8 = 2735913192

    Commands leading to the command that caused the error were:
    CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
    -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
    60 00 00 00 08 00 00 a3 12 b8 e8 40 00 16d+06:35:49.468  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 e0 40 00 16d+06:35:49.460  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 d8 40 00 16d+06:35:49.460  READ FPDMA QUEUED
    61 00 00 00 08 00 00 a3 12 b8 d8 40 00 16d+06:35:49.460  WRITE FPDMA QUEUED
    ef 00 10 00 02 00 00 00 00 00 00 a0 00 16d+06:35:49.459  SET FEATURES [Enable SATA feature]

  Error 381 [0] occurred at disk power-on lifetime: 34042 hours (1418 days + 10 hours)
    When the command that caused the error occurred, the device was active or idle.

    After command completion occurred, registers were:
    ER -- ST COUNT  LBA_48  LH LM LL DV DC
    -- -- -- == -- == == == -- -- -- -- --
    40 -- 53 00 00 00 00 a3 12 b8 d8 00 00  Error: UNC at LBA = 0xa312b8d8 = 2735913176

    Commands leading to the command that caused the error were:
    CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
    -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
    60 00 00 00 08 00 00 a3 12 b8 d8 40 00 16d+06:35:45.676  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 d0 40 00 16d+06:35:45.673  READ FPDMA QUEUED
    ef 00 10 00 02 00 00 00 00 00 00 a0 00 16d+06:35:45.673  SET FEATURES [Enable SATA feature]
    27 00 00 00 00 00 00 00 00 00 00 e0 00 16d+06:35:45.673  READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
    ec 00 00 00 00 00 00 00 00 00 00 a0 00 16d+06:35:45.672  IDENTIFY DEVICE

  Error 380 [19] occurred at disk power-on lifetime: 34042 hours (1418 days + 10 hours)
    When the command that caused the error occurred, the device was active or idle.

    After command completion occurred, registers were:
    ER -- ST COUNT  LBA_48  LH LM LL DV DC
    -- -- -- == -- == == == -- -- -- -- --
    40 -- 53 00 00 00 00 a3 12 b8 c8 00 00  Error: UNC at LBA = 0xa312b8c8 = 2735913160

    Commands leading to the command that caused the error were:
    CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
    -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
    60 00 00 00 08 00 00 a3 12 b8 c8 40 00 16d+06:35:39.283  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 c0 40 00 16d+06:35:39.282  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 b8 40 00 16d+06:35:39.282  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 b0 40 00 16d+06:35:39.270  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 a8 40 00 16d+06:35:39.270  READ FPDMA QUEUED

  Error 379 [18] occurred at disk power-on lifetime: 34042 hours (1418 days + 10 hours)
    When the command that caused the error occurred, the device was active or idle.

    After command completion occurred, registers were:
    ER -- ST COUNT  LBA_48  LH LM LL DV DC
    -- -- -- == -- == == == -- -- -- -- --
    40 -- 53 00 00 00 00 a3 12 b8 a8 00 00  Error: UNC at LBA = 0xa312b8a8 = 2735913128

    Commands leading to the command that caused the error were:
    CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
    -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
    60 00 00 00 08 00 00 a3 12 b8 a8 40 00 16d+06:35:35.558  READ FPDMA QUEUED
    61 00 00 05 78 00 00 65 ac 20 00 40 00 16d+06:35:35.557  WRITE FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 a0 40 00 16d+06:35:35.540  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 98 40 00 16d+06:35:35.532  READ FPDMA QUEUED
    ef 00 10 00 02 00 00 00 00 00 00 a0 00 16d+06:35:35.532  SET FEATURES [Enable SATA feature]

  Error 378 [17] occurred at disk power-on lifetime: 34042 hours (1418 days + 10 hours)
    When the command that caused the error occurred, the device was active or idle.

    After command completion occurred, registers were:
    ER -- ST COUNT  LBA_48  LH LM LL DV DC
    -- -- -- == -- == == == -- -- -- -- --
    40 -- 53 00 00 00 00 a3 12 b8 90 00 00  Error: UNC at LBA = 0xa312b890 = 2735913104

    Commands leading to the command that caused the error were:
    CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
    -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
    60 00 00 00 08 00 00 a3 12 b8 90 40 00 16d+06:35:31.406  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 88 40 00 16d+06:35:31.406  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 80 40 00 16d+06:35:31.405  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 78 40 00 16d+06:35:31.398  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 70 40 00 16d+06:35:31.397  READ FPDMA QUEUED

  Error 377 [16] occurred at disk power-on lifetime: 34042 hours (1418 days + 10 hours)
    When the command that caused the error occurred, the device was active or idle.

    After command completion occurred, registers were:
    ER -- ST COUNT  LBA_48  LH LM LL DV DC
    -- -- -- == -- == == == -- -- -- -- --
    40 -- 53 00 00 00 00 a3 12 b8 70 00 00  Error: UNC at LBA = 0xa312b870 = 2735913072

    Commands leading to the command that caused the error were:
    CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
    -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
    60 00 00 00 08 00 00 a3 12 b8 70 40 00 16d+06:35:27.414  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 68 40 00 16d+06:35:27.413  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 60 40 00 16d+06:35:27.402  READ FPDMA QUEUED
    60 00 00 00 08 00 00 a3 12 b8 58 40 00 16d+06:35:27.401  READ FPDMA QUEUED
    61 00 00 00 08 00 00 a3 12 b8 58 40 00 16d+06:35:27.401  WRITE FPDMA QUEUED

  SMART Extended Self-test Log Version: 1 (1 sectors)
  Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
  # 1  Short offline       Completed: read failure       90%     29204         771754056
  # 2  Short offline       Completed without error       00%        19         -
  # 3  Short offline       Completed without error       00%         0         -

  SMART Selective self-test log data structure revision number 1
   SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
      1        0        0  Not_testing
      2        0        0  Not_testing
      3        0        0  Not_testing
      4        0        0  Not_testing
      5        0        0  Not_testing
  Selective self-test flags (0x0):
    After scanning selected spans, do NOT read-scan remainder of disk.
  If Selective self-test is pending on power-up, resume after 0 minute delay.

  SCT Status Version:                  3
  SCT Version (vendor specific):       522 (0x020a)
  Device State:                        Active (0)
  Current Temperature:                    37 Celsius
  Power Cycle Min/Max Temperature:     37/41 Celsius
  Lifetime    Min/Max Temperature:     18/45 Celsius
  Under/Over Temperature Limit Count:   0/0

  SCT Data Table command not supported

  SCT Error Recovery Control command not supported

  Device Statistics (GP/SMART Log 0x04) not supported

  Pending Defects log (GP Log 0x0c) not supported

  SATA Phy Event Counters (GP Log 0x11)
  ID      Size     Value  Description
  0x000a  2          102  Device-to-host register FISes sent due to a COMRESET
  0x0001  2            0  Command failed due to ICRC error
  0x0003  2            0  R_ERR response for device-to-host data FIS
  0x0004  2            0  R_ERR response for host-to-device data FIS
  0x0006  2            0  R_ERR response for device-to-host non-data FIS
  0x0007  2            0  R_ERR response for host-to-device non-data FIS

Many thanks,
Ranjan


> On Fri, Aug 18, 2023 at 1:30 PM Ranjan Maitra <mlmaitra@xxxxxxx> wrote:
> >
> > Thanks, Roger!
> >
> >
> > On Fri Aug18'23 12:23:23PM, Roger Heflin wrote:
> > > From: Roger Heflin <rogerheflin@xxxxxxxxx>
> > > Date: Fri, 18 Aug 2023 12:23:23 -0500
> > > To: Community support for Fedora users <users@xxxxxxxxxxxxxxxxxxxxxxx>
> > > Reply-To: Community support for Fedora users <users@xxxxxxxxxxxxxxxxxxxxxxx>
> > > Subject: Re: slowness with kernel 6.4.10 and software raid
> > >
> > > Is it moving at all or just stopped?  If just stopped it appears that
> > > md126 is using external:/md127 for something and md127 looks wrong
> > > (both disks are spare) but I don't know in this external case what
> > > md127 should look like.
> >
> > It is moving, slowly. It is a 2 TB drive, but this is weird.
> >
> > >
> > > I would suggest checking messages with grep md12[67] /var/log/messages
> > > (and older messages files if the reboot was not this week) to see what
> > > is going on.
> >
> > Good idea! Here is the result from
> >
> > $ grep md126  /var/log/messages
> >
> >
> >   Aug 14 15:02:30 localhost mdadm[1035]: Rebuild60 event detected on md device /dev/md126
> >   Aug 16 14:21:20 localhost kernel: md/raid1:md126: active with 2 out of 2 mirrors
> >   Aug 16 14:21:20 localhost kernel: md126: detected capacity change from 0 to 3711741952
> >   Aug 16 14:21:20 localhost kernel: md126: p1
> >   Aug 16 14:21:23 localhost systemd[1]: Condition check resulted in dev-md126p1.device - /dev/md126p1 being skipped.
> >   Aug 16 14:21:28 localhost systemd-fsck[942]: /dev/md126p1: clean, 7345384/115998720 files, 409971205/463967488 blocks
> >   Aug 16 14:21:31 localhost kernel: EXT4-fs (md126p1): mounted filesystem 932eb81c-2ab4-4e6e-b093-46e43dbd6c28 r/w with ordered data mode. Quota mode: none.
> >   Aug 16 14:21:31 localhost mdadm[1033]: NewArray event detected on md device /dev/md126
> >   Aug 16 14:21:31 localhost mdadm[1033]: RebuildStarted event detected on md device /dev/md126
> >   Aug 16 14:21:31 localhost kernel: md: data-check of RAID array md126
> >   Aug 16 19:33:18 localhost kernel: md/raid1:md126: sda: rescheduling sector 2735900352
> >   Aug 16 19:33:22 localhost kernel: md/raid1:md126: sda: rescheduling sector 2735900864
> >   Aug 16 19:33:28 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735900496 on sda)
> >   Aug 16 19:33:36 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735900568 on sda)
> >   Aug 16 19:33:41 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735900576 on sda)
> >   Aug 16 19:33:50 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735900624 on sda)
> >   Aug 16 19:34:00 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735900640 on sda)
> >   Aug 16 19:34:10 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735900688 on sda)
> >   Aug 16 19:34:18 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735900712 on sda)
> >   Aug 16 19:34:28 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735900792 on sda)
> >   Aug 16 19:34:32 localhost kernel: md/raid1:md126: redirecting sector 2735900352 to other mirror: sdc
> >   Aug 16 19:34:37 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735900872 on sda)
> >   Aug 16 19:34:45 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735900920 on sda)
> >   Aug 16 19:34:54 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735900992 on sda)
> >   Aug 16 19:34:54 localhost kernel: md/raid1:md126: redirecting sector 2735900864 to other mirror: sdc
> >   Aug 16 19:35:07 localhost kernel: md/raid1:md126: sda: rescheduling sector 2735905704
> >   Aug 16 19:35:11 localhost kernel: md/raid1:md126: sda: rescheduling sector 2735905960
> >   Aug 16 19:35:18 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735905768 on sda)
> >   Aug 16 19:35:19 localhost kernel: md/raid1:md126: redirecting sector 2735905704 to other mirror: sdc
> >   Aug 16 19:35:24 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735906120 on sda)
> >   Aug 16 19:35:33 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735906192 on sda)
> >   Aug 16 19:35:39 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735906448 on sda)
> >   Aug 16 19:35:40 localhost kernel: md/raid1:md126: redirecting sector 2735905960 to other mirror: sdc
> >   Aug 16 19:35:45 localhost kernel: md/raid1:md126: sda: rescheduling sector 2735906472
> >   Aug 16 19:35:49 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735906504 on sda)
> >   Aug 16 19:35:52 localhost kernel: md/raid1:md126: redirecting sector 2735906472 to other mirror: sdc
> >   Aug 16 19:36:03 localhost kernel: md/raid1:md126: sda: rescheduling sector 2735908008
> >   Aug 16 19:36:08 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735908232 on sda)
> >   Aug 16 19:36:16 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735908344 on sda)
> >   Aug 16 19:36:21 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735908424 on sda)
> >   Aug 16 19:36:21 localhost kernel: md/raid1:md126: redirecting sector 2735908008 to other mirror: sda
> >   Aug 16 19:36:30 localhost kernel: md/raid1:md126: sda: rescheduling sector 2735908008
> >   Aug 16 19:36:37 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735908296 on sda)
> >   Aug 16 19:36:38 localhost kernel: md/raid1:md126: redirecting sector 2735908008 to other mirror: sdc
> >   Aug 16 19:36:42 localhost kernel: md/raid1:md126: sda: rescheduling sector 2735908776
> >   Aug 16 19:36:42 localhost kernel: md/raid1:md126: sda: rescheduling sector 2735909032
> >   Aug 16 19:36:46 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735908784 on sda)
> >   Aug 16 19:36:50 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735908944 on sda)
> >   Aug 16 19:36:50 localhost kernel: md/raid1:md126: redirecting sector 2735908776 to other mirror: sdc
> >   Aug 16 19:36:55 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735909312 on sda)
> >   Aug 16 19:37:00 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735909360 on sda)
> >   Aug 16 19:37:04 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735909400 on sda)
> >   Aug 16 19:37:11 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735909520 on sda)
> >   Aug 16 19:37:11 localhost kernel: md/raid1:md126: redirecting sector 2735909032 to other mirror: sdc
> >   Aug 16 19:37:21 localhost kernel: md/raid1:md126: sda: rescheduling sector 2735910056
> >   Aug 16 19:37:21 localhost kernel: md/raid1:md126: sda: rescheduling sector 2735910568
> >   Aug 16 19:37:25 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735910064 on sda)
> >   Aug 16 19:37:31 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735910080 on sda)
> >   Aug 16 19:38:00 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735910128 on sda)
> >   Aug 16 19:38:08 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735910240 on sda)
> >   Aug 16 19:38:12 localhost kernel: md/raid1:md126: redirecting sector 2735910056 to other mirror: sdc
> >   Aug 16 19:38:15 localhost kernel: md/raid1:md126: redirecting sector 2735910568 to other mirror: sdc
> >   Aug 16 19:38:23 localhost kernel: md/raid1:md126: sda: rescheduling sector 2735911080
> >   Aug 16 19:38:23 localhost kernel: md/raid1:md126: sda: rescheduling sector 2735911592
> >   Aug 16 19:38:27 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735911520 on sda)
> >   Aug 16 19:38:27 localhost kernel: md/raid1:md126: redirecting sector 2735911080 to other mirror: sdc
> >   Aug 16 19:38:28 localhost kernel: md/raid1:md126: redirecting sector 2735911592 to other mirror: sdc
> >   Aug 16 19:38:33 localhost kernel: md/raid1:md126: sda: rescheduling sector 2735912104
> >   Aug 16 19:38:37 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735912184 on sda)
> >   Aug 16 19:38:45 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735912240 on sda)
> >   Aug 16 19:38:49 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735912248 on sda)
> >   Aug 16 19:38:59 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735912288 on sda)
> >   Aug 16 19:39:05 localhost kernel: md/raid1:md126: redirecting sector 2735912104 to other mirror: sdc
> >   Aug 16 19:39:10 localhost kernel: md/raid1:md126: sda: rescheduling sector 2735912872
> >   Aug 16 19:39:14 localhost kernel: md/raid1:md126: sda: rescheduling sector 2735913128
> >   Aug 16 19:39:25 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735912976 on sda)
> >   Aug 16 19:39:33 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735913048 on sda)
> >   Aug 16 19:39:37 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735913072 on sda)
> >   Aug 16 19:39:41 localhost kernel: md/raid1:md126: redirecting sector 2735912872 to other mirror: sdc
> >   Aug 16 19:39:45 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735913128 on sda)
> >   Aug 16 19:39:55 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735913176 on sda)
> >   Aug 16 19:40:05 localhost kernel: md/raid1:md126: read error corrected (8 sectors at 2735913232 on sda)
> >
> >
> > And here is what I get from:
> >
> > $ grep  md127  /var/log/messages
> >
> >
> >   Aug 16 14:16:38 localhost systemd[1]: mdmon@md127.service: Deactivated successfully.
> >   Aug 16 14:16:38 localhost systemd[1]: mdmon@md127.service: Unit process 884 (mdmon) remains running after unit stopped.
> >   Aug 16 14:16:38 localhost systemd[1]: Stopped mdmon@md127.service - MD Metadata Monitor on /dev/md127.
> >   Aug 16 14:16:38 localhost audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=mdmon@md127 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
> >   Aug 16 14:16:38 localhost systemd[1]: mdmon@md127.service: Consumed 41.719s CPU time.
> >   Aug 16 14:21:20 localhost systemd[1]: Starting mdmon@md127.service - MD Metadata Monitor on /dev/md127...
> >   Aug 16 14:21:20 localhost systemd[1]: Started mdmon@md127.service - MD Metadata Monitor on /dev/md127.
> >   Aug 16 14:21:20 localhost audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=mdmon@md127 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
> >
> > >
> > > Maybe also if you have a prior good reboot in messages file include
> > > that and see what happened differently between the 2.
> >
> > Yeah, I do not know where to find this. I looked into /var/log/messages, but it looks like it starts on August 13, which was a surprise to me, and the last non-responsive instance for me was last week (August 10, I think, when I booted into the 6.4 kernel). I did reboot in frustration on August 16.
> >
> > Thanks,
> > Ranjan
> >
> >
> > >
> > > On Fri, Aug 18, 2023 at 7:46 AM Ranjan Maitra <mlmaitra@xxxxxxx> wrote:
> > > >
> > > > On Thu Aug17'23 10:37:29PM, Samuel Sieb wrote:
> > > > > From: Samuel Sieb <samuel@xxxxxxxx>
> > > > > Date: Thu, 17 Aug 2023 22:37:29 -0700
> > > > > To: users@xxxxxxxxxxxxxxxxxxxxxxx
> > > > > Reply-To: Community support for Fedora users <users@xxxxxxxxxxxxxxxxxxxxxxx>
> > > > > Subject: Re: slowness with kernel 6.4.10 and software raid
> > > > >
> > > > > On 8/17/23 21:38, Ranjan Maitra wrote:
> > > > > > $ cat /proc/mdstat
> > > > > >   Personalities : [raid1]
> > > > > >   md126 : active raid1 sda[1] sdc[0]
> > > > > >         1855870976 blocks super external:/md127/0 [2/2] [UU]
> > > > > >         [=>...................]  check =  8.8% (165001216/1855870976) finish=45465.2min speed=619K/sec
> > > > > >
> > > > > >   md127 : inactive sda[1](S) sdc[0](S)
> > > > > >         10402 blocks super external:imsm
> > > > > >
> > > > > >   unused devices: <none>
> > > > > >
> > > > > > I am not sure what it is doing, and I am a bit concerned that this will go on at this rate for about 20 days. No knowing what will happen after that, and also if this problem will recur with another reboot.
> > > > >
> > > > > After a certain amount of time, mdraid will do a verification of the data
> > > > > where it scans the entire array.  If you reboot, it will continue from where
> > > > > it left off.  But that is *really* slow, so you should find out what's going
> > > > > on there.
> > > >
> > > > Yes, I know, just not sure what to do. Thanks very much!
> > > >
> > > > Any suggestion is appreciated!
> > > >
> > > > Best wishes,
> > > > Ranjan
> > > > _______________________________________________
> > > > users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
> > > > To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
> > > > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > > List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx
> > > > Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
> > > _______________________________________________
> > > users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
> > > To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
> > > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx
> > > Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
> > _______________________________________________
> > users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
> > To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
> > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx
> > Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
> _______________________________________________
> users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx
> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
_______________________________________________
users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue



[Index of Archives]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [EPEL Devel]     [Fedora Magazine]     [Fedora Summer Coding]     [Fedora Laptop]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Desktop]     [Fedora Fonts]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Yosemite News]     [Gnome Users]     [KDE Users]     [Fedora Art]     [Fedora Docs]     [Fedora Sparc]     [Libvirt Users]     [Fedora ARM]

  Powered by Linux