A deadlock isn't related to the mdadm versjon - it'd be in the kernel. Vennlig hilsen roy -- Roy Sigurd Karlsbakk (+47) 98013356 http://blogg.karlsbakk.net/ GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt -- Hið góða skaltu í stein höggva, hið illa í snjó rita. ----- Original Message ----- > From: "search" <search@xxxxxxxxxxxxxx> > To: "Roy Sigurd Karlsbakk" <roy@xxxxxxxxxxxxx> > Cc: "Linux Raid" <linux-raid@xxxxxxxxxxxxxxx> > Sent: Wednesday, 27 February, 2019 18:06:53 > Subject: Re: Raid5 reshape stuck - md0 process hang > Hi Roy, Hi Wols, > > thanks for your suggestions. > > I already tried to force the assemble, that causes also a deadlock of > the mdadm process. > > I could manage the situation and found that it doesnt lock if I mount it > readonly. Now I exported all data over the day - seems to work fine for > now. Killed the array and restarted with a fresh raid10. We will see > what the future brings at import time. > > Btw. used mdadm 4.1. > > Greetings > > Brian > > Am 27.02.19 um 17:59 schrieb Roy Sigurd Karlsbakk: >> The event count on the different drives are almost the same - that is - the >> event count on the *drives* are the same, but not for md0, but then again - >> it's only 3 events off, so I doubt it'll make much of a problem. I'd try mdadm >> --assemble --force on that. Also - check if there are any more messages in >> dmesg/syslog regarding the drives. >> >> Vennlig hilsen >> >> roy >> -- >> Roy Sigurd Karlsbakk >> (+47) 98013356 >> http://blogg.karlsbakk.net/ >> GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt >> -- >> Hið góða skaltu í stein höggva, hið illa í snjó rita. >> >> ----- Original Message ----- >>> From: search@xxxxxxxxxxxxxx >>> To: "Linux Raid" <linux-raid@xxxxxxxxxxxxxxx> >>> Sent: Tuesday, 26 February, 2019 11:11:19 >>> Subject: Raid5 reshape stuck - md0 process hang >>> Hey, >>> >>> i am really stuck with a problem. I tried to reshape my raid5 with 3 disks, to 4 >>> disks. The reshape ran this night, until 94% and got stuck in this process. I >>> can see this message in syslog: >>> >>> „task md0_reshape blocked for more than 120 seconds“ >>> >>> After seeing it hanging I set the new drive to faulty and removed it during the >>> hanging reshape progress - I thought the added drive was the issue. >>> >>> If I take a look at top, there is a md0 process which is stuck at 100% cpu. >>> >>> Originally the system runs on xenserver 7.2 - I booted up a newer live disk of >>> ubuntu 18.04 to look how the newer mdadm version does. - Same behaviour. >>> >>> If I try to assemble my raid on the live dvd, there’s the same issue. Have done >>> this with: mdadm —assemble —scan >>> >>> Here are some logs: >>> >>> /dev/md0: >>> Version : 1.2 >>> Creation Time : Tue May 31 18:18:33 2016 >>> Raid Level : raid5 >>> Array Size : 3906766848 (3725.78 GiB 4000.53 GB) >>> Used Dev Size : 1953383424 (1862.89 GiB 2000.26 GB) >>> Raid Devices : 4 >>> Total Devices : 3 >>> Persistence : Superblock is persistent >>> >>> Intent Bitmap : Internal >>> >>> Update Time : Tue Feb 26 09:27:51 2019 >>> State : clean, degraded, reshaping >>> Active Devices : 3 >>> Working Devices : 3 >>> Failed Devices : 0 >>> Spare Devices : 0 >>> >>> Layout : left-symmetric >>> Chunk Size : 512K >>> >>> Consistency Policy : bitmap >>> >>> Reshape Status : 94% complete >>> Delta Devices : 1, (3->4) >>> >>> Name : maulwurfshuegel:0 >>> UUID : b0f13031:163a65b5:2acfee57:dc39a18f >>> Events : 8194152 >>> >>> Number Major Minor RaidDevice State >>> 0 8 33 0 active sync /dev/sdc1 >>> 1 8 1 1 active sync /dev/sda1 >>> 3 8 17 2 active sync /dev/sdb1 >>> - 0 0 3 removed >>> >>> >>> /dev/sda1: >>> Magic : a92b4efc >>> Version : 1.2 >>> Feature Map : 0x45 >>> Array UUID : b0f13031:163a65b5:2acfee57:dc39a18f >>> Name : maulwurfshuegel:0 >>> Creation Time : Tue May 31 18:18:33 2016 >>> Raid Level : raid5 >>> Raid Devices : 4 >>> >>> Avail Dev Size : 3906766991 (1862.89 GiB 2000.26 GB) >>> Array Size : 5860150272 (5588.67 GiB 6000.79 GB) >>> Used Dev Size : 3906766848 (1862.89 GiB 2000.26 GB) >>> Data Offset : 260096 sectors >>> New Offset : 257024 sectors >>> Super Offset : 8 sectors >>> State : clean >>> Device UUID : 4f7cc9bc:78c54dbd:9cd93978:7d1ca9fb >>> >>> Internal Bitmap : 8 sectors from superblock >>> Reshape pos'n : 5508850176 (5253.65 GiB 5641.06 GB) >>> Delta Devices : 1 (3->4) >>> >>> Update Time : Tue Feb 26 08:46:25 2019 >>> Bad Block Log : 512 entries available at offset 72 sectors >>> Checksum : eed30521 - correct >>> Events : 8194149 >>> >>> Layout : left-symmetric >>> Chunk Size : 512K >>> >>> Device Role : Active device 1 >>> Array State : AAA. ('A' == active, '.' == missing, 'R' == replacing) >>> >>> /dev/sdb1: >>> Magic : a92b4efc >>> Version : 1.2 >>> Feature Map : 0x4d >>> Array UUID : b0f13031:163a65b5:2acfee57:dc39a18f >>> Name : maulwurfshuegel:0 >>> Creation Time : Tue May 31 18:18:33 2016 >>> Raid Level : raid5 >>> Raid Devices : 4 >>> >>> Avail Dev Size : 3906767024 (1862.89 GiB 2000.26 GB) >>> Array Size : 5860150272 (5588.67 GiB 6000.79 GB) >>> Used Dev Size : 3906766848 (1862.89 GiB 2000.26 GB) >>> Data Offset : 260096 sectors >>> New Offset : 257024 sectors >>> Super Offset : 8 sectors >>> State : clean >>> Device UUID : 6696cd3f:a7439cef:c8d26a6e:186869ea >>> >>> Internal Bitmap : 8 sectors from superblock >>> Reshape pos'n : 5508850176 (5253.65 GiB 5641.06 GB) >>> Delta Devices : 1 (3->4) >>> >>> Update Time : Tue Feb 26 08:46:25 2019 >>> Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present. >>> Checksum : 8916e961 - correct >>> Events : 8194149 >>> >>> Layout : left-symmetric >>> Chunk Size : 512K >>> >>> Device Role : Active device 2 >>> Array State : AAA. ('A' == active, '.' == missing, 'R' == replacing) >>> >>> /dev/sdc1: >>> Magic : a92b4efc >>> Version : 1.2 >>> Feature Map : 0x4d >>> Array UUID : b0f13031:163a65b5:2acfee57:dc39a18f >>> Name : maulwurfshuegel:0 >>> Creation Time : Tue May 31 18:18:33 2016 >>> Raid Level : raid5 >>> Raid Devices : 4 >>> >>> Avail Dev Size : 3906766991 (1862.89 GiB 2000.26 GB) >>> Array Size : 5860150272 (5588.67 GiB 6000.79 GB) >>> Used Dev Size : 3906766848 (1862.89 GiB 2000.26 GB) >>> Data Offset : 260096 sectors >>> New Offset : 257024 sectors >>> Super Offset : 8 sectors >>> State : clean >>> Device UUID : 48b8f98f:e1c5639b:29b22511:6285341e >>> >>> Internal Bitmap : 8 sectors from superblock >>> Reshape pos'n : 5508850176 (5253.65 GiB 5641.06 GB) >>> Delta Devices : 1 (3->4) >>> >>> Update Time : Tue Feb 26 08:46:25 2019 >>> Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present. >>> Checksum : 5b908bc3 - correct >>> Events : 8194149 >>> >>> Layout : left-symmetric >>> Chunk Size : 512K >>> >>> Device Role : Active device 0 >>> Array State : AAA. ('A' == active, '.' == missing, 'R' == replacing) >>> >>> What could possibly the next steps? >>> >>> Thanks Guys! >>> >>> Greetings > >> Brian
