Hello list, I have a problem that I have some small hope that you can help me with. I had a raid 5 array of 5 1TB drives, set up in an LVM group and then cryptomapped to an XFS filesystem. Now one of the drives failed, so I had to buy a new one. As I was also running out of space, I decided to buy 5 new 3TB drives instead. Last time I did this, I just set up a new array and then rsynced the entire filesystem over to the new array, but this time I figured I should be clever. So I started replacing one disk at a time with the new drives - each time waiting until they had synced up before removing the next old drive. This worked pretty well, until the array failed completely when I yanked a drive. I believe it was because another of the old drives hiccupped at the same time so the array stopped since I was missing two drives. Because I had my entire LVM + cryptomap + XFS setup running, I figured it was just faster to reboot than to take everything down to re-assemble with a --force to get it running again. So I rebooted, and did a manual reassemble with --force. And the array was up, and I ran my little setup-script that does the cryptomapping and mounting of the array. Here comes the problem: "mount" produced "mount: Function not implemented" I became slightly worried, and tried "xfs_check", which claimed I had a log file and refused to work. So I ran "xfs_repair", which said "xfs_repair: warning - cannot set blocksize 512 on block device /dev/mapper/data: Invalid argument" This made me start looking at the block sizes of the different drives. I ran "blockdevice --getbsz", and it became evident that the old drives had block size 1024, and the new ones 4096. This begs a few questions: 1) What block size will an md device have? 1.1) When it has 5 drives with block size 1024? 1.2) When it has 3 drives with block size 1024 and 2 with block size 4096? 2) If it gets a different block size in case 1.2), why didn't it happen until I rebooted? (The XFS mount seemed to work just fine, I accessed files without a hitch) 3) Is there a way to "force" a block size for an md device? In the end, of course, I am not expert enough to be sure that the block size is the problem, but this is the only lead I have. My weak hypothesis is "the md device kept the same 1024 block size (which is odd, since XFS seems to want 512?) until I rebooted, when it checked the block sizes of the component drives and chose the biggest one - which caused XFS to refuse to mount the drive". This makes me suspect that I must buy at least one new 1TB drive with a block size of 1024, sync into the array and restart the array without any 4096 block size drives in it. But before I do this, I thought I better ask some experts what they figured... best regards (and a panicky cry for help) Martin -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
