Le 14/02/2013 16:03, Eric Sandeen a écrit :
On 2/14/13 9:00 AM, Rémi Cailletaud wrote:
Le 14/02/2013 10:39, Rémi Cailletaud a écrit :
Le 14/02/2013 09:21, Rémi Cailletaud a écrit :
Le 13/02/2013 22:38, Eric Sandeen a écrit :
On 2/13/13 2:12 PM, Eric Sandeen wrote:
On 2/13/13 1:50 PM, Eric Sandeen wrote:
On 2/13/13 12:09 PM, Rémi Cailletaud wrote:
Le 13/02/2013 18:52, Eric Sandeen a écrit :
<snip>
Did the filesystem grow actually work?
# xfs_db -c "sb 0" -c "p" /dev/vg0/tomo-201111
magicnum = 0x58465342
blocksize = 4096
dblocks = 10468982745
That looks like it's (still?) a 38TiB/42TB filesystem, with:
sectsize = 512
512 sectors.
How big was it before you tried to grow it, and how much did you try to grow it by? Maybe the size never changed.
Was 39, growing to 44. Testdisk says 48 TB / 44 TiB... There is some chance that it was never really growed.
At mount time it tries to set the sector size of the device; its' a hard-4k device, so setting it to 512 fails.
This may be as much of an LVM issue as anything; how do you get the LVM device back to something with 512-byte logical sectors? I have no idea...
*if* the fs didn't actually grow, and if the new 4k-sector space is not used by the filesystem, and if you can somehow remove that new space from the device and set the LV back to 512 sectors, you might be in good shape.
I dont either know how to see nor set LV sector size. It's 100% sure that anything was copied on 4k sector size, and pretty sure that the fs did not really grow.
I think the same blockdev command will tell you.
Proceed with extreme caution here, I wouldn't start just trying random things unless you have some other way to get your data back (backups?). I'd check with LVM folks as well, and maybe see if dchinner or the sgi folks have other suggestions.
Sigh... No backup (44To is too large for us...) ! I'm running a testdisk recover, but I'm not very confident about success...
Thanks to deeper investigate this...
First let's find out if the filesystem actually thinks it's living on the new space.
What is the way to make it talk about that ?
well, you have 10468982745 4k blocks in your filesystem, so 42880953323520 bytes of xfs filesystem.
Look at your lvm layout, does that extend into the new disk space or is it confined to the original disk space?
Seems it does not : lvm map shows 48378494844928 bytes, 1304432738304 on the 4K device.
lvm folks I talk to say that if you remove the 4k device from the lvm volume it should switch back to 512 sectors.
so if you can can convince yourself that 42880953323520 bytes does not cross into the newly added disk space, just remove it again, and everything should be happy.
Unless your rash decision to start running "testdisk" made things worse ;)
I tested this. I had a PV on a normal 512 device, then used scsi_debug to create a 4k device.
I created an LV on the 512 device& mounted it, then added the 4k device as you did. growfs failed immediately, and the device won't remount due to the sector size change.
I verified that removing the 4k device from the LV changes the LV back to a 512 sector size.
However, I'm not 100% sure how to remove just the 4K PV; when I did it, I did something wrong and it reduced the size of my LV to the point where it corrupted the filesystem. :) Perhaps you are a better lvm admin than I am.
How did you remove the pv ? I would tend to use vgreduce, but I'm a bit (a lot, in fact) scary about fs corruption. That's why I was wondering about pvmove'ing extents on a 512K device
Or may a vgcfgrestore be safer ? Should I ask lvm folks ?
I tried a test as you suggest using scsi_debug.
2 PV, one 512 and one 4096 bytes sector.
after adding the 4K device, growfs fail, and partition wont remount. I tried a vgcfgrestore and vgreduce, but it does not mount : same error...
XFS (dm-5): device supports 4096 byte sectors (not 512)
In that case I think you must not have actually (completely?) removed the 4k device.
That's it! After vgchange-ing un/available, lv mounts !
Following steps reproduce the "bug", considering we already have one pv
on /dev/sdc (512 sectors device) :
- create a virtual 4k scsi device and create a pv on it :
# modprobe scsi_debug sector_size=4096 dev_size_mb=256
# pvcreate /dev/sdd
- create a vg with both pv, and create an lv on sdc (I specified exact
extents count of sdc) :
# vgcreate vgtest /dev/sdc /dev/sdd
# lvcreate -n lvtest -l 3759 vgtest
- mkfs, mount :
# mkfs.xfs /dev/vgtest/lvtest
# mount /dev/vgtest/lvtest /mnt/tmp
- the bad thing : lvextend and growfs (should not lvm or xfs check this
sector size stuff ?):
# lvextend -l+40 /dev/vgtest/lvtest
# xfs_growfs /mnt/tmp
(fail with xfs_growfs: XFS_IOC_FSGROWFSDATA xfsctl failed)
- the scary part :
# umount /mnt/tmp
# mount /dev/vgtest/lvtest /mnt/tmp
mount: function not implemented
# tail -1 /var/log/messages
Feb 14 16:41:52 hamaika kernel: [ 481.055422] XFS (dm-5): device
supports 4096 byte sectors (not 512)
- the huge relief (restoring *before* lvextend) :
# vgcfgrestore -f /etc/lvm/archive/vgtest_00029-84595486.vg vgtest
# vgchange -a n vgtest
# vgchange -a y vgtest
# mount /dev/vgtest/lvtest /mnt/tmp
yippee, my datas are back !!
Should I submit a bug report ? On LVM, XFS, both ?
However, a great thanks for your help... I learned some stuff about lvm
and xfs today ;)
Cheers,
rémi
I think you'll need to seek help from LVM people in order to proceed... I'm sure it's possible to safely and completely remove the newly added space, but I don't know how.
-Eric
--
Rémi Cailletaud - IE CNRS
3SR - Laboratoire Sols, Solides, Structures - Risques
BP53, 38041 Grenoble CEDEX 0
FRANCE
remi.cailletaud@xxxxxxxxxxxxxxx
Tél: +33 (0)4 76 82 52 78
Fax: +33 (0)4 76 82 70 43
_______________________________________________
xfs mailing list
xfs@xxxxxxxxxxx
http://oss.sgi.com/mailman/listinfo/xfs
