On 03/12/2010 03:01 PM, Kapetanakis Giannis wrote:
Hi,
The last few days I'm trying to setup a large filesystem
14TB and is always getting corrupted...
This is my latest non-working setup. Sorry for long message
but I want to make clear my actual setup
2.6.18-164.11.1.el5PAE (x86)
4GB RAM
CONFIG_EFI_VARS=y
CONFIG_EFI=y
/dev/sdb - 6 x 1.5TB SATA drives in hardware RAID 5 (256 chunk size)
/dev/sdc - 6 x 1.5TB SATA drives in hardware RAID 5 (256 chunk size)
-- Both hardware raids are GPT labeled
Model: Adaptec ARRAY01 (scsi)
Disk /dev/sdb: 7489GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 17.4kB 7489GB 7489GB ARRAY01
Model: Adaptec ARRAY02 (scsi)
Disk /dev/sdc: 7489GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 17.4kB 7489GB 7489GB ARRAY02
-- /dev/md_d0 software raid0 on /dev/sdb1 and /dev/sdc1
-- (doing on /dev/sdb and /dev/sdc also corrupted)
-- this raid0 is also GPT labeled (256 chunk size)
mdadm --create /dev/md_d0 -a p1 -c 256 -l 0 -n 2 /dev/sdb1 /dev/sdc1
md_d0 : active raid0 sdb1[0] sdc1[1]
14627614208 blocks 256k chunks
Model: Unknown (unknown)
Disk /dev/md_d0: 15.0TB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 17.4kB 15.0TB 15.0TB primary
-- LVM2 on top of /dev/md_d0p1
--- Physical volume ---
PV Name /dev/md_d0p1
VG Name vgshare
PV Size 13.62 TB / not usable 3.47 MB
PE Size (KByte) 4096
--- Volume group ---
VG Name vgshare
System ID
Format lvm2
VG Size 13.62 TB
PE Size 4.00 MB
--- Logical volume ---
LV Name /dev/vgshare/share
VG Name vgshare
LV UUID Aoj27F-kf4U-i6XE-eNWg-hMLX-MS1h-s3oArp
LV Write Access read/write
LV Status available
# open 0
LV Size 7.00 TB
Current LE 1835008
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 2048
Block device 253:4
-- /dev/vgshare/share is ext4 formated with
mkfs.ext4 -b 4096 -E stride=64,stripe-width=128 /dev/md_d0p1
All well so far. And I start write data...
Then I create a new 2TB test lv
and format it.
/share_7TB is lost, /test_2TB is there....
Mar 12 21:13:28 server kernel: EXT4-fs error (device dm-4):
ext4_mb_generate_buddy: EXT4-fs: group 0: 32768 blocks in bitmap, 3248
in gd
Mar 12 21:13:30 server kernel: EXT4-fs error (device dm-4):
ext4_mb_generate_buddy: EXT4-fs: group 1648: 24544 blocks in bitmap,
153 in gd
Mar 12 21:13:31 server kernel: attempt to access beyond end of device
Mar 12 21:13:31 server kernel: dm-4: rw=2, want=15493450520,
limit=15032385536
Mar 12 21:13:31 server kernel: attempt to access beyond end of device
--snip
Mar 12 21:17:49 server kernel: EXT4-fs error (device dm-4):
ext4_mb_release_inode_pa: free 1802, pa_free 1458
Mar 12 21:17:49 server kernel: EXT4-fs: mballoc: 93430033 blocks
705745 reqs (482734 success)
Mar 12 21:17:49 server kernel: EXT4-fs: mballoc: 327298 extents
scanned, 241152 goal hits, 219206 2^N hits, 0 breaks, 0 lost
Mar 12 21:17:49 server kernel: EXT4-fs: mballoc: 9561 generated and it
took 2012010656
Mar 12 21:17:49 server kernel: EXT4-fs: mballoc: 85047925
preallocated, 30759591 discarded
Mar 12 21:18:09 server kernel: EXT4-fs: ext4_check_descriptors: Inode
table for group 0 not in group (block 1936681314)!
Mar 12 21:18:09 server kernel: EXT4-fs: group descriptors corrupted!
GFS instead of ext4 also corrupted.
ext4 on top of /dev/md0 (non partitioned) on top of /dev/sdb /dev/sdc
(without GPT) also corrupted.
I want to use software raid0 on top of the two hardware radi5
for better performance.
I understood that labeling GPT would solve this problem.
Is it x86 problem? Something fishy with my setup probably
but can't figure it out.
thanks and sorry for long message,
but I can't find a way to get this mirror server up
on it's feet again after this upgrade....
regards,
Giannis
This is probably an issue with the early version of ext4 you are using -
note that the support for ext4 > 16TB is still gated by some work done
up in the tools chain.
Have you tried xfs?
regards,
Ric
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
