Fwd: Kernel 3.0.0 + ext4 + ceph == ...

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Fyodor and I are struggling to get a fully stable ceph cluster up and running.

When we run an Ceph-Objectstore (OSD) ontop of an ext4 filesystem, we
get fsck errors, when we check the filesystem (see below).

Fyodor is running 3.0.
I am running a RHEL6.1 Kernel (2.6.32-131.6.1.el6.x86_64).

Any help or hints on how to trace the bug would be appreciated.

Thanks,
Christian

2011/7/30 Fyodor Ustinov <ufm@xxxxxx>:
> fail. Epic fail.
>
> Absolutely reproducible.
>
> I have ceph cluster with this configuration:
>
> 8 physical servers
> 14 osd servers.
> Each osd server have personal fs.
> 48T total size of ceph cluster.
> 17T used.
>
> Now, step by step:
>
> 1. Stop ceph server osd0
> /etc/init.d/ceph stop
>
> 2. Make fresh fs for osd
> umount /osd.0
> mkfs.ext4 /dev/sdc1
> tune2fs -o journal_data_writeback /dev/sdc1
> mount -a
> # string from /etc/fstab:
> # /dev/sdc1 /osd.0          ext4
>  user_xattr,rw,noexec,nodev,noatime,nodiratime,data=writeback,barrier=0
>    0       2
> ceph mon getmap -o /tmp/monmap
> cosd --mkfs -i 0 --monmap /tmp/monmap
>
> 3. Start ceph server osd0
> /etc/init.d/ceph start
>
> Now, make a big cup of coffee and begin to wait.
>
> After completion of rebalancing do:
> /etc/init.d/ceph stop
> umount /osd.0
> fsck.ext4 -fy /dev/sdc1
>
> and see many-many messages like:
>
> Inode 238551053, i_blocks is 24, should be 32.  Fix? yes
>
> Inode 238551054, i_blocks is 40, should be 32.  Fix? yes
>
> Inode 238551066, i_blocks is 24, should be 32.  Fix? yes
>
> Inode 238944257, i_blocks is 8, should be 16.  Fix? yes
>
> Inode 239206414, i_blocks is 8, should be 16.  Fix? yes
>
> Inode 239206416, i_blocks is 40, should be 32.  Fix? yes
>
> Inode 239206431, i_blocks is 8, should be 16.  Fix? yes
>
> Inode 239206441, i_blocks is 24, should be 32.  Fix? yes
>
> Voila.
>
> P.S. No any message in syslog. No any message in console.
>
> WBR,
>    Fyodor.
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Reiser Filesystem Development]     [Ceph FS]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Linux FS]     [Yosemite National Park]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Device Mapper]     [Linux Media]

  Powered by Linux