Fyodor and I are struggling to get a fully stable ceph cluster up and running. When we run an Ceph-Objectstore (OSD) ontop of an ext4 filesystem, we get fsck errors, when we check the filesystem (see below). Fyodor is running 3.0. I am running a RHEL6.1 Kernel (2.6.32-131.6.1.el6.x86_64). Any help or hints on how to trace the bug would be appreciated. Thanks, Christian 2011/7/30 Fyodor Ustinov <ufm@xxxxxx>: > fail. Epic fail. > > Absolutely reproducible. > > I have ceph cluster with this configuration: > > 8 physical servers > 14 osd servers. > Each osd server have personal fs. > 48T total size of ceph cluster. > 17T used. > > Now, step by step: > > 1. Stop ceph server osd0 > /etc/init.d/ceph stop > > 2. Make fresh fs for osd > umount /osd.0 > mkfs.ext4 /dev/sdc1 > tune2fs -o journal_data_writeback /dev/sdc1 > mount -a > # string from /etc/fstab: > # /dev/sdc1 /osd.0 ext4 > user_xattr,rw,noexec,nodev,noatime,nodiratime,data=writeback,barrier=0 > 0 2 > ceph mon getmap -o /tmp/monmap > cosd --mkfs -i 0 --monmap /tmp/monmap > > 3. Start ceph server osd0 > /etc/init.d/ceph start > > Now, make a big cup of coffee and begin to wait. > > After completion of rebalancing do: > /etc/init.d/ceph stop > umount /osd.0 > fsck.ext4 -fy /dev/sdc1 > > and see many-many messages like: > > Inode 238551053, i_blocks is 24, should be 32. Fix? yes > > Inode 238551054, i_blocks is 40, should be 32. Fix? yes > > Inode 238551066, i_blocks is 24, should be 32. Fix? yes > > Inode 238944257, i_blocks is 8, should be 16. Fix? yes > > Inode 239206414, i_blocks is 8, should be 16. Fix? yes > > Inode 239206416, i_blocks is 40, should be 32. Fix? yes > > Inode 239206431, i_blocks is 8, should be 16. Fix? yes > > Inode 239206441, i_blocks is 24, should be 32. Fix? yes > > Voila. > > P.S. No any message in syslog. No any message in console. > > WBR, > Fyodor. > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
