-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 7/30/11 9:53 AM, Christian Brunner wrote: > Fyodor and I are struggling to get a fully stable ceph cluster up and running. > > When we run an Ceph-Objectstore (OSD) ontop of an ext4 filesystem, we > get fsck errors, when we check the filesystem (see below). BTW, this should be fixed now as of my commit 6d6a435190bdf2e04c9465cde5bdc3ac68cf11a4 ext4: fix race in xattr block allocation path I think it made its way to a couple older -stable kernels, too. - -Eric > Fyodor is running 3.0. > I am running a RHEL6.1 Kernel (2.6.32-131.6.1.el6.x86_64). > > Any help or hints on how to trace the bug would be appreciated. > > Thanks, > Christian > > 2011/7/30 Fyodor Ustinov <ufm@xxxxxx>: >> fail. Epic fail. >> >> Absolutely reproducible. >> >> I have ceph cluster with this configuration: >> >> 8 physical servers >> 14 osd servers. >> Each osd server have personal fs. >> 48T total size of ceph cluster. >> 17T used. >> >> Now, step by step: >> >> 1. Stop ceph server osd0 >> /etc/init.d/ceph stop >> >> 2. Make fresh fs for osd >> umount /osd.0 >> mkfs.ext4 /dev/sdc1 >> tune2fs -o journal_data_writeback /dev/sdc1 >> mount -a >> # string from /etc/fstab: >> # /dev/sdc1 /osd.0 ext4 >> user_xattr,rw,noexec,nodev,noatime,nodiratime,data=writeback,barrier=0 >> 0 2 >> ceph mon getmap -o /tmp/monmap >> cosd --mkfs -i 0 --monmap /tmp/monmap >> >> 3. Start ceph server osd0 >> /etc/init.d/ceph start >> >> Now, make a big cup of coffee and begin to wait. >> >> After completion of rebalancing do: >> /etc/init.d/ceph stop >> umount /osd.0 >> fsck.ext4 -fy /dev/sdc1 >> >> and see many-many messages like: >> >> Inode 238551053, i_blocks is 24, should be 32. Fix? yes >> >> Inode 238551054, i_blocks is 40, should be 32. Fix? yes >> >> Inode 238551066, i_blocks is 24, should be 32. Fix? yes >> >> Inode 238944257, i_blocks is 8, should be 16. Fix? yes >> >> Inode 239206414, i_blocks is 8, should be 16. Fix? yes >> >> Inode 239206416, i_blocks is 40, should be 32. Fix? yes >> >> Inode 239206431, i_blocks is 8, should be 16. Fix? yes >> >> Inode 239206441, i_blocks is 24, should be 32. Fix? yes >> >> Voila. >> >> P.S. No any message in syslog. No any message in console. >> >> WBR, >> Fyodor. >> >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > -- > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJOwoljAAoJECCuFpLhPd7gjxIQAJ7B+f7EYxBZ+48gUrncmB5r Izkkv2ACza+27g/CUi9ku9j1o3pjZwLNhzo3Fj0gwweB3WaY9T+JMXnfInSFegeR GCT/8XQqGWFVoRQKKc4wUBKGgW5f+3HTgYLqUY0Z38MqMHpIMXYswXdOSB1Wc4MC p+jEjHmTWftklpIjv+Vm61AejpoUO93SFE5gUuBeKSZxwjifV1uTUXtaZCQXUG5N EFz+sS7YvGrttAldK+lbiq7sa7IKINnB5lbDs5ChSZoytSF9hPIRgDOTLrkAZ+k8 YovLWbu2gwGMcZEhu3ZLJ7NdtZbn45A/fh/grNU8nezTo0cTHBTYZCLqtjsUDuMr mwUIDNUEAv6LIz0OyeJMftDX4TzxjQyEQOgYg5wyCKCjE2Nyktyap2T5sAFKamJJ pgTUt0JSpXgDnDBL7Y3M6RbY8DQsDHIir3A7aOwdINGKweNiJXBYC3LWYHIXY0bd yoKXT6e/Bentlj+Peugg51bw91JtlqxJT4qJfk6HMF00uxrfWHlvzht7Lu61YxrW LBQgNyQ+Gu1drHIHyIFu95UePhzEGQcLXB3YUe7BKFGe4Vde8Jcrwn1RSFmILU6H o9jPncZVanQYy9URQqnrcHzqpRfViuVeyhuAUh3lPt4Q7jIrr+2Ug6xWxIkBrtTt /iKT0p8+aR3HhakrGqp4 =VbZG -----END PGP SIGNATURE----- -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
