On Wed, Jul 27, 2011 at 3:05 AM, Fyodor Ustinov <ufm@xxxxxx> wrote: > On 07/27/2011 01:24 AM, Gregory Farnum wrote: >> >> Hmm, I'm afraid I can't see anything useful in these logs right now -- >> just confusion! >> >> The only thing I can think of that might provide useful information >> now is to turn off osd5, update its config file to have heavy >> debugging information (debug osd = 20, at a minimum), and then start >> the OSD back up again. Then we can see if pg 0.256 is still having any >> issues (it is in the logs you sent me). >> >> If not I'll have to mark this as "Can't reproduce" and hope we see it >> again from somebody with higher log levels on. :( >> -Greg > > I started manual scrubbing on osd5 and got next message: > > 2011-07-27 11:38:30.360812 log 2011-07-27 11:38:28.540559 osd5 > 10.5.51.145:6803/19737 2842 : [ERR] 0.256 scrub stat mismatch, got 2196/2196 > objects, 0/0 clones, 9046200695/9046200817 bytes, 8834197/8834197 kb. > 2011-07-27 11:38:30.360812 log 2011-07-27 11:38:28.540571 osd5 > 10.5.51.145:6803/19737 2843 : [ERR] 0.256 scrub 1 errors > > "Hmmm" I said to myself. > > stop osd5 > fsck -fy /dev/sdb1 > > And got: > > Inode 230558734, i_blocks is 8, should be 16. Fix? yes > Inode 230559279, i_blocks is 8208, should be 8200. Fix? yes > Inode 230559410, i_blocks is 40, should be 32. Fix? yes > Inode 230559527, i_blocks is 24, should be 32. Fix? yes > Inode 231342174, i_blocks is 40, should be 32. Fix? yes > Inode 231342209, i_blocks is 24, should be 32. Fix? yes > > Ups. Familiar message. :( > > I login to different node and got the same. > > Greg, I think to catch any errors no sense. 2.6.39 have broken ext4. I will > try 3.0.0 (or 3.1.0) and tell about result. Ah, I think that would explain the errors. We'll have to get better at identifying such things! -Greg -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
