Le 03/07/2012 21:42, Tommi Virtanen a écrit :
On Tue, Jul 3, 2012 at 1:40 AM, Yann Dupont <Yann.Dupont@xxxxxxxxxxxxxx> wrote:
Upgraded the kernel to 3.5.0-rc4 + some patches, seems btrfs is OK right
now.
Tried to restart osd with 0.47.3, then next branch, and today with 0.48.
4 of 8 nodes fails with the same message :
ceph version 0.48argonaut (commit:c2b20ca74249892c8e5e40c12aa14446a2bf2030)
1: /usr/bin/ceph-osd() [0x701929]
...
13: (leveldb::InternalKeyComparator::FindShortestSeparator(std::string*,
leveldb::Slice const&) const+0x4d) [0x6e811d]
That looks like http://tracker.newdream.net/issues/2563 and the best
we have for that ticket is "looks like you have a corrupted leveldb
file". Is this reproducible with a freshly mkfs'ed data partition?
Probably not. I have multiple data volumes on each nodes (I was planning
xfs vs ext4 vs btrfs benchmarks before being ill) and thoses nodes start
OK with another data partition .
It's very probable that there is corruption somewhere, due to kernel bug
, probably triggered by btrfs.
Issue 2563 is probably the same.
I'd like to restart those nodes without formatting them, not because the
data is valuable, but because if the same thing happens in production, a
method similar to "fsck" the node could be of great value.
I saw the method to check the leveldb. Will try tomorrow without garantees.
In the case I could repair, do you think a crashed FS as it is right now
is valuable for you, for future reference , as I saw you can't reproduce
the problem ? I can make an archive (or a btrfs dump ?), but it will be
quite big.
Cheers,
--
Yann Dupont - Service IRTS, DSI Université de Nantes
Tel : 02.53.48.49.20 - Mail/Jabber : Yann.Dupont@xxxxxxxxxxxxxx
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
