Hi Sam,
here the crushmap
http://85.214.49.87/ceph/crushmap.txt
http://85.214.49.87/ceph/crushmap
-martin
Samuel Just schrieb:
It looks like a crushmap related problem. Could you send us the crushmap?
ceph osd getcrushmap
Thanks
-Sam
On Fri, Nov 18, 2011 at 10:13 AM, Gregory Farnum
<gregory.farnum@xxxxxxxxxxxxx> wrote:
On Fri, Nov 18, 2011 at 10:05 AM, Tommi Virtanen
<tommi.virtanen@xxxxxxxxxxxxx> wrote:
On Thu, Nov 17, 2011 at 12:48, Martin Mailand <martin@xxxxxxxxxxxx> wrote:
Hi,
I am doing cluster failure test, where I shut down one OSD an wait for the
cluster to sync. But the sync never finshed, at around 4-5% it stops. I
stoped osd2.
...
2011-11-17 16:42:45.520740 pg v1337: 600 pgs: 547 active+clean, 53
active+clean+degraded; 113 GB data, 184 GB used, 1141 GB / 1395 GB avail;
4025/82404 degraded (4.884%)
...
The osd log, the ceph.conf, pg dump, osd dump could be found here.
http://85.214.49.87/ceph/
This looks a bit worrying:
2011-11-17 17:56:35.771574 7f704c834700 -- 192.168.42.113:0/2424 >>
192.168.42.114:6802/21115 pipe(0x2596c80 sd=17 pgs=0 cs=0 l=0).connect
claims to be 192.168.42.114:6802/21507 not 192.168.42.114:6802/21115 -
wrong node!
So osd.0 is basically refusing to talk to one of the other OSDs. I
don't understand the messenger well enough to know why this would be,
but it wouldn't surprise me if this problem kept the objects degraded
-- it looks like a breakage in the osd<->osd communication.
Now if this was the reason, I'd expect a restart of all the OSDs to
get it back in shape; messenger state is ephemeral. Can you confirm
that?
Probably not — that wrong node thing can occur for a lot of different
reasons, some of which matter and most of which don't. Sam's looking
into the problem; there's something going wrong with the CRUSH
calculations or the monitor PG placement overrides or something...
-Greg
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
