On 12/10/2014 09:05 PM, Gregory Farnum wrote:
What version is he running?
Joao, does this make any sense to you?
From the MonMap code I'm pretty sure that the client should have built
the monmap from the [mon.X] sections, and solely based on 'mon addr'.
'mon_initial_members' is only useful to the monitors anyway, so it can
be disregarded.
Thus, there are two ways for a client to build a monmap:
1) based on 'mon_hosts' on the config (or -m on cli); or
2) based on 'mon addr = ip1,ip2...' from the [mon.X] sections
I don't see a 'mon hosts = ip1,ip2,...' on the config file, and I'm
assuming a '-m ip1,ip2...' has been supplied on the cli, so we would
have been left with the 'mon addr' options on each individual [mon.X]
section.
We are left with two options here: assume there was unexpected behavior
on this code path -- logs or steps to reproduce would be appreciated in
this case! -- or assume something else failed:
- are the ips on the remaining mon sections correct (nodo-1 && nodo-2)?
- were all the remaining monitors up and running when the failure occurred?
- were the remaining monitors reachable by the client?
In case you are able to reproduce this behavior, would be nice if you
could provide logs with 'debug monc = 10' and 'debug ms = 1'.
Cheers!
-Joao
-Greg
On Wed, Dec 10, 2014 at 11:54 AM, Christopher Armstrong
<chris@xxxxxxxxxxxx> wrote:
Thanks Greg - I thought the same thing, but confirmed with the user that it
appears the radosgw client is indeed using initial members - when he added
all of his hosts to initial members, things worked just fine. In either
event, all of the monitors were always fully enumerated later in the config
file. Is this potentially a bug specific to radosgw? Here's his config file:
[global]
fsid = fc0e2e09-ade3-4ff6-b23e-f789775b2515
mon initial members = nodo-3
auth cluster required = cephx
auth service required = cephx
auth client required = cephx
osd pool default size = 3
osd pool default min_size = 1
osd pool default pg_num = 128
osd pool default pgp_num = 128
osd recovery delay start = 15
log file = /dev/stdout
mon_clock_drift_allowed = 1
[mon.nodo-1]
host = nodo-1
mon addr = 192.168.2.200:6789
[mon.nodo-2]
host = nodo-2
mon addr = 192.168.2.201:6789
[mon.nodo-3]
host = nodo-3
mon addr = 192.168.2.202:6789
[client.radosgw.gateway]
host = deis-store-gateway
keyring = /etc/ceph/ceph.client.radosgw.keyring
rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
log file = /dev/stdout
On Wed, Dec 10, 2014 at 11:40 AM, Gregory Farnum <greg@xxxxxxxxxxx> wrote:
On Tue, Dec 9, 2014 at 3:11 PM, Christopher Armstrong
<chris@xxxxxxxxxxxx> wrote:
Hi folks,
I think we have a bit of confusion around how initial members is used. I
understand that we can specify a single monitor (or a subset of
monitors) so
that the cluster can form a quorum when it first comes up. This is how
we're
using the setting now - so the cluster can come up with just one
monitor,
with the other monitors to follow later.
However, a Deis user reported that when the monitor in his initial
members
list went down, radosgw stopped functioning, even though there are three
mons in his config file. I would think that the radosgw client would
connect
to any of the nodes in the config file to get the state of the cluster,
and
that the initial members list is only used when the monitors first come
up
and are trying to achieve quorum.
The issue he filed is here: https://github.com/deis/deis/issues/2711
He also found this Ceph issue filed:
https://github.com/ceph/ceph/pull/1233
Nope, this has nothing to do with it.
Is that what we're seeing here? Can anyone point us in the right
direction?
I didn't see the actual conf file posted anywhere to look at, but my
guess is simply that (since it looks like you're using generated conf
files which can differ across hosts) that the one on the server(s) in
question don't have the monitors listed in them. I'm only skimming the
code, but from it and my recollection, when a Ceph client starts up it
will try to assemble a list of monitors to contact from:
1) the contents of the "mon host" config entry
2) the "mon addr" value in any of the "global", "mon" or "mon.X" sections
The clients don't even look at mon_initial_members that I can see,
actually — so perhaps your client config only lists the initial
monitor, without adding the others?
-Greg
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
Joao Eduardo Luis
Software Engineer | http://ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
