Hi Magnus,
Yes this is a workaroudn for the problem. However this means that if you want to have your rbd-mirroring daemon HA, you will need to create 2+ more machines in your infrastructure instead
of being able to collocate it on the same machines as your MDS,MGR and MON.
Best Regards
Jocelyn Thode
From: Magnus Grönlund [mailto:magnus@xxxxxxxxxxx]
Sent: jeudi, 9 août 2018 14:33
To: Thode Jocelyn <jocelyn.thode@xxxxxxx>
Cc: Erik McCormick <emccormick@xxxxxxxxxxxxxxx>; ceph-users@xxxxxxxxxxxxxx
Subject: Re: [ceph-users] [Ceph-deploy] Cluster Name
Hi Jocelyn,
I'm in the process of setting up rdb-mirroring myself and stumbled on the same problem. But I think that the "trick" here is to _not_ colocate the RDB-mirror daemon with any other part of the cluster(s), it should be run on a separate host.
That way you can change the CLUSTER_NAME variable in /etc/sysconfig/ceph without affecting any of the mons, osd etc.
2018-08-09 7:41 GMT+02:00 Thode Jocelyn <jocelyn.thode@xxxxxxx>:
Hi Erik,
The thing is that the rbd-mirror service uses the /etc/sysconfig/ceph file to determine which configuration file to use (from CLUSTER_NAME). So you need to set this to the name you chose for rbd-mirror to work. However setting this CLUSTER_NAME variable in
/etc/sysconfig/ceph makes it so that the mon, osd etc services will also use this variable. Because of this they cannot start anymore as all their path are set with "ceph" as cluster name.
However there might be something that I missed which would make this point moot
Best Regards
Jocelyn Thode
-----Original Message-----
From: Erik McCormick [mailto:emccormick@xxxxxxxxxxxxxxx]
Sent: mercredi, 8 août 2018 16:39
To: Thode Jocelyn <jocelyn.thode@xxxxxxx>
Cc: Glen Baars <glen@xxxxxxxxxxxxxxxxxxxxxx>; Vasu Kulkarni <vakulkar@xxxxxxxxxx>;
ceph-users@xxxxxxxxxxxxxx
Subject: Re: [ceph-users] [Ceph-deploy] Cluster Name
I'm not using this feature, so maybe I'm missing something, but from the way I understand cluster naming to work...
I still don't understand why this is blocking for you. Unless you are attempting to mirror between two clusters running on the same hosts (why would you do this?) then systemd doesn't come into play. The --cluster flag on the rbd command will simply set the
name of a configuration file with the FSID and settings of the appropriate cluster. Cluster name is just a way of telling ceph commands and systemd units where to find the configs.
So, what you end up with is something like:
/etc/ceph/ceph.conf (your local cluster configuration) on both clusters /etc/ceph/local.conf (config of the source cluster. Just a copy of ceph.conf of the source clsuter) /etc/ceph/remote.conf (config of destination peer cluster. Just a copy of ceph.conf of
the remote cluster).
Run all your rbd mirror commands against local and remote names.
However when starting things like mons, osds, mds, etc. you need no cluster name as it can use ceph.conf (cluster name of ceph).
Am I making sense, or have I completely missed something?
-Erik
On Wed, Aug 8, 2018 at 8:34 AM, Thode Jocelyn <jocelyn.thode@xxxxxxx> wrote:
> Hi,
>
>
>
> We are still blocked by this problem on our end. Glen did you or
> someone else figure out something for this ?
>
>
>
> Regards
>
> Jocelyn Thode
>
>
>
> From: Glen Baars [mailto:glen@xxxxxxxxxxxxxxxxxxxxxx]
> Sent: jeudi, 2 août 2018 05:43
> To: Erik McCormick <emccormick@xxxxxxxxxxxxxxx>
> Cc: Thode Jocelyn <jocelyn.thode@xxxxxxx>; Vasu Kulkarni
> <vakulkar@xxxxxxxxxx>;
ceph-users@xxxxxxxxxxxxxx
> Subject: RE: [ceph-users] [Ceph-deploy] Cluster Name
>
>
>
> Hello Erik,
>
>
>
> We are going to use RBD-mirror to replicate the clusters. This seems
> to need separate cluster names.
>
> Kind regards,
>
> Glen Baars
>
>
>
> From: Erik McCormick <emccormick@xxxxxxxxxxxxxxx>
> Sent: Thursday, 2 August 2018 9:39 AM
> To: Glen Baars <glen@xxxxxxxxxxxxxxxxxxxxxx>
> Cc: Thode Jocelyn <jocelyn.thode@xxxxxxx>; Vasu Kulkarni
> <vakulkar@xxxxxxxxxx>;
ceph-users@xxxxxxxxxxxxxx
> Subject: Re: [ceph-users] [Ceph-deploy] Cluster Name
>
>
>
> Don't set a cluster name. It's no longer supported. It really only
> matters if you're running two or more independent clusters on the same
> boxes. That's generally inadvisable anyway.
>
>
>
> Cheers,
>
> Erik
>
>
>
> On Wed, Aug 1, 2018, 9:17 PM Glen Baars <glen@xxxxxxxxxxxxxxxxxxxxxx> wrote:
>
> Hello Ceph Users,
>
> Does anyone know how to set the Cluster Name when deploying with
> Ceph-deploy? I have 3 clusters to configure and need to correctly set
> the name.
>
> Kind regards,
> Glen Baars
>
> -----Original Message-----
> From: ceph-users <ceph-users-bounces@xxxxxxxxxxxxxx> On Behalf Of Glen
> Baars
> Sent: Monday, 23 July 2018 5:59 PM
> To: Thode Jocelyn <jocelyn.thode@xxxxxxx>; Vasu Kulkarni
> <vakulkar@xxxxxxxxxx>
> Cc: ceph-users@xxxxxxxxxxxxxx
> Subject: Re: [ceph-users] [Ceph-deploy] Cluster Name
>
> How very timely, I am facing the exact same issue.
>
> Kind regards,
> Glen Baars
>
> -----Original Message-----
> From: ceph-users <ceph-users-bounces@xxxxxxxxxxxxxx> On Behalf Of
> Thode Jocelyn
> Sent: Monday, 23 July 2018 1:42 PM
> To: Vasu Kulkarni <vakulkar@xxxxxxxxxx>
> Cc: ceph-users@xxxxxxxxxxxxxx
> Subject: Re: [ceph-users] [Ceph-deploy] Cluster Name
>
> Hi,
>
> Yes my rbd-mirror is coloctaed with my mon/osd. It only affects nodes
> where they are collocated as they all use the "/etc/sysconfig/ceph"
> configuration file.
>
> Best
> Jocelyn Thode
>
> -----Original Message-----
> From: Vasu Kulkarni [mailto:vakulkar@xxxxxxxxxx]
> Sent: vendredi, 20 juillet 2018 17:25
> To: Thode Jocelyn <jocelyn.thode@xxxxxxx>
> Cc: ceph-users@xxxxxxxxxxxxxx
> Subject: Re: [ceph-users] [Ceph-deploy] Cluster Name
>
> On Fri, Jul 20, 2018 at 7:29 AM, Thode Jocelyn <jocelyn.thode@xxxxxxx>
> wrote:
>> Hi,
>>
>>
>>
>> I noticed that in commit
>>
https://github.com/ceph/ceph-deploy/commit/b1c27b85d524f2553af2487a98
>> 0 23b60efe421f3, the ability to specify a cluster name was removed.
>> Is there a reason for this removal ?
>>
>>
>>
>> Because right now, there are no possibility to create a ceph cluster
>> with a different name with ceph-deploy which is a big problem when
>> having two clusters replicating with rbd-mirror as we need different
>> names.
>>
>>
>>
>> And even when following the doc here:
>>
https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/3/
>> h
>> tml/block_device_guide/block_device_mirroring#rbd-mirroring-clusters-
>> w
>> ith-the-same-name
>>
>>
>>
>> This is not sufficient as once we change the CLUSTER variable in the
>> sysconfig file, mon,osd, mds etc. all use it and fail to start on a
>> reboot as they then try to load data from a path in /var/lib/ceph
>> containing the cluster name.
>
> Is you rbd-mirror client also colocated with mon/osd? This needs to be
> changed only on the client side where you are doing mirroring, rest of
> the nodes are not affected?
>
>
>>
>>
>>
>> Is there a solution to this problem ?
>>
>>
>>
>> Best Regards
>>
>> Jocelyn Thode
>>
>>
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@xxxxxxxxxxxxxx
>>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> This e-mail is intended solely for the benefit of the addressee(s) and
> any other named recipient. It is confidential and may contain legally
> privileged or confidential information. If you are not the recipient,
> any use, distribution, disclosure or copying of this e-mail is
> prohibited. The confidentiality and legal privilege attached to this
> communication is not waived or lost by reason of the mistaken
> transmission or delivery to you. If you have received this e-mail in error, please notify us immediately.
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> This e-mail is intended solely for the benefit of the addressee(s) and
> any other named recipient. It is confidential and may contain legally
> privileged or confidential information. If you are not the recipient,
> any use, distribution, disclosure or copying of this e-mail is
> prohibited. The confidentiality and legal privilege attached to this
> communication is not waived or lost by reason of the mistaken
> transmission or delivery to you. If you have received this e-mail in error, please notify us immediately.
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
> This e-mail is intended solely for the benefit of the addressee(s) and
> any other named recipient. It is confidential and may contain legally
> privileged or confidential information. If you are not the recipient,
> any use, distribution, disclosure or copying of this e-mail is
> prohibited. The confidentiality and legal privilege attached to this
> communication is not waived or lost by reason of the mistaken
> transmission or delivery to you. If you have received this e-mail in error, please notify us immediately.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
|
