Thanks Jason for great suggestions.
but somehow rbd mirror status not working from secondary to primary. Here;s the status from both sides. cluster name is ceph on primary side and cephdr on remote site. mirrordr is the user on DR side and mirrorprod is on primary prod side.
# rbd mirror pool info nfs
Mode: image
Peers:
UUID NAME CLIENT
3ccd7a67-2343-44bf-960b-02d9b1258371 cephdr client.mirrordr.
rbd --cluster cephdr mirror pool info nfs
Mode: image
Peers:
UUID NAME CLIENT
e6b9ba05-48de-462c-ad5f-0b51d0ee733f ceph client.mirrorprod
From primary site, when i query the remote site, its looks good.
# rbd --cluster cephdr --id mirrordr mirror pool status nfs
health: OK
images: 0 total
but when i query from secondary site to primary side, I'm getting this error
# rbd --cluster ceph --id mirrorprod mirror pool status nfs
2018-10-03 10:21:06.645903 7f27a44ed700 0 -- 165.x.x.202:0/1310074448 >> 192.168.3.21:6804/3835 pipe(0x55ed47daf480 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x55ed47db0740).fault
We were using 192.168.3.x for MON network before we renamed it to use 165 address since its routeable. why its trying to connect to 192.x address instead of 165.x.y address?
I could do ceph -s from both side and they can see each other. Only rbd command is having issue.
Thanks,
-Vikas
On Tue, Oct 2, 2018 at 5:14 PM Jason Dillaman <jdillama@xxxxxxxxxx> wrote:
On Tue, Oct 2, 2018 at 4:47 PM Vikas Rana <vikasrana3@xxxxxxxxx> wrote:
>
> Hi,
>
> We have a CEPH 3 node cluster at primary site. We created a RBD image and the image has about 100TB of data.
>
> Now we installed another 3 node cluster on secondary site. We want to replicate the image at primary site to this new cluster on secondary site.
>
> As per documentation, we enabled journaling on primary site. We followed all the procedure and peering looks good but the image is not copying.
> The status is always showing down.
Do you have an "rbd-mirror" daemon running on the secondary site? Are
you running "rbd mirror pool status" against the primary site or the
secondary site? The mirroring status is only available on the sites
running "rbd-mirror" daemon (the "down" means that the cluster you are
connected to doesn't have the daemon running).
> So my question is, is it possible to replicate a image which already have some data before enabling journalling?
Indeed -- it will perform a full image sync to the secondary site.
> We are using the image mirroring instead of pool mirroring. Do we need to create the RBD image on secondary site? As per documentation, its not required.
The only difference between the two modes is whether or not you need
to run "rbd mirror image enable" or not.
> Is there any other option to copy the image to the remote site?
No other procedure should be required.
> Thanks,
> -Vikas
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
Jason
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
