Changing it to rx-tx seems to do the whole magic. After restarting rbd-mirror on the backup site the image was synced. Thanks a lot for your perfect and fast support. Solving problems this way is fun. Yves -----Ursprüngliche Nachricht----- Von: Jason Dillaman <jdillama@xxxxxxxxxx> Gesendet: Freitag, 24. Juli 2020 23:11 An: Yves Kretzschmar-Schwipper <yveskretzschmar@xxxxxx> Cc: dillaman <dillaman@xxxxxxxxxx>; ceph-users <ceph-users@xxxxxxx> Betreff: Re: Re: journal based mirroring works but snapshot based not On Fri, Jul 24, 2020 at 3:49 PM Yves Kretzschmar-Schwipper <yveskretzschmar@xxxxxx> wrote: > > Yes, there are some: > > 2020-07-24T21:46:25.974+0200 7f932fccd700 10 rbd::mirror::RemotePollPoller: 0x5628339d92b0 handle_task: > 2020-07-24T21:46:25.974+0200 7f93304ce700 10 rbd::mirror::RemotePollPoller: 0x5628339d92b0 get_mirror_uuid: > 2020-07-24T21:46:25.978+0200 7f931dca9700 10 > rbd::mirror::RemotePollPoller: 0x5628339d92b0 handle_get_mirror_uuid: > r=0 > 2020-07-24T21:46:25.978+0200 7f931dca9700 10 > rbd::mirror::RemotePollPoller: 0x5628339d92b0 handle_get_mirror_uuid: > remote_mirror_uuid=ce9e8f12-c8e2-4c16-bd74-f28f29d405a3 > 2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller: 0x5628339d92b0 mirror_peer_ping: > 2020-07-24T21:46:25.978+0200 7f931dca9700 10 > rbd::mirror::RemotePollPoller: 0x5628339d92b0 handle_mirror_peer_ping: > r=0 > 2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller: 0x5628339d92b0 mirror_peer_list: > 2020-07-24T21:46:25.978+0200 7f931dca9700 10 > rbd::mirror::RemotePollPoller: 0x5628339d92b0 handle_mirror_peer_list: > r=0 > 2020-07-24T21:46:25.978+0200 7f931dca9700 10 > rbd::mirror::RemotePollPoller: 0x5628339d92b0 handle_mirror_peer_list: > remote_mirror_peer_uuid= Yeah, it's not finding itself (by site-name) in the remote peer cluster. Presuming this was a log from your "backup" site's rbd-mirror daemon, I think the issue is that your peer record on "master" for "backup" is listed as a "tx-only" peer, which means that "backup" will only transmit images to "master". Can you try changing the peer direction on "master" to "rx-only" (i.e. it will only receive images from "master") or "rx-tx" for bi-directional mirroring? > 2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller: 0x5628339d92b0 schedule_task: > > > -----Ursprüngliche Nachricht----- > Von: Jason Dillaman <jdillama@xxxxxxxxxx> > Gesendet: Freitag, 24. Juli 2020 21:34 > An: Yves Kretzschmar-Schwipper <yveskretzschmar@xxxxxx> > Cc: ceph-users <ceph-users@xxxxxxx> > Betreff: Re: Re: journal based mirroring works but > snapshot based not > > On Fri, Jul 24, 2020 at 3:22 PM <yveskretzschmar@xxxxxx> wrote: > > > > Hi Jason, > > > > now I'm a bit confused ... > > > > I recreated the peers again with names clear: > > > > #main site: > > rbd mirror pool info testpool > > Mode: image > > Site Name: master > > Peer Sites: > > UUID: e7443ebd-b9e2-47ab-bcfe-f40cb75a6294 > > Name: backup > > Mirror UUID: e0d03424-1913-4e0b-90dd-61defabae51a > > Direction: tx-only > > > > #backup site: > > rbd --cluster backup mirror pool info testpool > > Mode: image > > Site Name: backup > > Peer Sites: > > UUID: a7ffac0b-09e3-427c-a3a1-5f001a55dcf1 > > Name: master > > Mirror UUID: > > Direction: rx-tx > > Client: client.admin > > > > > > In my opinion this looks ok, but the error seems the same: > > > > 2020-07-24T21:14:21.559+0200 7fb9d709a700 -1 > > rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800 init: > > failed to retrieve mirror peer uuid from remote pool > > 2020-07-24T21:14:21.559+0200 7fb9e28b1700 -1 > > rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800 > > get_replay_status: replay not running > > 2020-07-24T21:14:22.515+0200 7fb9e28b1700 5 rbd::mirror::Instances: > > 0x56122af49d40 handle_acked: instance_ids=[10464751] > > 2020-07-24T21:14:27.519+0200 7fb9e28b1700 5 rbd::mirror::Instances: > > 0x56122af49d40 handle_acked: instance_ids=[10464751] > > 2020-07-24T21:14:31.559+0200 7fb9e28b1700 -1 > > rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800 > > get_replay_status: replay not running > > 2020-07-24T21:14:32.519+0200 7fb9e28b1700 5 rbd::mirror::Instances: > > 0x56122af49d40 handle_acked: instance_ids=[10464751] > > Turn your rbd-mirror debug level up from 5 to at least 10 and retry. > Every 30 seconds or so there should be a log message from the "RemotePoolPoller" which is responsible for pulling the mirror peer uuid from the remote cluster. > > > > > If I disable mirroring on the testimage on main site, the mirror daemon tells me: > > > > 2020-07-24T21:15:58.775+0200 7f3cd6f3b700 -1 > > rbd::mirror::image_replayer::snapshot::Replayer: 0x55fbad17c800 > > handle_unregister_remote_update_watcher: failed to unregister remote > > update watcher: (2) No such file or directory > > 2020-07-24T21:15:58.775+0200 7f3cd6f3b700 -1 > > rbd::mirror::image_replayer::snapshot::Replayer: 0x55fbad17c800 > > handle_unregister_local_update_watcher: failed to unregister local > > update watcher: (2) No such file or directory > > > > The created image on the backup site doesn't get removed. > > > > > > Journal based mirroring works without any problem on the same image. > > > > > > Thanks for any hints > > Yves > > > > > > > > > > -----Ursprüngliche Nachricht----- > > Betreff: Re: Re: Re: journal based mirroring works but > > snapshot based not > > > > You need to have unique names for your site -- so "ceph" is not an appropriate name. Right now, your backup set is named "ceph" and you have a peer named "ceph". > > > > On Fri, Jul 24, 2020 at 8:43 AM Yves wrote: > > > > > > Runing "rbd mirror pool peer set testpool 0622b647-2c24-4840-93d0-efdee180ab15 site-name backup" > > > get's me: > > > 2020-07-24T14:07:57.205+0200 7fc414af53c0 -1 librbd::api::Mirror: > > > peer_site_set_name: failed to update site > > > '0622b647-2c24-4840-93d0-efdee180ab15': (17) File exists > > > > > > So I deleted peers again and recreated with same site-name. > > > > > > On main site it looks this way: > > > rbd mirror pool info testpool > > > Mode: image > > > Site Name: ceph > > > Peer Sites: > > > UUID: e68b09de-1d2c-4ec6-9350-a6ccad26e1b7 > > > Name: ceph > > > Mirror UUID: 4d7f87f4-47be-46dd-85f1-79caa3fa23da > > > Direction: tx-only > > > > > > > > > On backup site: > > > rbd --cluster backup mirror pool info testpool > > > Mode: image > > > Site Name: ceph > > > Peer Sites: > > > UUID: 399e902c-deee-498c-bcdc-1c4f7a59433a > > > Name: ceph > > > Mirror UUID: > > > Direction: rx-tx > > > Client: client.admin > > > > > > > > > But situation didn't change. > > > > > > > > > Gesendet: Freitag, 24. Juli 2020 um 14:04 Uhr > > > Von: "Jason Dillaman" > > > Cc: "ceph-users" <ceph-users@xxxxxxx> > > > > > > > > Hi, > > > > > > > > this is the main site: > > > > > > > > rbd mirror pool info testpool > > > > Mode: image > > > > Site Name: ceph > > > > > > > > Peer Sites: > > > > > > > > UUID: 1f1877cb-5753-4a0e-8b8c-5e5547c0619e > > > > Name: backup > > > > Mirror UUID: e9e2c4a0-1900-4db6-b828-e655be5ed9d8 > > > > Direction: tx-only > > > > > > > > > > > > and the backup site: > > > > > > > > rbd --cluster backup mirror pool info testpool > > > > Mode: image > > > > Site Name: backup > > > > > > > > Peer Sites: > > > > > > > > UUID: 0622b647-2c24-4840-93d0-efdee180ab15 > > > > Name: ceph > > > > Direction: rx-only > > > > Client: client.rbd-mirror-peer > > > > > > At the backup site, run "rbd mirror pool peer set testpool > > > 0622b647-2c24-4840-93d0-efdee180ab15 site-name backup" to update > > > the site-name to match. > > > > > > > _______________________________________________ > > > > ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe > > > > send an email to ceph-users-leave@xxxxxxx > > > > > > > > > > > > > -- > > > Jason > > > > > _______________________________________________ > > ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an > > email to ceph-users-leave@xxxxxxx > > > > -- > Jason > > -- Jason _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |