Hi Eugen, Thanks for the suggestions. It has worked for me before. It's certainly possible it's a misconfiguration, however I've reproduced this on upgrade of some long lived systems that have been happily syncing away on Octopus for several years. Definitely keen to understand if I'm missing something in the configuration. I am currently combing through network captures of the sync traffic trying to figure out the difference. It's also odd that my "sync status" command is not reporting an error but is claiming we are in sync, despite that not being the case; my best guess at the moment is that the uplevel version is not correctly checking the downlevels logs. I know a lot of refactoring of the multisite sync process has been done recently so I wonder if that may be related. For reference, I raised this tracker: Bug #68819: rgw: multisite sync between Squid and Quincy does not work in one direction - rgw - Ceph<https://tracker.ceph.com/issues/68819>. Best wishes, Alex ________________________________ From: Eugen Block <eblock@xxxxxx> Sent: Wednesday, November 6, 2024 11:35 AM To: ceph-users@xxxxxxx <ceph-users@xxxxxxx> Subject: [EXTERNAL] Re: Ceph Multisite Version Compatibility Hi Alex, I don't have a real good answer, just wanted to mention that one of our customers had some issues with multi-site when they were on the same major version (Octopus) but not on the same minor version. But it wasn't that the sync didn't work at all, it worked in general. Only from time to time, the sync status would show errors. And after updating the second site to the same minor version, the issues never came back. But from what I read in this list, it doesn't appear to be a general problem with version mismatch, so I wouldn't expect one sync direction to fail entirely. Maybe it's a configuration issue? Has it ever worked before or have you just set up the second site and it failed right from the start? Zitat von "Alex Hussein-Kershaw (HE/HIM)" <alexhus@xxxxxxxxxxxxx>: > I wondered if this applies: Ceph Releases (general) — Ceph > Documentation<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.ceph.com%2Fen%2Flatest%2Freleases%2Fgeneral%2F&data=05%7C02%7Calexhus%40microsoft.com%7C62ba60cb463c4dd1ef7f08dcfe57410e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638664897954416415%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=%2Fbf7xrHmMZE%2BaOGYRsp9WZgVtmEdxi2iAxKdV1hMLSY%3D&reserved=0<https://docs.ceph.com/en/latest/releases/general/>>. > > "Online, rolling upgrade support and testing from the last two (2) > stable release(s) (starting from Luminous)." - which does imply I'm > doing something invalid with one site on Squid and one on Octopus. > > However, I've reproduced this between Quincy (17.2.7) and Squid > (19.2.0) now too, which according to the link above is a valid > upgrade path. To be clear, I have: > > * > SiteA (Quincy) < --- syncing --- > SiteB (Squid). > * > Write objects to SiteB, they appear on siteA shortly after. > * > Write objects to SiteA, they never appear on siteB. > > It seems to be 100% reproducible. Suspect I need to raise a tracker. > I welcome any suggestions that I'm doing this wrong meanwhile. > > ________________________________ > From: Alex Hussein-Kershaw (HE/HIM) > Sent: Friday, November 1, 2024 8:49 AM > To: ceph-users <ceph-users@xxxxxxx> > Subject: Ceph Multisite Version Compatibility > > Hi folks. I'm looking for some guidance on RGW multisite version > sync compatibility. Particularly between Octopus and Squid. Context > is I have two sites in a multisite pair replicating all S3 data. One > is on Squid, one is on Octopus. Should I expect the multisite sync > to just work between these versions? > > I'm observing that both sites "radosgw-admin sync status" reports > that we're in sync, but objects from the Octopus zone are not > replicated to the Squid zone (but the opposite direction is fine). > Might be this just isn't a valid setup, but failing to a reference > that claims something like "must be within +/- 1 version of the > other zones"). > > Thanks, > Alex > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |