responded inline On Tue, Sep 3, 2024 at 3:05 PM Alex Hussein-Kershaw (HE/HIM) < alexhus@xxxxxxxxxxxxx> wrote: > Hi Yuval, > > Thanks for the response. I did managed to disable the feature, however I > hope you can understand my hesitancy to design our move away from pubsub > onto a deprecated replacement (i.e. "notifications v1"). > > > agree. notification v2 provides better observability, e.g. list of buckets that use topic etc. > The difference between this and the other operations that require > forwarding to the master site, for me, is that I don't rely on any of those > operations for day-to-day function. For example, S3 user creation I do at > deployment time and never again; so if I lose the ability to do that in the > event of a site failure that's OK. Perhaps I just need to implement a > process to ensure the remaining zone is quickly promoted on a site failure. > > > this is probably the correct approach in multisite > A further concern of the new feature... My intention is to use the bucket > notification feature to send events generated by the multisite sync to my > application server. If the topic is now multisite, will all kicks be > duplicated? For example consider: > > SiteA receives an S3 write for object > SiteA writes to the event queue and asynchronously delivers the event > SiteB syncs object from siteA > we do not send regular notifications upon sync. note that we have other type of notifications: s3:ObjectSynced:*/s3:Replication:* that are sent only when an object is synced > SiteB writes to the event queue and asynchronously delivers the event > > Really I only want the second event in this example, which is what I get > with the pubsub module, and I think is what I will get with > "notifications_v1". > > Best Wishes, > Alex > ------------------------------ > *From:* Yuval Lifshitz <ylifshit@xxxxxxxxxx> > *Sent:* Tuesday, September 3, 2024 12:15 PM > *To:* Alex Hussein-Kershaw (HE/HIM) <alexhus@xxxxxxxxxxxxx> > *Cc:* ceph-users <ceph-users@xxxxxxx> > *Subject:* [EXTERNAL] Re: Bucket Notifications v2 & > Multisite Redundancy > > Hi Alex, > It should be possible to disable the v2 feature through an admin command. > radosgw-admin onegroup modify --disable-feature=notification_v2 > > Also note that when creating a new squid cluster, v2 is enabled by > default. But, when upgrading an existing cluster, you need to call: > radosgw-admin onegroup modify --enable-feature=notification_v2 > to enable v2. > > However, we do encourage moving to v2, as there are many benefits to the > new data model (besides site syncing). > > Regarding the problem you encountered when the "master" site is down, how > is this different from other operations that are forwarded to "master"? > > Yuval > > On Tue, Sep 3, 2024 at 11:31 AM Alex Hussein-Kershaw (HE/HIM) < > alexhus@xxxxxxxxxxxxx> wrote: > > Hi Folks, > > I see in the pending release notes for Squid a description of the > "notification_v2" feature. > ceph/PendingReleaseNotes at main · ceph/ceph (github.com)< > https://github.com/ceph/ceph/blob/main/PendingReleaseNotes#L43> > > I have some concerns about the multisite nature of this feature, I am > using Ceph multisite to provide geographic redundancy for my application > data. That seems to be a mainline use case. My application is designed to > cope if an entire zone fails. > > My application currently relies on pubsub (and soon to be replaced by > bucket notifications) to provide service. More specifically I rely on the > ability to semi-regularly automatically reconfigure topics to adjust the > HTTP endpoint. > > I believe with the "notification_v2" feature all adjustments of the topic > are routed through the master for metadata site (much like creating an S3 > user). Fundamentally I think this breaks the combination of Ceph multisite, > notifications_v2 and reconfiguring a topic; you cannot reconfigure a topic > unless both sites are up. > > I tested this, by setting up a deployment, shutting down a site and trying > to reconfigure a topic. I get a 500 HTTP response to my createTopic, and > can see this in the RGW logs: > debug 2024-09-03T08:24:39.997+0000 7f53a3372640 20 ERROR: curl error: > Couldn't connect to server req_data->error_buf=Failed to connect to > 10.225.20.19 port 7480: No route to host > debug 2024-09-03T08:24:39.997+0000 7f5329a7f640 4 req 2968544598970267460 > 0.686004043s sns:pubsub_topic_create CreateTopic forward_request_to_master > returned ret = -2200 > > Have the implications of this been considered? For now I may be able to > disable the notifications_v2 feature, however in the release note the > predecessor is deprecated in Squid. Hopefully I'm missing something obvious. > > Kind regards, > Alex > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |