Thanks for clarification > according to what i tested, this is not the case. deletion of a topic only > prevents the creation of new notifications with that topic. > it does not effect the deletion of notifications with that topic, not the > actual sending of these notifications. > > note that we also added a cascade delete process to delete all > notifications of a bucket when a bucket is deleted. > (it should be in pacific: https://github.com/ceph/ceph/pull/38351) > You are right, this is cleaned up as expected. I re-ran my tests and it turned out problems show up after deleting the internal topic (created upon creation of notification). When I did that initially I didn't know about internal topics and in my eagerness I deleted all topics to start over. This triggered an issue with deleting notifications. In my case with topics as below # radosgw-admin topic list --tenant marvel --uid none { "topics": [ { "topic": { "user": "marvel", "name": "MyTopic", "dest": { "bucket_name": "", "oid_prefix": "", "push_endpoint": "amqp://127.0.0.1", "push_endpoint_args": "amqp-exchange=rgw-exchange&push-endpoint=amqp://127.0.0.1 &use-ssl=false&verify-ssl=false", "push_endpoint_topic": "MyTopic", "stored_secret": "false", "persistent": "false" }, "arn": "arn:aws:sns:default:marvel:MyTopic", "opaqueData": "" }, "subs": [] }, { "topic": { "user": "marvel", "name": "bucket-by-thanos_notification_MyTopic", "dest": { "bucket_name": "", "oid_prefix": "", "push_endpoint": "amqp://127.0.0.1", "push_endpoint_args": "amqp-exchange=rgw-exchange&push-endpoint=amqp://127.0.0.1 &use-ssl=false&verify-ssl=false", "push_endpoint_topic": "MyTopic", "stored_secret": "false", "persistent": "false" }, "arn": "arn:aws:sns:default:marvel:MyTopic", "opaqueData": "" }, "subs": [] } ] } Once I delete both of those and attempt to delete the notification on the bucket, RGW logs as below. Returned status code is 200 but notification can still be listed. Log debug 2021-06-25T08:36:59.823+0000 7fefed33a700 10 cache get: name=my-rgw.rgw.log++pubsub.marvel.bucket.bucket-by-thanos/103132f8-3d78-4592-ae7b-067ecf2a1b92.9081601.36 : hit (requested=0x6, cached=0x17) debug 2021-06-25T08:36:59.823+0000 7fefed33a700 20 get_system_obj_state: s->obj_tag was set empty debug 2021-06-25T08:36:59.823+0000 7fefed33a700 10 cache get: name=my-rgw.rgw.log++pubsub.marvel.bucket.bucket-by-thanos/103132f8-3d78-4592-ae7b-067ecf2a1b92.9081601.36 : hit (requested=0x1, cached=0x17) debug 2021-06-25T08:36:59.823+0000 7fefed33a700 20 get_system_obj_state: rctx=0x558dd865b1c0 obj=my-rgw.rgw.log:pubsub.marvel state=0x558dd5ff9720 s->prefetch_data=0 debug 2021-06-25T08:36:59.823+0000 7fefed33a700 10 cache get: name=my-rgw.rgw.log++pubsub.marvel : hit (requested=0x16, cached=0x17) debug 2021-06-25T08:36:59.823+0000 7fefed33a700 20 get_system_obj_state: s->obj_tag was set empty debug 2021-06-25T08:36:59.823+0000 7fefed33a700 10 cache get: name=my-rgw.rgw.log++pubsub.marvel : hit (requested=0x11, cached=0x17) debug 2021-06-25T08:36:59.823+0000 7fefed33a700 10 distributing notification oid=my-rgw.rgw.control:notify.0 bl.length()=189 debug 2021-06-25T08:36:59.823+0000 7ff108570700 10 RGWWatcher::handle_notify() notify_id 3573413718119 cookie 94067650647552 notifier 14638216 bl.length()=189 debug 2021-06-25T08:36:59.831+0000 7fefe3b27700 20 get_system_obj_state: rctx=0x558dd865b1c0 obj=my-rgw.rgw.log:pubsub.marvel state=0x558dd5ff9720 s->prefetch_data=0 debug 2021-06-25T08:36:59.831+0000 7fefe3b27700 10 cache get: name=my-rgw.rgw.log++pubsub.marvel : hit (requested=0x1, cached=0x17) debug 2021-06-25T08:36:59.839+0000 7fefe3b27700 1 ERROR: topic not found debug 2021-06-25T08:36:59.839+0000 7fefe3b27700 1 ERROR: failed to read topic info: ret=-2 debug 2021-06-25T08:36:59.839+0000 7fefe3b27700 1 failed to remove notification of topic 'bucket-by-thanos_notification_MyTopic', ret=-2 debug 2021-06-25T08:36:59.839+0000 7fefe3b27700 20 get_system_obj_state: rctx=0x558dd865b1c0 obj=my-rgw.rgw.log:pubsub.marvel state=0x558dd5ff9720 s->prefetch_data=0 debug 2021-06-25T08:36:59.839+0000 7fefe3b27700 10 cache get: name=my-rgw.rgw.log++pubsub.marvel : hit (requested=0x11, cached=0x17) debug 2021-06-25T08:36:59.843+0000 7ff007b6f700 10 cache put: name=my-rgw.rgw.log++pubsub.marvel info.flags=0x17 debug 2021-06-25T08:36:59.843+0000 7ff007b6f700 10 moving my-rgw.rgw.log++pubsub.marvel to cache LRU end debug 2021-06-25T08:36:59.843+0000 7ff007b6f700 10 distributing notification oid=my-rgw.rgw.control:notify.4 bl.length()=189 debug 2021-06-25T08:36:59.843+0000 7ff108d71700 10 RGWWatcher::handle_notify() notify_id 3573413545888 cookie 94067664846848 notifier 14638216 bl.length()=189 debug 2021-06-25T08:36:59.843+0000 7ff108d71700 10 cache put: name=my-rgw.rgw.log++pubsub.marvel info.flags=0x17 debug 2021-06-25T08:36:59.843+0000 7ff108d71700 10 moving my-rgw.rgw.log++pubsub.marvel to cache LRU end debug 2021-06-25T08:36:59.843+0000 7feff6b4d700 2 req 329 0.019999718s s3:pubsub_notification_delete_s3 completing debug 2021-06-25T08:36:59.843+0000 7feff6b4d700 20 get_system_obj_state: rctx=0x7ff122a72750 obj=my-rgw.rgw.log:script.postrequest.marvel state=0x558dd5ff9720 s->prefetch_data=0 debug 2021-06-25T08:36:59.843+0000 7feff6b4d700 10 cache get: name=my-rgw.rgw.log++script.postrequest.marvel : hit (negative entry) debug 2021-06-25T08:36:59.843+0000 7feff6b4d700 2 req 329 0.019999718s s3:pubsub_notification_delete_s3 op status=0 debug 2021-06-25T08:36:59.847+0000 7feff6b4d700 2 req 329 0.023999661s s3:pubsub_notification_delete_s3 http status=200 debug 2021-06-25T08:36:59.847+0000 7feff6b4d700 1 ====== req done req=0x7ff122a73620 op status=0 http_status=200 latency=0.023999661s ====== debug 2021-06-25T08:36:59.847+0000 7feff6b4d700 1 beast: 0x7ff122a73620: 11.1.150.14 - marvel$oidc$f1be270a-d6a0-4f97-b444-c1f23816b25a [25/Jun/2021:08:36:59.823 +0000] "DELETE /bucket-by-thanos?notification=bucket-by-thanos_notification HTTP/1.1" 200 0 - "PostmanRuntime/7.28.0" - Thanks Daniel _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |