Re: RGW topic created in wrong (default) tenant

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Thanks for clarification


> according to what i tested, this is not the case. deletion of a topic only
> prevents the creation of new notifications with that topic.
> it does not effect the deletion of notifications with that topic, not the
> actual sending of these notifications.
>
> note that we also added a cascade delete process to delete all
> notifications of a bucket when a bucket is deleted.
> (it should be in pacific: https://github.com/ceph/ceph/pull/38351)
>

You are right, this is cleaned up as expected.
I re-ran my tests and it turned out problems show up after deleting the
internal topic (created upon creation of notification).
When I did that initially I didn't know about internal topics and in my
eagerness I deleted all topics to start over. This triggered an issue with
deleting notifications.

In my case with topics as below

# radosgw-admin topic list --tenant marvel --uid none
{
    "topics": [
        {
            "topic": {
                "user": "marvel",
                "name": "MyTopic",
                "dest": {
                    "bucket_name": "",
                    "oid_prefix": "",
                    "push_endpoint": "amqp://127.0.0.1",
                    "push_endpoint_args":
"amqp-exchange=rgw-exchange&push-endpoint=amqp://127.0.0.1
&use-ssl=false&verify-ssl=false",
                    "push_endpoint_topic": "MyTopic",
                    "stored_secret": "false",
                    "persistent": "false"
                },
                "arn": "arn:aws:sns:default:marvel:MyTopic",
                "opaqueData": ""
            },
            "subs": []
        },
        {
            "topic": {
                "user": "marvel",
                "name": "bucket-by-thanos_notification_MyTopic",
                "dest": {
                    "bucket_name": "",
                    "oid_prefix": "",
                    "push_endpoint": "amqp://127.0.0.1",
                    "push_endpoint_args":
"amqp-exchange=rgw-exchange&push-endpoint=amqp://127.0.0.1
&use-ssl=false&verify-ssl=false",
                    "push_endpoint_topic": "MyTopic",
                    "stored_secret": "false",
                    "persistent": "false"
                },
                "arn": "arn:aws:sns:default:marvel:MyTopic",
                "opaqueData": ""
            },
            "subs": []
        }
    ]
}

Once I delete both of those and attempt to delete the notification on the
bucket, RGW logs as below.
Returned status code is 200 but notification can still be listed.

Log

debug 2021-06-25T08:36:59.823+0000 7fefed33a700 10 cache get:
name=my-rgw.rgw.log++pubsub.marvel.bucket.bucket-by-thanos/103132f8-3d78-4592-ae7b-067ecf2a1b92.9081601.36
: hit (requested=0x6, cached=0x17)
debug 2021-06-25T08:36:59.823+0000 7fefed33a700 20 get_system_obj_state:
s->obj_tag was set empty
debug 2021-06-25T08:36:59.823+0000 7fefed33a700 10 cache get:
name=my-rgw.rgw.log++pubsub.marvel.bucket.bucket-by-thanos/103132f8-3d78-4592-ae7b-067ecf2a1b92.9081601.36
: hit (requested=0x1, cached=0x17)
debug 2021-06-25T08:36:59.823+0000 7fefed33a700 20 get_system_obj_state:
rctx=0x558dd865b1c0 obj=my-rgw.rgw.log:pubsub.marvel state=0x558dd5ff9720
s->prefetch_data=0
debug 2021-06-25T08:36:59.823+0000 7fefed33a700 10 cache get:
name=my-rgw.rgw.log++pubsub.marvel : hit (requested=0x16, cached=0x17)
debug 2021-06-25T08:36:59.823+0000 7fefed33a700 20 get_system_obj_state:
s->obj_tag was set empty
debug 2021-06-25T08:36:59.823+0000 7fefed33a700 10 cache get:
name=my-rgw.rgw.log++pubsub.marvel : hit (requested=0x11, cached=0x17)
debug 2021-06-25T08:36:59.823+0000 7fefed33a700 10 distributing
notification oid=my-rgw.rgw.control:notify.0 bl.length()=189
debug 2021-06-25T08:36:59.823+0000 7ff108570700 10
RGWWatcher::handle_notify()  notify_id 3573413718119 cookie 94067650647552
notifier 14638216 bl.length()=189
debug 2021-06-25T08:36:59.831+0000 7fefe3b27700 20 get_system_obj_state:
rctx=0x558dd865b1c0 obj=my-rgw.rgw.log:pubsub.marvel state=0x558dd5ff9720
s->prefetch_data=0
debug 2021-06-25T08:36:59.831+0000 7fefe3b27700 10 cache get:
name=my-rgw.rgw.log++pubsub.marvel : hit (requested=0x1, cached=0x17)
debug 2021-06-25T08:36:59.839+0000 7fefe3b27700  1 ERROR: topic not found
debug 2021-06-25T08:36:59.839+0000 7fefe3b27700  1 ERROR: failed to read
topic info: ret=-2
debug 2021-06-25T08:36:59.839+0000 7fefe3b27700  1 failed to remove
notification of topic 'bucket-by-thanos_notification_MyTopic', ret=-2
debug 2021-06-25T08:36:59.839+0000 7fefe3b27700 20 get_system_obj_state:
rctx=0x558dd865b1c0 obj=my-rgw.rgw.log:pubsub.marvel state=0x558dd5ff9720
s->prefetch_data=0
debug 2021-06-25T08:36:59.839+0000 7fefe3b27700 10 cache get:
name=my-rgw.rgw.log++pubsub.marvel : hit (requested=0x11, cached=0x17)
debug 2021-06-25T08:36:59.843+0000 7ff007b6f700 10 cache put:
name=my-rgw.rgw.log++pubsub.marvel info.flags=0x17
debug 2021-06-25T08:36:59.843+0000 7ff007b6f700 10 moving
my-rgw.rgw.log++pubsub.marvel to cache LRU end
debug 2021-06-25T08:36:59.843+0000 7ff007b6f700 10 distributing
notification oid=my-rgw.rgw.control:notify.4 bl.length()=189
debug 2021-06-25T08:36:59.843+0000 7ff108d71700 10
RGWWatcher::handle_notify()  notify_id 3573413545888 cookie 94067664846848
notifier 14638216 bl.length()=189
debug 2021-06-25T08:36:59.843+0000 7ff108d71700 10 cache put:
name=my-rgw.rgw.log++pubsub.marvel info.flags=0x17
debug 2021-06-25T08:36:59.843+0000 7ff108d71700 10 moving
my-rgw.rgw.log++pubsub.marvel to cache LRU end
debug 2021-06-25T08:36:59.843+0000 7feff6b4d700  2 req 329 0.019999718s
s3:pubsub_notification_delete_s3 completing
debug 2021-06-25T08:36:59.843+0000 7feff6b4d700 20 get_system_obj_state:
rctx=0x7ff122a72750 obj=my-rgw.rgw.log:script.postrequest.marvel
state=0x558dd5ff9720 s->prefetch_data=0
debug 2021-06-25T08:36:59.843+0000 7feff6b4d700 10 cache get:
name=my-rgw.rgw.log++script.postrequest.marvel : hit (negative entry)
debug 2021-06-25T08:36:59.843+0000 7feff6b4d700  2 req 329 0.019999718s
s3:pubsub_notification_delete_s3 op status=0
debug 2021-06-25T08:36:59.847+0000 7feff6b4d700  2 req 329 0.023999661s
s3:pubsub_notification_delete_s3 http status=200
debug 2021-06-25T08:36:59.847+0000 7feff6b4d700  1 ====== req done
req=0x7ff122a73620 op status=0 http_status=200 latency=0.023999661s ======
debug 2021-06-25T08:36:59.847+0000 7feff6b4d700  1 beast: 0x7ff122a73620:
11.1.150.14 - marvel$oidc$f1be270a-d6a0-4f97-b444-c1f23816b25a
[25/Jun/2021:08:36:59.823 +0000] "DELETE
/bucket-by-thanos?notification=bucket-by-thanos_notification HTTP/1.1" 200
0 - "PostmanRuntime/7.28.0" -

Thanks
Daniel
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux