Hi Sanjeev, You need to install our aws cli extensions. Please see the instructions here: https://github.com/ceph/ceph/blob/master/examples/boto3/README.md If you are using a version of ceph which is older than "pacific", you would also need to change the default SNS signer to use v2 instead of v4: aws configure set default.sns.signature_version s3 Please let me know if it helped, Yuval On Tue, Sep 21, 2021 at 6:12 PM Sanjeev Jha <sanjeev_mac@xxxxxxxxxxx> wrote: > Hi Yuval, > > I stuck on the first step where I am trying to create sns topic but not > able to create it. I am not able to figure out the issue. AMQP server is > ready, up and running with AMQP 0.9.1. > > [root@ceprgw01 ~]# aws --endpoint-url http://localhost:8000 sns create-topic --name=mytopic --attributes='{"push-endpoint": "amqp://10.xx.xx.xx:5672", "amqp-exchange": "ex1", "amqp-ack-level": "broker"}' > > getting below error: > Unknown options: {"push-endpoint": "amqp://10.xx.xx.xx:15672", "amqp-ack-level": "broker", "persistent": "true", "amqp-exchange": "topic_logs"}, set-topic-attributes > > I am using Ceph 4.2. > > [root@ceprgw01 ~]# aws --version > aws-cli/1.14.28 Python/2.7.5 Linux/3.10.0-1160.25.1.el7.x86_64 botocore/1.8.35 > > > > Best regards, > Sanjeev > > ------------------------------ > *From:* Yuval Lifshitz <ylifshit@xxxxxxxxxx> > *Sent:* Friday, August 20, 2021 3:40 PM > *To:* Sanjeev Jha <sanjeev_mac@xxxxxxxxxxx> > *Cc:* ceph-users@xxxxxxx <ceph-users@xxxxxxx> > *Subject:* Re: S3 Bucket Notification requirement > > > > On Thu, Aug 19, 2021 at 6:30 PM Sanjeev Jha <sanjeev_mac@xxxxxxxxxxx> > wrote: > > Hi Yuval, > > Thanks very much for your reply. > > I am using AMQP 0.9.1. > > Can I use aws sns create-topic command to create a topic in Ceph's RadosGW > ? > > yes. note that you define the topic on the RGW, not the rabbitmq broker. > e.g. > > aws --endpoint-url http://<rgw host>:<rgw port> sns create-topic > --name=mytopic --attributes='{"push-endpoint": "amqp://<amqp host>:5672", > "amqp-exchange": "ex1"}' > > (see also: https://github.com/ceph/ceph/tree/master/examples/boto3#aws-cli > ) > > ,, If Yes then How and where to define notifications to associate with > created topics? Basically, I want to understand the tasks needs to be > defined in Ceph and RabbitMQ Broker. > > you can use the aws cli to define the topic. a minimal setup to get all > notifications for object creation would be: > > aws --region=default --endpoint-url http://<rgw host>:<rgw posrt> s3api > put-bucket-notification-configuration --bucket=mybucket > --notification-configuration='{"TopicConfigurations": [{"Id": "notif1", > "TopicArn": "arn:aws:sns:default::mytopic", "Events": > ["s3:ObjectCreated:*"]}]}' > > > I have found few formats like below in the developer documentation but > wondering how/where to define it: -- Where and how to define these below > codes, In ceph or AMQP broker? > > > in ceph (RGW). > these are the formats of the raw HTTP messages used to create the topic. > when you use AWS cli, they crafty these messages and send them to the RGW. > unless you want to craft these messages yourself (e.g. using CURL) you > should not really care about them. > > > For Example: > POST > Action=CreateTopic > &Name=<topic-name> > &push-endpoint=<endpoint> > [&Attributes.entry.1.key=amqp-exchange&Attributes.entry.1.value=<exchange>] > ... > .... > > > > Java is being used in RabbitMQ broker with AMQP 0.9.1 protocol.. > > creating topics and notifications is also possible using the AWS Java SDK: > * topics: > https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/examples-simple-notification-service.html > * notifications: > > https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/BucketNotificationConfiguration.html > > so, if your application wants to create them, the application developers > will have to write code that does that (sadly, we don't have Java examples > like the CLI and python ones, but you can use the example from the AWS > docs). > > if the topics and notifications are more static, you can create them up > front using the AWS cli, and then you don't need to do anything in your > Java code. > > > Best regards, > Sanjeev > > > ------------------------------ > *From:* Yuval Lifshitz <ylifshit@xxxxxxxxxx> > *Sent:* Thursday, August 19, 2021 8:38 PM > *To:* Sanjeev Jha <sanjeev_mac@xxxxxxxxxxx> > *Cc:* ceph-users@xxxxxxx <ceph-users@xxxxxxx> > *Subject:* Re: S3 Bucket Notification requirement > > Hi Sanjeev, > Welcome to the Ceph community! > > Which protocol do you intend to use in ActiveMQ? > If you want to use AMQP1.0, you will have to wait, as this is still not > officially supported [1]. > Currently, we support AMQP0.9.1, Kafka, and HTTP. > > As for the more general question. > To make bucket notifications work you first need to define a "topic", with > an endpoint configured to point your message broker. > Next, you need to define a "notification" that associates the topic with a > bucket on which you want to get notifications. > In the "notification" you can define a filter and the events on which you > want to be notified. > > You can find examples of how to do that using the AWS CLI tool (or in > python, using the boto3 library) here [2]. > If you are looking for a different client or client SDK let me know. > > More details on the bucket notification feature (without any client > specific examples) could be found here [3]. > > Yuval > > [1] https://github.com/ceph/ceph/pull/42548 > [2] https://github.com/ceph/ceph/tree/master/examples/boto3 > [3] https://docs.ceph.com/en/latest/radosgw/notifications/ > > > On Thu, Aug 19, 2021 at 5:10 PM Sanjeev Jha <sanjeev_mac@xxxxxxxxxxx> > wrote: > > Dear Sir, > > I would like to inform you that I am new to Ceph, specially new to the S3 > bucket notification topic and not able to understand how exactly to > configure it in RadaosGW. We are using an ActiveMQ broker. > There is no easy technical documentation for Ceph available and I am not > sure how and where to configure the coding. > I am not the developer, rather just an administrator who manages Ceph. > I would appreciate If you could let me know the steps if possible so that > I could achieve this goal. > > Best regards, > Sanjeev > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |