Hi, What we have found seems like it is a blocking issue when I terminate https on a loadbalancer and between the loadbalancer and rgw http is the mode. So seems liket he ssl termination has to be done on the rgw and can't be done on the loadbalancer? Or how we can workaround it any idea? Here are the debug logs: With loadbalancer https endpoint: https://justpaste.it/5d93w Directly with rgw ip without loadbalancer: https://justpaste.it/9rn28 Both case the issue is like this: "endpoint validation error: sending password over insecure transport" To be honest I want to do the ssl on loadbalancer don't want to do on rgw. Maybe you can suggest something. Istvan Szabo Senior Infrastructure Engineer --------------------------------------------------- Agoda Services Co., Ltd. e: istvan.szabo@xxxxxxxxx --------------------------------------------------- -----Original Message----- From: Yuval Lifshitz <ylifshit@xxxxxxxxxx> Sent: Tuesday, April 27, 2021 11:49 PM To: Szabo, Istvan (Agoda) <Istvan.Szabo@xxxxxxxxx> Cc: ceph-users@xxxxxxx; Raveendran, Vigneshwaran (Agoda) <Vigneshwaran.Raveendran@xxxxxxxxx> Subject: [Suspicious newsletter] Re: Getting `InvalidInput` when trying to create a notification topic with Kafka endpoint On Tue, Apr 27, 2021 at 1:59 PM Szabo, Istvan (Agoda) < Istvan.Szabo@xxxxxxxxx> wrote: > Hello, > > Thank you very much to pickup the question and sorry for the late response. > > Yes, we are sending in cleartext also using HTTPS, but how it should > be send if not like this? > > if you send the user/password using HTTPS connection between the client and the RGW there should be no error. could you please provide the RGW debug log, to see why "invalid argument" was replied? > Also connected to this issue a bit, when we subscribe a bucket to a > topic with non-ACL kafka topic, any operations (PUT or DELETE) is > simply blocking and not returning. Not even any error response. > > this would be the case when the kafka broker is down (or the > parameters you provided to the topic were incorrect). a workaround for this issue is to mark the endpoint with "kafka-ack-level=none", this will not block for the reply, but note that if the broker is down or misconfigured, the notification will be lost. a better option (if you are using "pacific" and up) is to mark the topic with the "persistent" flag. this would mean that even if the broker is down or misconfigured, the notification will be retired until successful, and, in addition, will not block the request. > $ s3cmd -c ~/.s3cfg put --add-header x-amz-meta-foo:bar3 > certificate.pdf s3://vig-test > WARNING: certificate.pdf: Owner groupname not known. Storing > GID=1354917867 instead. > WARNING: Module python-magic is not available. Guessing MIME types > based on file extensions. > upload: 'certificate.pdf' -> 's3://vig-test/certificate.pdf' [1 of 1] > 65536 of 91224 71% in 0s 291.17 KB/s > > > > Istvan Szabo > Senior Infrastructure Engineer > --------------------------------------------------- > Agoda Services Co., Ltd. > e: istvan.szabo@xxxxxxxxx > --------------------------------------------------- > > > > *From:* Yuval Lifshitz <ylifshit@xxxxxxxxxx> > *Sent:* Wednesday, April 21, 2021 10:34 PM > *To:* Szabo, Istvan (Agoda) <Istvan.Szabo@xxxxxxxxx> > *Cc:* ceph-users@xxxxxxx > *Subject:* Re: Getting `InvalidInput` when trying to > create a notification topic with Kafka endpoint > > > > Hi Istvan, > > Can you please share the relevant part for the radosgw log, indicating > which input was invalid? > > The only way I managed to reproduce that error is by sending the > request to a non-HTTPS radosgw (which does not seem to be your case). > In such a case it replies with "InvalidInput" because we are trying to > send user/password in cleartext. > > I used curl, similarly to what you did against a vstart cluster based > off of master: https://paste.sh/SQ_8IrB5#BxBYbh1kTh15n7OKvjB5wEOM > > > > Yuval > > > > On Wed, Apr 21, 2021 at 11:23 AM Szabo, Istvan (Agoda) < > Istvan.Szabo@xxxxxxxxx> wrote: > > Hi Ceph Users, > Here is the latest request I tried but still not working > > curl -v -H 'Date: Tue, 20 Apr 2021 16:05:47 +0000' -H 'Authorization: > AWS <accessid>:<signature>' -L -H 'content-type: > application/x-www-form-urlencoded' -k -X POST https://servername -d > Action=CreateTopic&Name=test-ceph-event-replication&Attributes.entry.8 > .key=push-endpoint&Attributes.entry.8.value=kafka://<username>:<passwo > rd>@servername2:9093&Attributes.entry.5.key=use-ssl&Attributes.entry.5 > .value=true > > And the response I get is still Invalid Input <?xml version="1.0" > encoding="UTF-8"?><Error><Code>InvalidInput</Code><RequestId>tx0000000 > 00000007993081-00607efbdd-1c7e96b-hkg</RequestId><HostId>1c7e96b-hkg-d > ata</HostId></Error> > Can someone please help with this? > Istvan Szabo > Senior Infrastructure Engineer > --------------------------------------------------- > Agoda Services Co., Ltd. > e: istvan.szabo@xxxxxxxxx<mailto:istvan.szabo@xxxxxxxxx> > --------------------------------------------------- > > > ________________________________ > This message is confidential and is for the sole use of the intended > recipient(s). It may also be privileged or otherwise protected by > copyright or other legal rules. If you have received it by mistake > please let us know by reply email and delete it from your system. It > is prohibited to copy this message or disclose its content to anyone. > Any confidentiality or privilege is not waived or lost by any mistaken > delivery or unauthorized disclosure of the message. All messages sent > to and from Agoda may be monitored to ensure compliance with company > policies, to protect the company's interests and to remove potential > malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses. > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an > email to ceph-users-leave@xxxxxxx > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses. _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |