“Is there a usecase for sending a notification when the upload starts?” – Not for me. Only having to watch for ObjectCreated:CompleteMultipartUpload and ObjectCreated:Put works for me. Quincy here I come. Thanks! -- Mark Selby Sr Linux Administrator, The Voleon Group mselby@xxxxxxxxxx This email is subject to important conditions and disclosures that are listed on this web page: https://voleon.com/disclaimer/. From: Yuval Lifshitz <ylifshit@xxxxxxxxxx> Date: Thursday, July 21, 2022 at 12:21 AM To: Mark Selby <mselby@xxxxxxxxxx> Cc: "dang@xxxxxxxxxx" <dang@xxxxxxxxxx>, "ceph-users@xxxxxxx" <ceph-users@xxxxxxx> Subject: Re: Re: [EXTERNAL] Re: RGW Bucket Notifications and MultiPart Uploads Hi Mark, Starting from quincy, we send 1 notification, "ObjectCreated:CompleteMultipartUpload", when the upload is complete. See: https://docs.ceph.com/en/quincy/radosgw/s3-notification-compatibility/ We don't send the "ObjectCreated:Post" notification when the upload starts as it will be confusing with other objects "POST" uploads. Is there a usecase for sending a notification when the upload starts? Yuval Yuval On Thu, Jul 21, 2022 at 12:53 AM Mark Selby <mselby@xxxxxxxxxx> wrote: I have not tested with Quincy/17.x yet so I do not know which notifications are sent for Multipart uploads in this release set. I know that for Pacific.16.x I needed to add some code/logic to only act on notifications that represented the end state of an Object creation. My tests show that when a multipart upload is in progress if you perform a head on the object before the final part is uploaded that you will get back a 200 with the size that the object eventually will be. The multiple noitifications that occur with the Multipart upload have size set to the chunk that is being uploaded, not the total size. In order to get the behavior that I wanted, act on an object after it has been uploaded I had to code the following: - get a notification and get it's size - head the object in RGW and get it's size - if the sizes do not match then do nothing. - When the size in the notification matches the size on the head request the event type is ObjectCreated:CompleteMultipartUpload and we know the upload is complete. This is a bunch of extra code and round trips that.I wish that I did not have to make. Hopefully Quincy only sends 2 notifications for a multipart upload (1) The initial Post and (2) The find Put. -- Mark Selby Sr Linux Administrator, The Voleon Group mselby@xxxxxxxxxx This email is subject to important conditions and disclosures that are listed on this web page: https://voleon.com/disclaimer/. On 7/20/22, 5:57 AM, "Daniel Gryniewicz" <dang@xxxxxxxxxx> wrote: Seems like the notification for a multipart upload should look different to a normal upload? Daniel On 7/20/22 08:53, Yehuda Sadeh-Weinraub wrote: > Can maybe leverage one of the other calls to check for upload completion: > list multipart uploads and/or list parts. The latter should work if you > have the upload id at hand. > > Yehuda > > On Wed, Jul 20, 2022, 8:40 AM Casey Bodley <cbodley@xxxxxxxxxx> wrote: > >> On Wed, Jul 20, 2022 at 12:57 AM Yuval Lifshitz <ylifshit@xxxxxxxxxx> >> wrote: >>> >>> yes, that would work. you would get a "404" until the object is fully >>> uploaded. >> >> just note that you won't always get 404 before multipart complete, >> because multipart uploads can overwrite existing objects >> >> _______________________________________________ >> ceph-users mailing list -- ceph-users@xxxxxxx >> To unsubscribe send an email to ceph-users-leave@xxxxxxx >> >> > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
