Hi Mark, It is always good to move forward with the ceph versions :-) However, this was also backported to pacific [1]. So, if you want to wait for the next pacific release, the fix should be there. Yuval [1] https://github.com/ceph/ceph/pull/47175 On Thu, Jul 21, 2022 at 11:10 PM Mark Selby <mselby@xxxxxxxxxx> wrote: > “Is there a usecase for sending a notification when the upload starts?” – > Not for me. Only having to watch for ObjectCreated:CompleteMultipartUpload > and ObjectCreated:Put works for me. Quincy here I come. Thanks! > > > > > > -- > > Mark Selby > > Sr Linux Administrator, The Voleon Group > > mselby@xxxxxxxxxx > > > > This email is subject to important conditions and disclosures that are > listed on this web page: https://voleon.com/disclaimer/. > > > > > > *From: *Yuval Lifshitz <ylifshit@xxxxxxxxxx> > *Date: *Thursday, July 21, 2022 at 12:21 AM > *To: *Mark Selby <mselby@xxxxxxxxxx> > *Cc: *"dang@xxxxxxxxxx" <dang@xxxxxxxxxx>, "ceph-users@xxxxxxx" < > ceph-users@xxxxxxx> > *Subject: *Re: Re: [EXTERNAL] Re: RGW Bucket Notifications > and MultiPart Uploads > > > > Hi Mark, > > Starting from quincy, we send 1 notification, > "ObjectCreated:CompleteMultipartUpload", when the upload is complete. See: > https://docs.ceph.com/en/quincy/radosgw/s3-notification-compatibility/ > <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.ceph.com%2Fen%2Fquincy%2Fradosgw%2Fs3-notification-compatibility%2F&data=05%7C01%7Cmselby%40voleon.com%7C575f0827f7d24c11344108da6ae9932f%7C45212fd85f544a19a6ba493ff6e072b1%7C0%7C0%7C637939848683524799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=9us%2BsEKZqY2fxYg1OhEw2frGMOrLtvxZRK7OEXt1lR8%3D&reserved=0> > > We don't send the "ObjectCreated:Post" notification when the upload starts > as it will be confusing with other objects "POST" uploads. > > Is there a usecase for sending a notification when the upload starts? > > > > Yuval > > > > > > Yuval > > > > On Thu, Jul 21, 2022 at 12:53 AM Mark Selby <mselby@xxxxxxxxxx> wrote: > > I have not tested with Quincy/17.x yet so I do not know which > notifications are sent for Multipart uploads in this release set. > > I know that for Pacific.16.x I needed to add some code/logic to only act > on notifications that represented the end state of an Object creation. > > My tests show that when a multipart upload is in progress if you perform a > head on the object before the final part is uploaded that you will get back > a 200 with the size that the object eventually will be. The multiple > noitifications that occur with the Multipart upload have size set to the > chunk that is being uploaded, not the total size. > > In order to get the behavior that I wanted, act on an object after it has > been uploaded I had to code the following: > > - get a notification and get it's size > - head the object in RGW and get it's size > - if the sizes do not match then do nothing. > - When the size in the notification matches the size on the head request > the event type is ObjectCreated:CompleteMultipartUpload and we know the > upload is complete. > > This is a bunch of extra code and round trips that.I wish that I did not > have to make. > > Hopefully Quincy only sends 2 notifications for a multipart upload (1) The > initial Post and (2) The find Put. > > > > -- > > > Mark Selby > Sr Linux Administrator, The Voleon Group > mselby@xxxxxxxxxx > > This email is subject to important conditions and disclosures that are > listed on this web page: https://voleon.com/disclaimer/ > <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fvoleon.com%2Fdisclaimer%2F&data=05%7C01%7Cmselby%40voleon.com%7C575f0827f7d24c11344108da6ae9932f%7C45212fd85f544a19a6ba493ff6e072b1%7C0%7C0%7C637939848683524799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3hPSckiUcDuychmFpyiqXKEYmc8ZXEbaFGGGlehFsv0%3D&reserved=0> > . > > > On 7/20/22, 5:57 AM, "Daniel Gryniewicz" <dang@xxxxxxxxxx> wrote: > > Seems like the notification for a multipart upload should look > different > to a normal upload? > > Daniel > > On 7/20/22 08:53, Yehuda Sadeh-Weinraub wrote: > > Can maybe leverage one of the other calls to check for upload > completion: > > list multipart uploads and/or list parts. The latter should work if > you > > have the upload id at hand. > > > > Yehuda > > > > On Wed, Jul 20, 2022, 8:40 AM Casey Bodley <cbodley@xxxxxxxxxx> > wrote: > > > >> On Wed, Jul 20, 2022 at 12:57 AM Yuval Lifshitz < > ylifshit@xxxxxxxxxx> > >> wrote: > >>> > >>> yes, that would work. you would get a "404" until the object is > fully > >>> uploaded. > >> > >> just note that you won't always get 404 before multipart complete, > >> because multipart uploads can overwrite existing objects > >> > >> _______________________________________________ > >> ceph-users mailing list -- ceph-users@xxxxxxx > >> To unsubscribe send an email to ceph-users-leave@xxxxxxx > >> > >> > > _______________________________________________ > > ceph-users mailing list -- ceph-users@xxxxxxx > > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |