Hi Boris, I’m a little confused. The pastebin seems to show that you can stat "ff7a8b0c-07e6-463a-861b-78f0adeba8ad.81095307.17600__multipart_available_file1.pdf.2~ve8VhAEvaRSzAPfacz9rI-aLMpLY_Yw.1”, but I thought it was missing. Can you clarify? The bug has been in RGW for a quite a while, well before octopus. It involves a race condition with a very narrow window, so normally only encountered in large, busy clusters. Also, I think it’s up to the s3 client whether to use multipart upload. Do you know which s3 client the user was using? Eric (he/him) > On Jun 14, 2022, at 1:02 AM, Boris Behrens <bb@xxxxxxxxx> wrote: > > Hmm.. I will check what the user is deleting. Maybe this is it. > Do you know if this bug is new in 15.2.16? > > I can't share the data, but I can share the metadata: > https://pastebin.com/raw/T1YYLuec <https://pastebin.com/raw/T1YYLuec> > > For the missing files I have, the multipart file is not available in rados, but the 0 byte file is. > The rest is more or less identical. > > The seem to use the aws-sdk-dotnet (aws-sdk-dotnet-coreclr/3.3.110.57 <http://3.3.110.57/> aws-sdk-dotnet-core/3.3.106.11 <http://3.3.106.11/>), but so small multiparts are very strange. I guess you can really screw up configs but who am I to judge. > > Am Di., 14. Juni 2022 um 00:29 Uhr schrieb J. Eric Ivancich <ivancich@xxxxxxxxxx <mailto:ivancich@xxxxxxxxxx>>: > There is no known bug that would cause the rados objects underlying an RGW object to be removed without a user requesting the RGW object be deleted. > > There is a known bug where the bucket index might not get updated correctly after user-requested operations. So perhaps the user removed the rgw object, but it still incorrectly shows up in the bucket index. The PR for the fix for that bug merged into the octopus branch, but after 15.2.16. See: > > https://github.com/ceph/ceph/pull/45902 <https://github.com/ceph/ceph/pull/45902> > > So it should be in the next octopus release. > > I also find it odd that a 250KB file gets a multipart object. What do we know about the original object? Do we know it’s size? Could the multipart upload never have completed? In that case there could be incomplete multipart entries in the bucket index, but they should never have been finalized into a regular bucket index entry. > > Are you willing to share all the bucket index entries related to this object? > > Eric > (he/him) _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |