As noted in the bug report, the issue has affected only multipart
objects at this time. I have added some more remarks there.
And yes, multipart objects tend to have 0 byte head objects in general.
The affected objects are simply missing all shadow objects, leaving us
with nothing but the empty head object and a few metadata.
On 13/11/2020 20:14, Eric Ivancich wrote:
Thank you for the answers to those questions, Janek.
And in case anyone hasn’t seen it, we do have a tracker for this issue:
https://tracker.ceph.com/issues/47866
We may want to move most of the conversation to the comments there, so
everything’s together.
I do want to follow up on your answer to Question 4, Janek:
On Nov 13, 2020, at 12:22 PM, Janek Bevendorff
<janek.bevendorff@xxxxxxxxxxxxx
<mailto:janek.bevendorff@xxxxxxxxxxxxx>> wrote:
4. Is anyone experiencing this issue willing to run their RGWs with
'debug_ms=1'? That would allow us to see a request from an RGW to
either remove a tail object or decrement its reference counter (and
when its counter reaches 0 it will be deleted).
I haven't had any new data loss in the last few days (at least I
think so, I read 1byte from all objects, but didn't compare
checksums, so I cannot say if all objects are complete, but at least
all are there).
With multipart uploads I believe this is a sufficient test, as the
first bit of data is in the first tail object, and it’s tail objects
that seem to be disappearing.
However if the object is not uploaded via multipart and if it does
have tail (_shadow_) objects, then the initial data is stored in the
head object. So this test would not be truly diagnostic. This could be
done with a large object, for example, with `s3cmd put
--disable-multipart …`.
Eric
--
J. Eric Ivancich
he / him / his
Red Hat Storage
Ann Arbor, Michigan, USA
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
