Yeah my bad... That cern doc is different from what you're trying to achieve. .. dan On Wed, 5 Aug 2020, 15:31 Chris Palmer, <chris@xxxxxxxxxxxxxxxxxxxxx> wrote: > Hi Dan > > The second link you refer to is, I believe, version-agnostic. (And s3cmd > is also version-unaware). So "an object" would expire after x days, rather > than "non-current versions of an object". > > I've re-read the first description several times, and remembering that it > only applies to versioned buckets, still think that it is the correct xml > to specify how long non-current versions are retained (which is what I > want). It is also the exact same policy that I used without problem on > 15.2.3, and also one that I use on AWS S3 itself.... > > Regards, Chris > > On 05/08/2020 14:08, Dan van der Ster wrote: > > Hi Chris, > > Is it possible that this is the correct behaviour for NoncurrentDays > expiration? > > > https://docs.aws.amazon.com/AmazonS3/latest/API/API_NoncurrentVersionExpiration.html > > AFAIK, the xml to expire *only* the versions older than x days is somewhat > different: > > > https://clouddocs.web.cern.ch/object_store/s3cmd.html#object-expiration-with-s3cmd > > .. Dan > > > > > On Wed, 5 Aug 2020, 14:52 Chris Palmer, <chris@xxxxxxxxxxxxxxxxxxxxx> > wrote: > >> This is starting to look like a regression error in Octopus 15.2.4. >> >> After cleaning things up by deleting all old versions, and deleting and >> recreating the bucket lifecycle policy (see below), I then let it run. >> Each day a new version got created, dating back to 17 July (correct). >> Until this morning when we went over the 18-day non-current expiration: >> this morning all but the latest version of each object disappeared - >> which is what happened the midnight following after our 15.2.3->15.2.4 >> upgrade. >> >> So instead of a gently rolling 18-days of versions (on 15.2.3), we now >> build up to 18 days after which all non-current versions get deleted (on >> 15.2.4). >> >> Anyone come across versioning problems on 15.2.4? >> >> Thanks, Chris >> >> On 17/07/2020 09:11, Chris Palmer wrote: >> > This got worse this morning. An RGW daemon crashed at midnight with a >> > segfault, and the backtrace hints that it was processing the >> > expiration rule: >> > >> > "backtrace": [ >> > "(()+0x12730) [0x7f97b8c4e730]", >> > "(()+0x15878a) [0x7f97b862378a]", >> > "(std::__cxx11::basic_string<char, std::char_traits<char>, >> > std::allocator<char> >::compare(std::__cxx11::basic_string<char, >> > std::char_traits<char>, std::allocator<char> > const&) const+0x23) >> > [0x7f97c25d3e43]", >> > "(LCOpAction_DMExpiration::check(lc_op_ctx&, >> > std::chrono::time_point<ceph::time_detail::real_clock, >> > std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> > >> > >*)+0x87) [0x7f97c283d127]", >> > "(LCOpRule::process(rgw_bucket_dir_entry&, DoutPrefixProvider >> > const*)+0x1b8) [0x7f97c281cbc8]", >> > "(()+0x5b836d) [0x7f97c281d36d]", >> > "(WorkQ::entry()+0x247) [0x7f97c28302d7]", >> > "(()+0x7fa3) [0x7f97b8c43fa3]", >> > "(clone()+0x3f) [0x7f97b85c44cf]" >> > >> > One object version got removed when it should not have. >> > >> > In an attempt to clean things up I have manually deleted all >> > non-current versions, and removed and recreated the (same) lifecycle >> > policy. I will also create a new test bucket with a similar policy and >> > test that in parallel. We will see what happens tomorrow.... >> > >> > Thanks, Chris >> > >> > >> > On 16/07/2020 08:22, Chris Palmer wrote: >> >> I have an RGW bucket (backups) that is versioned. A nightly job >> >> creates a new version of a few objects. There is a lifecycle policy >> >> (see below) that keeps 18 days of versions. This has been working >> >> perfectly and has not been changed. Until I upgraded Octopus... >> >> >> >> The nightly job creates separate log files, including a listing of >> >> the object versions. From these I can see that: >> >> >> >> 13/7 02:14 versions from 13/7 01:13 back to 24/6 01:17 (correct) >> >> >> >> 14/7 02:14 versions from 14/7 01:13 back to 25/6 01:14 (correct) >> >> >> >> 14/7 10:00 upgrade Octopus 15.2.3 -> 15.2.4 >> >> >> >> 15/7 02:14 versions from 15/7 01:13 back to 25/6 01:14 (would have >> >> expected 25/6 to have expired) >> >> >> >> 16/7 02:14 versions from 16/7 01:13 back to 15/7 01:13 (now all >> >> pre-upgrade versions have wrongly disappeared) >> >> >> >> It's not a big deal for me as they are only backups, providing it >> >> continues to work correctly from now on. However it may affect some >> >> other people much more. >> >> >> >> Any ideas on the root cause? And if it is likely to be stable again >> now? >> >> >> >> Thanks, Chris >> >> >> >> { >> >> "Rules": [ >> >> { >> >> "Expiration": { >> >> "ExpiredObjectDeleteMarker": true >> >> }, >> >> "ID": "Expiration & incomplete uploads", >> >> "Prefix": "", >> >> "Status": "Enabled", >> >> "NoncurrentVersionExpiration": { >> >> "NoncurrentDays": 18 >> >> }, >> >> "AbortIncompleteMultipartUpload": { >> >> "DaysAfterInitiation": 1 >> >> } >> >> } >> >> ] >> >> } >> >> >> >> _______________________________________________ >> >> ceph-users mailing list -- ceph-users@xxxxxxx >> >> To unsubscribe send an email to ceph-users-leave@xxxxxxx >> > _______________________________________________ >> > ceph-users mailing list -- ceph-users@xxxxxxx >> > To unsubscribe send an email to ceph-users-leave@xxxxxxx >> _______________________________________________ >> ceph-users mailing list -- ceph-users@xxxxxxx >> To unsubscribe send an email to ceph-users-leave@xxxxxxx >> > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |