Hi. Basic logic: 1.bucket policy transition 2.radosgw-admin gc process --include-all 3.1.rados ls -p pool | grep <bucket_marker_id> >bucket_objects.txt 3.2.rados listxattr -p pool objname | xargs -L1 echo rados getattr -p pool objname >> objname.txt 3.3.rados create -p pool objname 3.4.cat objname.txt | xargs -L1 echo rados setattr -p pool objname attr value 4.radosgw-admin metadata get bucket.instance.<bucket>.<marker_id> | tee bucket.json bucket_backup.json 5.change placement_rule to "default-placement/NEW_CLASS", 6.radosgw-admin metadata rm bucket.instance.<bucket>.<marker_id> 7.radosgw-admin metadata put bucket.instance.<bucket>.<marker_id> < bucker.json вт, 25 янв. 2022 г. в 21:31, Frédéric Nass <frederic.nass@xxxxxxxxxxxxxxxx>: > > Le 25/01/2022 à 18:28, Casey Bodley a écrit : > > On Tue, Jan 25, 2022 at 11:59 AM Frédéric Nass > > <frederic.nass@xxxxxxxxxxxxxxxx> wrote: > >> > >> Le 25/01/2022 à 14:48, Casey Bodley a écrit : > >>> On Tue, Jan 25, 2022 at 4:49 AM Frédéric Nass > >>> <frederic.nass@xxxxxxxxxxxxxxxx> wrote: > >>>> Hello, > >>>> > >>>> I've just heard about storage classes and imagined how we could use > them > >>>> to migrate all S3 objects within a placement pool from an ec pool to a > >>>> replicated pool (or vice-versa) for data resiliency reasons, not to > save > >>>> space. > >>>> > >>>> It looks possible since ; > >>>> > >>>> 1. data pools are associated to storage classes in a placement pool > >>>> 2. bucket lifecycle policies can take care of moving data from a > storage > >>>> class to another > >>>> 3. we can set a user's default_storage_class to have all new objects > >>>> written by this user reach the new storage class / data pool. > >>>> 4. after all objects have been transitioned to the new storage class, > we > >>>> can delete the old storage class, rename the new storage class to > >>>> STANDARD so that it's been used by default and unset any user's > >>>> default_storage_class setting. > >>> i don't think renaming the storage class will work the way you're > >>> hoping. this storage class string is stored in each object and used to > >>> locate its data, so renaming it could render the transitioned objects > >>> unreadable > >> Hello Casey, > >> > >> Thanks for pointing that out. > >> > >> Do you believe this scenario would work if stopped at step 3.? (keeping > >> default_storage_class set on users's profiles and not renaming the new > >> storage class to STANDARD. Could we delete the STANDARD storage class > >> btw since we would not use it anymore?). > >> > >> If there is no way to define the default storage class of a placement > >> pool without naming it STANDARD could we imaging transitioning all > >> objects again by: > >> > >> 4. deleting the storage class named STANDARD > >> 5. creating a new one named STANDARD (using a ceph pool of the same data > >> placement scheme than the one used by the temporary storage class > >> created above) > > instead of deleting/recreating STANDARD, you could probably just > > modify it's data pool. only do this once you're certain that there are > > no more objects in the old data pool. you might need to wait for > > garbage collection to clean up the tail objects there too (or force it > > with 'radosgw-admin gc process --include-all') > > Interesting scenario. So in the end we'd have objects named after both > storage classes in the same ceph pool, the old ones named after the new > storage class name and the new ones being written after the STANDARD > storage class, right? > > > > >> 6. transitioning all objects again to the new STANDARD storage class. > >> Then delete the temporary storage class. > > i think this step 6 would run into the > > https://tracker.ceph.com/issues/50974 that Konstantin shared - if the > > two storage classes have the same pool name, the transition doesn't > > actually take effect. you might consider leaving this 'temporary' > > storage class around, but pointing the defaults back at STANDARD > > Well, in step 6., I'd thought about using another new pool for the > recreated STANDARD storage class (to avoid the issue shared by > Konstantin , thanks to him btw) and move all objects to this new pool > again in a new global transition. > > But, I understand you'd recommend avoiding deleting/recreating STANDARD > and just modify the STANDARD data pool after GC execution, am I right? > > Frédéric. > > > > >> ? > >> > >> Best regards, > >> > >> Frédéric. > >> > >>>> Would that work? > >>>> > >>>> Anyone tried this with success yet? > >>>> > >>>> Best regards, > >>>> > >>>> Frédéric. > >>>> > >>>> -- > >>>> Cordialement, > >>>> > >>>> Frédéric Nass > >>>> Direction du Numérique > >>>> Sous-direction Infrastructures et Services > >>>> > >>>> Tél : 03.72.74.11.35 > >>>> > >>>> _______________________________________________ > >>>> ceph-users mailing list -- ceph-users@xxxxxxx > >>>> To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |