Le 25/01/2022 à 18:28, Casey Bodley a écrit :
On Tue, Jan 25, 2022 at 11:59 AM Frédéric Nass
<frederic.nass@xxxxxxxxxxxxxxxx> wrote:
Le 25/01/2022 à 14:48, Casey Bodley a écrit :
On Tue, Jan 25, 2022 at 4:49 AM Frédéric Nass
<frederic.nass@xxxxxxxxxxxxxxxx> wrote:
Hello,
I've just heard about storage classes and imagined how we could use them
to migrate all S3 objects within a placement pool from an ec pool to a
replicated pool (or vice-versa) for data resiliency reasons, not to save
space.
It looks possible since ;
1. data pools are associated to storage classes in a placement pool
2. bucket lifecycle policies can take care of moving data from a storage
class to another
3. we can set a user's default_storage_class to have all new objects
written by this user reach the new storage class / data pool.
4. after all objects have been transitioned to the new storage class, we
can delete the old storage class, rename the new storage class to
STANDARD so that it's been used by default and unset any user's
default_storage_class setting.
i don't think renaming the storage class will work the way you're
hoping. this storage class string is stored in each object and used to
locate its data, so renaming it could render the transitioned objects
unreadable
Hello Casey,
Thanks for pointing that out.
Do you believe this scenario would work if stopped at step 3.? (keeping
default_storage_class set on users's profiles and not renaming the new
storage class to STANDARD. Could we delete the STANDARD storage class
btw since we would not use it anymore?).
If there is no way to define the default storage class of a placement
pool without naming it STANDARD could we imaging transitioning all
objects again by:
4. deleting the storage class named STANDARD
5. creating a new one named STANDARD (using a ceph pool of the same data
placement scheme than the one used by the temporary storage class
created above)
instead of deleting/recreating STANDARD, you could probably just
modify it's data pool. only do this once you're certain that there are
no more objects in the old data pool. you might need to wait for
garbage collection to clean up the tail objects there too (or force it
with 'radosgw-admin gc process --include-all')
Interesting scenario. So in the end we'd have objects named after both
storage classes in the same ceph pool, the old ones named after the new
storage class name and the new ones being written after the STANDARD
storage class, right?
6. transitioning all objects again to the new STANDARD storage class.
Then delete the temporary storage class.
i think this step 6 would run into the
https://tracker.ceph.com/issues/50974 that Konstantin shared - if the
two storage classes have the same pool name, the transition doesn't
actually take effect. you might consider leaving this 'temporary'
storage class around, but pointing the defaults back at STANDARD
Well, in step 6., I'd thought about using another new pool for the
recreated STANDARD storage class (to avoid the issue shared by
Konstantin , thanks to him btw) and move all objects to this new pool
again in a new global transition.
But, I understand you'd recommend avoiding deleting/recreating STANDARD
and just modify the STANDARD data pool after GC execution, am I right?
Frédéric.
?
Best regards,
Frédéric.
Would that work?
Anyone tried this with success yet?
Best regards,
Frédéric.
--
Cordialement,
Frédéric Nass
Direction du Numérique
Sous-direction Infrastructures et Services
Tél : 03.72.74.11.35
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
