EC pool migrations

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi all,

Wondering if anyone has come up with a quick and minimal impact way of
moving data between erasure coded pools? We want to shrink an existing
EC pool (also changing the EC profile at the same time) that backs our
main RGW buckets. Thus far the only successful way I've found of
managing the switch-over uses rados cppool like:

1) stop radosgw services
2) rados cppool <oldpool> <newpool>
3) ceph osd pool rename <oldpool> <oldpool>.old
4) ceph osd pool rename <newpool> <oldpool>
5) start radosgw services

This is slow and error prone. From what I can tell it actually
transfers the data between the pools via the client. Not good. Also
means radosgw service is offline throughout the transfer.

My next idea was to use cache tiering, but the obvious process there
(insert new pool as backing under old pool as cache) doesn't work
because the cache tier cannot be erasure coded. I'm now wondering if
it would be feasible to:

1) insert a large enough temporary replicated pool as a cache tier
2) somehow force promotion of every object into the cache (don't see
any way to do that other than actually read them - but at least some
creative scripting could do that in parallel)
3) once #objects in cache = #objects in old backing pool
then stop radosgw services
4) remove overlay and tier remove
6) now we should have identical or newer data in the temporary
replicated pool and no caching relationship
then add the temporary replicated pool as a tier (--force-nonempty) to
the new EC pool
7) finally cache-flush-evict-all and remove the temporary replicated pool

I'm starting to work through testing this now but would appreciate if
anyone can save me some time or suggest a better option.

-- 
Cheers,
~Blairo
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux