On Tue, May 13, 2014 at 11:36 AM, Fred Yang <frederic.yang at gmail.com> wrote: > I have to say I'm shocked to see the suggestion is rbd import/export if 'you > care the data'. These kind of operation is common use case and should be an > essential part of any distributed storage. What if I have a hundred node > cluster running for years and need to do hardware refresh? There are no > clear procedure itself sounds scary to me.. You misunderstand. Migrating between machines for incrementally upgrading your hardware is normal behavior and well-tested (likewise for swapping in all-new hardware, as long as you understand the IO requirements involved). So is decommissioning old hardware. But if you only care about (for instance, numbers pulled out of thin air) 30GB out of 100TB of data in the cluster, it will be *faster* to move only the 30GB you care about, instead of rebalancing all the data in the cluster across to new machines. :) -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com