On Wed, Aug 10, 2016 at 1:35 AM, Vijay Bellur <vbellur@xxxxxxxxxx> wrote:
On 08/08/2016 04:57 AM, Alok Srivastava wrote:
Reviving the old thread..
I was thinking about a possible NDMP based migration tool built into
GlusterFS.
Migration from any other file system storage (source) to gluster storage
(target) can be challenging because of lack of a tool built into
GlusterFS. Host based migration is a common approach but it can be
painful for large datasets.
The proposed migration tool will transfer data over the Network Data
Management Protocol (NDMP). This tool will make use of the snapshots
feature on the source system to ensure that a point-in- time copy of the
file system is transferred to the target. The proposed tool can also be
utilized to migrate from gluster on old hardware to gluster on new hardware.
Following is the proposed workflow:
*1) **First copy*
Migrate recursively from source directory to the gluster cluster. This
is expected to be a time consuming step.
* 2) Incremental copy*
Migrating any changes that occur after the first copy.
3) Cutover
Restrict clients from accessing data on the source , migrate remaining
changes , and redirect clients to access their data on the gluster cluster.
Note that there may be issues with migrating existing file permissions,
quota enforcements, etc on the source.
Missing pieces in gluster
-NDMP support
-May have to revive “ndmp-server” project?
Why is NDMP better than rsync from NFS (src) to FUSE (destination)?. Is it owing to the incremental copy that NDMP can handle better?
Right, Determining the files for incremental migration will take longer with rsync. Also, the proposed approach is based on NDMP + Snapshots, Hence it's not a one size fits all approach. However, Making use of the snapshots will ensure that a point in time copy is migrated and the in-flight directories are also accessible to the clients connected to the source.
NDMP support with gluster will help in integration with other backup utilities as well.
I have not seen the ndmp-server project in action or the code to comment on how functional it was. We might have to implement from scratch if we decide to go down this route.
Aravinda - do you remember the state of this project?
Thanks,
Vijay
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
