On Fri, Aug 19, 2016 at 02:53:57PM +0530, Alok Srivastava wrote: > On Wed, Aug 10, 2016 at 1:35 AM, Vijay Bellur <vbellur@xxxxxxxxxx> wrote: > > > On 08/08/2016 04:57 AM, Alok Srivastava wrote: > > > >> Reviving the old thread.. > >> > >> I was thinking about a possible NDMP based migration tool built into > >> GlusterFS. > >> Migration from any other file system storage (source) to gluster storage > >> (target) can be challenging because of lack of a tool built into > >> GlusterFS. Host based migration is a common approach but it can be > >> painful for large datasets. > >> > >> The proposed migration tool will transfer data over the Network Data > >> Management Protocol (NDMP). This tool will make use of the snapshots > >> feature on the source system to ensure that a point-in- time copy of the > >> file system is transferred to the target. The proposed tool can also be > >> utilized to migrate from gluster on old hardware to gluster on new > >> hardware. > >> > >> Following is the proposed workflow: > >> > >> *1) **First copy* > >> > >> Migrate recursively from source directory to the gluster cluster. This > >> is expected to be a time consuming step. > >> > >> * 2) Incremental copy* > >> > >> Migrating any changes that occur after the first copy. > >> > >> 3) Cutover > >> > >> Restrict clients from accessing data on the source , migrate remaining > >> changes , and redirect clients to access their data on the gluster > >> cluster. > >> > >> > >> Note that there may be issues with migrating existing file permissions, > >> quota enforcements, etc on the source. > >> > >> Missing pieces in gluster > >> > >> > >> -NDMP support > >> > >> -May have to revive “ndmp-server” project? > >> > >> > >> > > > > Why is NDMP better than rsync from NFS (src) to FUSE (destination)?. Is it > > owing to the incremental copy that NDMP can handle better? > > > > Right, Determining the files for incremental migration will take longer > with rsync. Also, the proposed approach is based on NDMP + Snapshots, Hence > it's not a one size fits all approach. However, Making use of the snapshots > will ensure that a point in time copy is migrated and the in-flight > directories are also accessible to the clients connected to the source. > > NDMP support with gluster will help in integration with other backup > utilities as well. Do you have examples of backup utilities/communities that would like to see NDMP support? I have never seen any requests from users about this. We already have a nice integration with Bareos, and that supports NDMP I think. Maybe it would be a better approach to let backup software handle the NDMP part, and Gluster just offers a nice integrated way to consume point-in-time snapshots? Niels > > > > > I have not seen the ndmp-server project in action or the code to comment > > on how functional it was. We might have to implement from scratch if we > > decide to go down this route. > > > > Aravinda - do you remember the state of this project? > > > > Thanks, > > Vijay > > > > > > > _______________________________________________ > Gluster-devel mailing list > Gluster-devel@xxxxxxxxxxx > http://www.gluster.org/mailman/listinfo/gluster-devel
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
