FYI: While working on another project, I discovered that the file system attributes had been restored to their original state (or at least what I remember their original state to be): > [eric at sn1 srv]$ for x in sda7 sdb7 sda8 sdb8 ; do sudo getfattr -m - /srv/$x 2> /dev/null ; done > # file: srv/sda7 > trusted.afr.Repositories-client-0 > trusted.afr.Repositories-client-1 > trusted.gfid > trusted.glusterfs.dht > trusted.glusterfs.volume-id > > # file: srv/sdb7 > trusted.afr.Repositories-client-2 > trusted.afr.Repositories-client-3 > trusted.gfid > trusted.glusterfs.dht > trusted.glusterfs.volume-id > > ... Eric Pretorious Truckee, CA >________________________________ > From: Eric <epretorious at yahoo.com> >To: "gluster-users at gluster.org" <gluster-users at gluster.org> >Sent: Wednesday, September 5, 2012 9:34 PM >Subject: Re: migration operations: Stopping a migration > > >I chose to clear the attributes, leave the files intact, and replace the new brick with the old brick: > > >> [eric at sn1 srv]$ for x in `sudo getfattr -m - /srv/sda7 2> /dev/null | tail -n +2` ; do sudo setfattr -x $x /srv/sda7 ; done > > >> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda8 192.168.1.1:/srv/sda7 start >> replace-brick started successfully >> >> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda8 192.168.1.1:/srv/sda7 status >> Number of files migrated = 24631??????? Migration complete >> >> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda8 192.168.1.1:/srv/sda7 commit >> replace-brick commit successful >> >> gluster> volume info Repositories >>? >> Volume Name: Repositories >> Type: Distributed-Replicate >> Volume ID: 926262ae-2aa6-4bf7-b19e-cf674431b06c >> Status: Started >> Number of Bricks: 2 x 2 = 4 >> Transport-type: tcp >> Bricks: >> Brick1: 192.168.1.1:/srv/sda7 >> Brick2: 192.168.1.2:/srv/sda7 >> Brick3: 192.168.1.1:/srv/sdb7 >> Brick4: 192.168.1.2:/srv/sdb7 > > >...but the file system attributes of /srv/sda7 (i.e., the old brick) are not the same as they were when I began this journey: > > >> [eric at sn1 srv]$ for x in sda7 sdb7 sda8 ; do sudo getfattr -m - /srv/$x 2> /dev/null ; done >> # file: srv/sda7 >> trusted.gfid >> trusted.glusterfs.volume-id >> >> # file: srv/sdb7 >> trusted.afr.Repositories-client-2 >> trusted.afr.Repositories-client-3 >> trusted.gfid >> trusted.glusterfs.dht >> trusted.glusterfs.volume-id > > >Why is this? What will the long-term effects of this change? Are there other features that depend on those attributes? > > > >(The *original* attributes of /srv/sda7 are at the very bottom of this message.) > > >Eric Pretorious >Truckee, CA > > >>________________________________ >> From: Eric <epretorious at yahoo.com> >>To: "gluster-users at gluster.org" <gluster-users at gluster.org> >>Sent: Wednesday, September 5, 2012 9:08 PM >>Subject: Re: migration operations: Stopping a migration >> >> >>On yet another related note: I noticed that - consistent with the keeping the file system attributes - the files them selves are left intact on the old brick. Once I remove the file system attributes... >> >> >>> # for x in `getfattr -m - /srv/sda7 2> /dev/null | tail -n +2` ; do setfattr -x $x /srv/sda7 ; done >> >> >>....should I remove the old files? Or should I leave the files intact and migrate back to the original brick: >> >> >>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda7 192.168.1.1:/srv/sda8 start >> >> >> >>...and then heal the volume? >> >> >>Eric Pretorious >>Truckee, CA >> >> >> >> >>>________________________________ >>> From: Eric <epretorious at yahoo.com> >>>To: "gluster-users at gluster.org" <gluster-users at gluster.org> >>>Sent: Wednesday, September 5, 2012 5:42 PM >>>Subject: Re: migration operations: Stopping a migration >>> >>> >>>On a related note: Now that the PoC has been completed, I'm not able to migrate back to the original brick because of the undocumented, save-you-from-yourself file system attribute feature: >>> >>> >>>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda8 192.168.1.1:/srv/sda7 start >>>> /srv/sda7 or a prefix of it is already part of a volume >>> >>> >>> >>>Is there a simpler, more-direct method of migrating back to the original brick or should I wipe the file system attributes manually? I only ask because: >>> >>> >>>1. the long-term effects of this strategy aren't addressed in the Administration Guide AFAICT, and; >>> >>>2. though the intent of the feature has merit, it lacks elegance. e.g., the addition of a "force" attribute >>> >>>?? (like that of the commit feature) could be justified in this instance, IMHO. >>> >>> >>>?? > gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda8 192.168.1.1:/srv/sda7 start force >>>?? > Usage: volume replace-brick <VOLNAME> <BRICK> <NEW-BRICK> {start|pause|abort|status|commit [force]} >>> >>> >>> >>>Eric Pretorious >>>Truckee, CA >>> >>> >>> >>> >>>>________________________________ >>>> From: Eric <epretorious at yahoo.com> >>>>To: "gluster-users at gluster.org" <gluster-users at gluster.org> >>>>Sent: Wednesday, September 5, 2012 5:27 PM >>>>Subject: Re: migration operations: Stopping a migration >>>> >>>> >>>>On a hunch, I attempted the "volume replace-brick <VOLNAME> <BRICK> <NEW-BRICK> commit" command and, without much fanfare, the volume information was updated: >>>> >>>> >>>>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda7 192.168.1.1:/srv/sda8 commit >>>>> replace-brick commit successful >>>>> >>>>> gluster> volume info >>>>>? >>>>> Volume Name: Repositories >>>>> Type: Distributed-Replicate >>>>> Volume ID: 926262ae-2aa6-4bf7-b19e-cf674431b06c >>>>> Status: Started >>>>> Number of Bricks: 2 x 2 = 4 >>>>> Transport-type: tcp >>>>> Bricks: >>>>> Brick1: 192.168.1.1:/srv/sda8 >>>>> Brick2: 192.168.1.2:/srv/sda7 >>>>> Brick3: 192.168.1.1:/srv/sdb7 >>>>> Brick4: 192.168.1.2:/srv/sdb7 >>>>> >>>>> gluster> volume status >>>>> Status of volume: Repositories >>>>> Gluster process??? ??? ??? ??? ??? ??? Port??? Online??? Pid >>>>> ------------------------------------------------------------------------------ >>>>> Brick 192.168.1.1:/srv/sda8??? ??? ??? ??? 24012??? Y??? 13796 >>>>> Brick 192.168.1.2:/srv/sda7??? ??? ??? ??? 24009??? Y??? 4946 >>>>> Brick 192.168.1.1:/srv/sdb7??? ??? ??? ??? 24010??? Y??? 5438 >>>>> Brick 192.168.1.2:/srv/sdb7??? ??? ??? ??? 24010??? Y??? 4951 >>>>> NFS Server on localhost??? ??? ??? ??? ??? 38467??? Y??? 13803 >>>>> Self-heal Daemon on localhost??? ??? ??? ??? N/A??? Y??? 13808 >>>>> NFS Server on 192.168.1.2??? ??? ??? ??? 38467??? Y??? 7969 >>>>> Self-heal Daemon on 192.168.1.2??? ??? ??? ??? N/A??? Y??? 7974 >>>> >>>> >>>>The XFS attributes are still intact on the old brick, however: >>>> >>>> >>>>> [eric at sn1 ~]$ for x in sda7 sdb7 sda8 ; do sudo getfattr -m - /srv/$x 2> /dev/null ; done >>>>> # file: srv/sda7 >>>>> trusted.afr.Repositories-client-0 >>>>> trusted.afr.Repositories-client-1 >>>>> trusted.afr.Repositories-io-threads >>>>> trusted.afr.Repositories-replace-brick >>>>> trusted.gfid >>>>> trusted.glusterfs.dht >>>>> trusted.glusterfs.volume-id >>>>> >>>>> # file: srv/sdb7 >>>>> trusted.afr.Repositories-client-2 >>>>> trusted.afr.Repositories-client-3 >>>>> trusted.gfid >>>>> trusted.glusterfs.dht >>>>> trusted.glusterfs.volume-id >>>>> >>>>> # file: srv/sda8 >>>>> trusted.afr.Repositories-io-threads >>>>> trusted.afr.Repositories-replace-brick >>>>> trusted.gfid >>>>> trusted.glusterfs.volume-id >>>> >>>> >>>> >>>>Is this intentional (i.e., leaving the the attributes intact)? Or functionality that has yet to be implemented? >>>> >>>> >>>>Eric Pretorious >>>>Truckee, CA >>>> >>>> >>>> >>>>>________________________________ >>>>> From: Eric <epretorious at yahoo.com> >>>>>To: "gluster-users at gluster.org" <gluster-users at gluster.org> >>>>>Sent: Wednesday, September 5, 2012 5:05 PM >>>>>Subject: migration operations: Stopping a migration >>>>> >>>>> >>>>>I've created a distributed replicated volume: >>>>> >>>>> >>>>>> gluster> volume info >>>>>>? >>>>>> Volume Name: Repositories >>>>>> Type: Distributed-Replicate >>>>>> Volume ID: 926262ae-2aa6-4bf7-b19e-cf674431b06c >>>>>> Status: Started >>>>>> Number of Bricks: 2 x 2 = 4 >>>>>> Transport-type: tcp >>>>>> Bricks: >>>>>> Brick1: 192.168.1.1:/srv/sda7 >>>>>> Brick2: 192.168.1.2:/srv/sda7 >>>>>> Brick3: 192.168.1.1:/srv/sdb7 >>>>>> Brick4: 192.168.1.2:/srv/sdb7 >>>>> >>>>> >>>>> >>>>>...and begun migrating data from one brick to another as a PoC: >>>>> >>>>> >>>>>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda7 192.168.1.1:/srv/sda8 start >>>>>> replace-brick started successfully >>>>>> >>>>>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda7 192.168.1.1:/srv/sda8 status >>>>>> Number of files migrated = 5147?????? Current file= /centos/5.8/os/x86_64/CentOS/gnome-pilot-conduits-2.0.13-7.el5.x86_64.rpm >>>>>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda7 192.168.1.1:/srv/sda8 status >>>>>> Number of files migrated = 24631??????? Migration complete >>>>> >>>>> >>>>>After the migration is finished, though, the list of bricks is wrong: >>>>> >>>>> >>>>> >>>>>> gluster> volume heal Repositories info????????????????????????????????????????????????????????? >>>>>> Heal operation on volume Repositories has been successful >>>>>> >>>>>> Brick 192.168.1.1:/srv/sda7 >>>>>> Number of entries: 0 >>>>>> >>>>>> Brick 192.168.1.2:/srv/sda7 >>>>>> Number of entries: 0 >>>>>> >>>>>> Brick 192.168.1.1:/srv/sdb7 >>>>>> Number of entries: 0 >>>>>> >>>>>> Brick 192.168.1.2:/srv/sdb7 >>>>>> Number of entries: 0 >>>>> >>>>> >>>>>...and the XFS attributes are still intact on the old brick: >>>>> >>>>> >>>>>> [eric at sn1 ~]$ for x in sda7 sdb7 sda8 ; do sudo getfattr -m - /srv/$x 2> /dev/null ; done >>>>>> # file: srv/sda7 >>>>>> trusted.afr.Repositories-client-0 >>>>>> trusted.afr.Repositories-client-1 >>>>>> trusted.afr.Repositories-io-threads >>>>>> trusted.afr.Repositories-replace-brick >>>>>> trusted.gfid >>>>>> trusted.glusterfs.dht >>>>>> trusted.glusterfs.pump-path >>>>>> trusted.glusterfs.volume-id >>>>>> >>>>>> # file: srv/sdb7 >>>>>> trusted.afr.Repositories-client-2 >>>>>> trusted.afr.Repositories-client-3 >>>>>> trusted.gfid >>>>>> trusted.glusterfs.dht >>>>>> trusted.glusterfs.volume-id >>>>>> >>>>>> # file: srv/sda8 >>>>>> trusted.afr.Repositories-io-threads >>>>>> trusted.afr.Repositories-replace-brick >>>>>> trusted.gfid >>>>>> trusted.glusterfs.volume-id >>>>> >>>>> >>>>> >>>>>Have I missed a step? Or: Is this (i.e., clean-up) a bug or functionality that hasn't been implemented yet? >>>>> >>>>> >>>>>Eric Pretorious >>>>>Truckee, CA >>>>> >>>>>_______________________________________________ >>>>>Gluster-users mailing list >>>>>Gluster-users at gluster.org >>>>>http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >>>>> >>>>> >>>>> >>>>_______________________________________________ >>>>Gluster-users mailing list >>>>Gluster-users at gluster.org >>>>http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >>>> >>>> >>>> >>>_______________________________________________ >>>Gluster-users mailing list >>>Gluster-users at gluster.org >>>http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >>> >>> >>> >>_______________________________________________ >>Gluster-users mailing list >>Gluster-users at gluster.org >>http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >> >> >> >_______________________________________________ >Gluster-users mailing list >Gluster-users at gluster.org >http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://gluster.org/pipermail/gluster-users/attachments/20120905/46f9ea22/attachment-0001.htm>