I chose to clear the attributes, leave the files intact, and replace the new brick with the old brick:
> [eric at sn1 srv]$ for x in `sudo getfattr -m - /srv/sda7 2> /dev/null | tail -n +2` ; do sudo setfattr -x $x /srv/sda7 ; done
> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda8 192.168.1.1:/srv/sda7 start
> replace-brick started successfully
>
> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda8 192.168.1.1:/srv/sda7 status
> Number of files migrated = 24631??????? Migration complete
>
> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda8 192.168.1.1:/srv/sda7 commit
> replace-brick commit successful
>
> gluster> volume info Repositories
>?
> Volume Name: Repositories
> Type: Distributed-Replicate
> Volume ID: 926262ae-2aa6-4bf7-b19e-cf674431b06c
> Status: Started
> Number of Bricks: 2 x 2 = 4
> Transport-type: tcp
> Bricks:
> Brick1: 192.168.1.1:/srv/sda7
> Brick2: 192.168.1.2:/srv/sda7
> Brick3: 192.168.1.1:/srv/sdb7
> Brick4: 192.168.1.2:/srv/sdb7
...but the file system attributes of /srv/sda7 (i.e., the old brick) are not the same as they were when I began this journey:
> [eric at sn1 srv]$ for x in sda7 sdb7 sda8 ; do sudo getfattr -m - /srv/$x 2> /dev/null ; done
> # file: srv/sda7
> trusted.gfid
> trusted.glusterfs.volume-id
>
> # file: srv/sdb7
> trusted.afr.Repositories-client-2
> trusted.afr.Repositories-client-3
> trusted.gfid
> trusted.glusterfs.dht
> trusted.glusterfs.volume-id
Why is this? What will the long-term effects of this change? Are there other features that depend on those attributes?
(The *original* attributes of /srv/sda7 are at the very bottom of this message.)
Eric Pretorious
Truckee, CA
>________________________________
> From: Eric <epretorious at yahoo.com>
>To: "gluster-users at gluster.org" <gluster-users at gluster.org>
>Sent: Wednesday, September 5, 2012 9:08 PM
>Subject: Re: migration operations: Stopping a migration
>
>
>On yet another related note: I noticed that - consistent with the keeping the file system attributes - the files them selves are left intact on the old brick. Once I remove the file system attributes...
>
>
>> # for x in `getfattr -m - /srv/sda7 2> /dev/null | tail -n +2` ; do setfattr -x $x /srv/sda7 ; done
>
>
>....should I remove the old files? Or should I leave the files intact and migrate back to the original brick:
>
>
>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda7 192.168.1.1:/srv/sda8 start
>
>
>
>...and then heal the volume?
>
>
>Eric Pretorious
>Truckee, CA
>
>
>
>
>>________________________________
>> From: Eric <epretorious at yahoo.com>
>>To: "gluster-users at gluster.org" <gluster-users at gluster.org>
>>Sent: Wednesday, September 5, 2012 5:42 PM
>>Subject: Re: migration operations: Stopping a migration
>>
>>
>>On a related note: Now that the PoC has been completed, I'm not able to migrate back to the original brick because of the undocumented, save-you-from-yourself file system attribute feature:
>>
>>
>>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda8 192.168.1.1:/srv/sda7 start
>>> /srv/sda7 or a prefix of it is already part of a volume
>>
>>
>>
>>Is there a simpler, more-direct method of migrating back to the original brick or should I wipe the file system attributes manually? I only ask because:
>>
>>
>>1. the long-term effects of this strategy aren't addressed in the Administration Guide AFAICT, and;
>>
>>2. though the intent of the feature has merit, it lacks elegance. e.g., the addition of a "force" attribute
>>
>>?? (like that of the commit feature) could be justified in this instance, IMHO.
>>
>>
>>?? > gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda8 192.168.1.1:/srv/sda7 start force
>>?? > Usage: volume replace-brick <VOLNAME> <BRICK> <NEW-BRICK> {start|pause|abort|status|commit [force]}
>>
>>
>>
>>Eric Pretorious
>>Truckee, CA
>>
>>
>>
>>
>>>________________________________
>>> From: Eric <epretorious at yahoo.com>
>>>To: "gluster-users at gluster.org" <gluster-users at gluster.org>
>>>Sent: Wednesday, September 5, 2012 5:27 PM
>>>Subject: Re: migration operations: Stopping a migration
>>>
>>>
>>>On a hunch, I attempted the "volume replace-brick <VOLNAME> <BRICK> <NEW-BRICK> commit" command and, without much fanfare, the volume information was updated:
>>>
>>>
>>>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda7 192.168.1.1:/srv/sda8 commit
>>>> replace-brick commit successful
>>>>
>>>> gluster> volume info
>>>>?
>>>> Volume Name: Repositories
>>>> Type:
Distributed-Replicate
>>>> Volume ID: 926262ae-2aa6-4bf7-b19e-cf674431b06c
>>>> Status: Started
>>>> Number of Bricks: 2 x 2 = 4
>>>> Transport-type: tcp
>>>> Bricks:
>>>> Brick1: 192.168.1.1:/srv/sda8
>>>> Brick2: 192.168.1.2:/srv/sda7
>>>> Brick3: 192.168.1.1:/srv/sdb7
>>>> Brick4: 192.168.1.2:/srv/sdb7
>>>>
>>>> gluster> volume status
>>>> Status of volume: Repositories
>>>> Gluster process??? ??? ??? ??? ??? ??? Port??? Online??? Pid
>>>> ------------------------------------------------------------------------------
>>>> Brick 192.168.1.1:/srv/sda8??? ??? ??? ??? 24012??? Y??? 13796
>>>> Brick 192.168.1.2:/srv/sda7??? ??? ??? ???
24009??? Y??? 4946
>>>> Brick 192.168.1.1:/srv/sdb7??? ??? ??? ??? 24010??? Y??? 5438
>>>> Brick 192.168.1.2:/srv/sdb7??? ??? ??? ??? 24010??? Y??? 4951
>>>> NFS Server on localhost??? ??? ??? ??? ??? 38467??? Y??? 13803
>>>> Self-heal Daemon on localhost??? ??? ??? ??? N/A??? Y??? 13808
>>>> NFS Server on 192.168.1.2??? ??? ??? ??? 38467??? Y??? 7969
>>>> Self-heal Daemon on 192.168.1.2??? ??? ???
??? N/A??? Y??? 7974
>>>
>>>
>>>The XFS attributes are still intact on the old brick, however:
>>>
>>>
>>>> [eric at sn1 ~]$ for x in sda7 sdb7 sda8 ; do sudo getfattr -m - /srv/$x 2> /dev/null ; done
>>>> # file: srv/sda7
>>>> trusted.afr.Repositories-client-0
>>>> trusted.afr.Repositories-client-1
>>>> trusted.afr.Repositories-io-threads
>>>> trusted.afr.Repositories-replace-brick
>>>> trusted.gfid
>>>> trusted.glusterfs.dht
>>>> trusted.glusterfs.volume-id
>>>>
>>>> # file: srv/sdb7
>>>> trusted.afr.Repositories-client-2
>>>> trusted.afr.Repositories-client-3
>>>> trusted.gfid
>>>> trusted.glusterfs.dht
>>>> trusted.glusterfs.volume-id
>>>>
>>>> # file:
srv/sda8
>>>> trusted.afr.Repositories-io-threads
>>>> trusted.afr.Repositories-replace-brick
>>>> trusted.gfid
>>>> trusted.glusterfs.volume-id
>>>
>>>
>>>
>>>Is this intentional (i.e., leaving the the attributes intact)? Or functionality that has yet to be implemented?
>>>
>>>
>>>Eric Pretorious
>>>Truckee, CA
>>>
>>>
>>>
>>>>________________________________
>>>> From: Eric <epretorious at yahoo.com>
>>>>To: "gluster-users at gluster.org" <gluster-users at gluster.org>
>>>>Sent: Wednesday, September 5, 2012 5:05 PM
>>>>Subject: migration operations: Stopping a migration
>>>>
>>>>
>>>>I've created a distributed replicated volume:
>>>>
>>>>
>>>>> gluster> volume info
>>>>>?
>>>>> Volume Name: Repositories
>>>>> Type: Distributed-Replicate
>>>>> Volume ID: 926262ae-2aa6-4bf7-b19e-cf674431b06c
>>>>> Status: Started
>>>>> Number of Bricks: 2 x 2 = 4
>>>>> Transport-type: tcp
>>>>> Bricks:
>>>>> Brick1: 192.168.1.1:/srv/sda7
>>>>> Brick2: 192.168.1.2:/srv/sda7
>>>>> Brick3: 192.168.1.1:/srv/sdb7
>>>>> Brick4: 192.168.1.2:/srv/sdb7
>>>>
>>>>
>>>>
>>>>...and begun migrating data from one brick to another as a PoC:
>>>>
>>>>
>>>>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda7 192.168.1.1:/srv/sda8 start
>>>>> replace-brick started successfully
>>>>>
>>>>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda7 192.168.1.1:/srv/sda8 status
>>>>> Number of files migrated = 5147?????? Current file= /centos/5.8/os/x86_64/CentOS/gnome-pilot-conduits-2.0.13-7.el5.x86_64.rpm
>>>>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda7
192.168.1.1:/srv/sda8 status
>>>>> Number of files migrated = 24631??????? Migration complete
>>>>
>>>>
>>>>After the migration is finished, though, the list of bricks is wrong:
>>>>
>>>>
>>>>
>>>>> gluster> volume heal Repositories info?????????????????????????????????????????????????????????
>>>>> Heal operation on volume Repositories has been successful
>>>>>
>>>>> Brick 192.168.1.1:/srv/sda7
>>>>> Number of entries: 0
>>>>>
>>>>> Brick 192.168.1.2:/srv/sda7
>>>>> Number of entries: 0
>>>>>
>>>>> Brick 192.168.1.1:/srv/sdb7
>>>>> Number of entries: 0
>>>>>
>>>>> Brick 192.168.1.2:/srv/sdb7
>>>>> Number of entries: 0
>>>>
>>>>
>>>>...and the XFS attributes are still intact on the old brick:
>>>>
>>>>
>>>>> [eric at sn1 ~]$ for x in sda7 sdb7 sda8 ; do sudo getfattr -m - /srv/$x 2> /dev/null ; done
>>>>> # file: srv/sda7
>>>>> trusted.afr.Repositories-client-0
>>>>> trusted.afr.Repositories-client-1
>>>>> trusted.afr.Repositories-io-threads
>>>>> trusted.afr.Repositories-replace-brick
>>>>> trusted.gfid
>>>>> trusted.glusterfs.dht
>>>>> trusted.glusterfs.pump-path
>>>>> trusted.glusterfs.volume-id
>>>>>
>>>>> # file:
srv/sdb7
>>>>> trusted.afr.Repositories-client-2
>>>>> trusted.afr.Repositories-client-3
>>>>> trusted.gfid
>>>>> trusted.glusterfs.dht
>>>>> trusted.glusterfs.volume-id
>>>>>
>>>>> # file: srv/sda8
>>>>> trusted.afr.Repositories-io-threads
>>>>> trusted.afr.Repositories-replace-brick
>>>>> trusted.gfid
>>>>> trusted.glusterfs.volume-id
>>>>
>>>>
>>>>
>>>>Have I missed a step? Or: Is this (i.e., clean-up) a bug or functionality that hasn't been implemented yet?
>>>>
>>>>
>>>>Eric Pretorious
>>>>Truckee, CA
>>>>
>>>>_______________________________________________
>>>>Gluster-users mailing list
>>>>Gluster-users at gluster.org
>>>>http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>>>
>>>>
>>>>
>>>_______________________________________________
>>>Gluster-users mailing list
>>>Gluster-users at gluster.org
>>>http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>>
>>>
>>>
>>_______________________________________________
>>Gluster-users mailing list
>>Gluster-users at gluster.org
>>http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>
>>
>>
>_______________________________________________
>Gluster-users mailing list
>Gluster-users at gluster.org
>http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gluster.org/pipermail/gluster-users/attachments/20120905/46c05111/attachment-0001.htm>
