Re: trying to avoid a penalty for renaming every file

[Christian Rice <crice@xxxxxxxxxxx>]

argh, I note for sake of correction that I renamed my volume from REGEXED to YAY after the first set of tests.  Because I wanted to test rebuilding from completely nothing…but changing the name may add a touch of confusion for someone who was trying to comprehend…

anyway, a fabulous little feature, I hope it’s stable :)

From: <gluster-users-bounces@xxxxxxxxxxx> on behalf of Christian Rice <crice@xxxxxxxxxxx>
Date: Thursday, October 27, 2016 at 10:02 PM
To: Joe Julian <joe@xxxxxxxxxxxxxxxx>
Cc: "gluster-users@xxxxxxxxxxx" <gluster-users@xxxxxxxxxxx>
Subject: Re: [Gluster-users] trying to avoid a penalty for renaming every file

That is great.  I appreciate the encouragement to continue in the right direction.  What a relief!

I’m including my test procedure, in case it helps someone to visualize in ways a slide cannot.  Using Debian 7/Wheezy, gluster bits 3.7.15-1, two hosts: burr and van

from burr:

gluster peer probe van

vol create YAY transport tcp burr:/archive/gluster van:/archive/gluster

vol set YAY cluster.extra-hash-regex "(.*)\.COPYING"

check to make sure it looks right in /var/lib/glusterd/vols/YAY/trusted-REGEXED.tcp-fuse.vol

vol start YAY

mount -t glusterfs 127.0.0.1:REGEXED ~crice/tiempo

vol info on both hosts, it should look like so:

crice@burr:~$ sudo gluster vol info

 

Volume Name: REGEXED

Type: Distribute

Volume ID: e751a668-97dc-4638-81cd-40b68c2438f2

Status: Started

Number of Bricks: 2

Transport-type: tcp

Bricks:

Brick1: burr:/archive/gluster

Brick2: van:/archive/gluster

Options Reconfigured:

performance.readdir-ahead: on

cluster.extra-hash-regex: "(.*)\.COPYING"

create two dirs, testA and testB

testA will have a ten files named foo1-10, but they were created as foo1-10.COPYING then moved

testB wil have ten fies named bar1.10, but they were created as bar1-10.BAD

cd testA

for i in `seq 1 10`; do dd if=/dev/zero of=foo.${i}.COPYING bs=1M count=10; mv foo.${i}.COPYING foo.$i; done

cd testB

for i in `seq 1 10`; do dd if=/dev/zero of=foo.${i}.BAD bs=1M count=10; mv foo.${i}.BAD foo.$i; done

then check /archive/gluster/test{A,B} for link files on both burr and van.  The testA dir should have no link files.

results:

crice@burr:/archive/gluster$ ls -lR testA testB

testA:

total 61440

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:16 foo.10

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:16 foo.3

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:16 foo.4

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:16 foo.5

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:16 foo.6

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:16 foo.7

testB:

total 61440

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:17 foo.1

---------T 2 crice crice        0 Oct 27 21:17 foo.10

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:17 foo.2

---------T 2 crice crice        0 Oct 27 21:17 foo.3

---------T 2 crice crice        0 Oct 27 21:17 foo.4

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:17 foo.5

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:17 foo.6

---------T 2 crice crice        0 Oct 27 21:17 foo.7

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:17 foo.8

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:17 foo.9

crice@van:/archive/gluster$ ls -lR testA testB

testA:

total 40960

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:16 foo.1

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:16 foo.2

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:16 foo.8

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:16 foo.9

testB:

total 40960

---------T 2 crice crice        0 Oct 27 21:17 foo.1

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:17 foo.10

---------T 2 crice crice        0 Oct 27 21:17 foo.2

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:17 foo.3

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:17 foo.4

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:17 foo.7

---------T 2 crice crice        0 Oct 27 21:17 foo.8

---------T 2 crice crice        0 Oct 27 21:17 foo.9

that totally effin worked!  and validated by repeating procedure based on my notes

But having been tested using fuse, to be absolutely sure (because I don’t wanna read code)  off I ran:

gluster vol set YAY nfs.disable false (glusterd immediately died, so I restarted glusterfs-server, created a testC and testD dir and ran one more set over NFS:

cd /mnt/net/burr/YAY/

mkdir testC testD

cd testC

for i in `seq 1 10`; do dd if=/dev/zero of=foo.${i}.COPYING bs=1M count=10; mv foo.${i}.COPYING foo.$i; done

cd testD

for i in `seq 1 10`; do dd if=/dev/zero of=foo.${i}.BAD bs=1M count=10; mv foo.${i}.BAD foo.$i; done

and it was also good:

crice@burr:/archive/gluster$ ls -lR testC testD

testC:

total 98304

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:41 foo.10

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:41 foo.3

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:41 foo.4

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:41 foo.5

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:41 foo.6

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:41 foo.7

testD:

total 65536

---------T 2 crice crice        0 Oct 27 21:42 foo.1

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:42 foo.10

---------T 2 crice crice        0 Oct 27 21:42 foo.2

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:42 foo.3

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:42 foo.4

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:42 foo.7

---------T 2 crice crice        0 Oct 27 21:42 foo.8

---------T 2 crice crice        0 Oct 27 21:42 foo.9

crice@van:/archive/gluster$ ls -lR testC testD

testC:

total 65536

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:41 foo.1

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:41 foo.2

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:41 foo.8

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:41 foo.9

testD:

total 98304

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:42 foo.1

---------T 2 crice crice        0 Oct 27 21:42 foo.10

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:42 foo.2

---------T 2 crice crice        0 Oct 27 21:42 foo.3

---------T 2 crice crice        0 Oct 27 21:42 foo.4

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:42 foo.5

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:42 foo.6

---------T 2 crice crice        0 Oct 27 21:42 foo.7

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:42 foo.8

-rw-r--r-- 2 crice crice 10485760 Oct 27 21:42 foo.9

happy happy!  Thanks for sharing!

From: <gluster-users-bounces@xxxxxxxxxxx> on behalf of Joe Julian <joe@xxxxxxxxxxxxxxxx>
Date: Thursday, October 27, 2016 at 1:23 PM
To: "gluster-users@xxxxxxxxxxx" <gluster-users@xxxxxxxxxxx>
Subject: Re: [Gluster-users] trying to avoid a penalty for renaming every file

The best slide of all from the Gluster Developer Summit

https://twitter.com/JoeCyberGuru/status/784310321980674048

On 10/27/2016 11:52 AM, Christian Rice wrote:

We have a gluster pool where our developers are writing large files with the suffix .COPYING, then removing that suffix when the copy is done (renaming the file), so consumers of the data know the file is safe to read.

Back in 2014, Jeff Darcy wrote the below in a thread.  I guess I am basically asking if this is still the supported/encouraged approach, and how do I implement this when I’m using gluster’s NFS?

"That said, there's also a trick you can use to avoid creation of a

linkfile.  Other tools, such as rsync and our own object interface,

use the same write-then-rename idiom.  To serve them, there's an

option called extra-hash-regex that can be used to place files on the

"right" brick according to their final name even though they're created

with another.  Unfortunately, specifying that option via the command line

doesn't seem to work (it creates a malformed volfile) so you have to

mount a bit differently.  For example:

   glusterfs --volfile-server=a_server --volfile-id=a_volume \

   --xlator-option a_volume-dht.extra_hash_regex='(.*+)tmp' \

   /a/mountpoint

The important part is that second line.  That causes any file with a

"tmp" suffix to be hashed and placed as though only the part in the

first parenthesized part of the regex (i.e. without the "tmp") was

there.  Therefore, creating "xxxtmp" and then renaming it to "xxx" is

the same as just creating "xxx" in the first place as far as linkfiles

etc. are concerned.  Note that the excluded part can be anything that

a regex can match, including a unique random number.  If I recall,

rsync uses temp files something like this:

   fubar = .fubar.NNNNNN (where NNNNNNN is a random number)

I know this probably seems a little voodoo-ish, but with a little bit

of experimentation to find the right regex you should be able to avoid

those dreaded linkfiles altogether."

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxxhttp://www.gluster.org/mailman/listinfo/gluster-users

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users