OK, I've repeated the test with the following hierarchy: * 10 top-level folders with 10 second-level folders each; * 10 000 files in each second-level folder. So, this composes 10×10×10000=1M files and 100 folders Initial brick used space: 33 M Initial inodes count: 24 After test: * each brick in replica took 18G, and the arbiter brick took 836M; * inodes count: 1066036 So: (836 - 33) / (1066036 - 24) == 790 bytes per inode. So, yes, it is slightly bigger value than with previous test due to, I guess, lots of files in one folder, but it is still too far from 4k. Given a good engineer should consider 30% reserve, the ratio is about 1k per stored inode. Correct me if I'm missing something (regarding average workload and not corner cases). Test script is here: [1] Regards, Oleksandr. [1] http://termbin.com/qlvz On вівторок, 8 березня 2016 р. 19:13:05 EET Ravishankar N wrote: > On 03/05/2016 03:45 PM, Oleksandr Natalenko wrote: > > In order to estimate GlusterFS arbiter brick size, I've deployed test > > setup > > with replica 3 arbiter 1 volume within one node. Each brick is located on > > separate HDD (XFS with inode size == 512). Using GlusterFS v3.7.6 + > > memleak > > patches. Volume options are kept default. > > > > Here is the script that creates files and folders in mounted volume: [1] > > > > The script creates 1M of files of random size (between 1 and 32768 bytes) > > and some amount of folders. After running it I've got 1036637 folders. > > So, in total it is 2036637 files and folders. > > > > The initial used space on each brick is 42M . After running script I've > > got: > > > > replica brick 1 and 2: 19867168 kbytes == 19G > > arbiter brick: 1872308 kbytes == 1.8G > > > > The amount of inodes on each brick is 3139091. So here goes estimation. > > > > Dividing arbiter used space by files+folders we get: > > > > (1872308 - 42000)/2036637 == 899 bytes per file or folder > > > > Dividing arbiter used space by inodes we get: > > > > (1872308 - 42000)/3139091 == 583 bytes per inode > > > > Not sure about what calculation is correct. > > I think the first one is right because you still haven't used up all the > inodes.(2036637 used vs. the max. permissible 3139091). But again this > is an approximation because not all files would be 899 bytes. For > example if there are a thousand files present in a directory, then du > <dirname> would be more than du <file> because the directory will take > some disk space to store the dentries. > > > I guess we should consider the one > > > > that accounts inodes because of .glusterfs/ folder data. > > > > Nevertheless, in contrast, documentation [2] says it should be 4096 bytes > > per file. Am I wrong with my calculations? > > The 4KB is a conservative estimate considering the fact that though the > arbiter brick does not store data, it still keeps a copy of both user > and gluster xattrs. For example, if the application sets a lot of > xattrs, it can consume a data block if they cannot be accommodated on > the inode itself. Also there is the .glusterfs folder like you said > which would take up some space. Here is what I tried on an XFS brick: > [root@ravi4 brick]# touch file > > [root@ravi4 brick]# ls -l file > -rw-r--r-- 1 root root 0 Mar 8 12:54 file > > [root@ravi4 brick]# du file > *0 file** > * > [root@ravi4 brick]# for i in {1..100} > > > do > > setfattr -n user.value$i -v value$i file > > done > > [root@ravi4 brick]# ll -l file > -rw-r--r-- 1 root root 0 Mar 8 12:54 file > > [root@ravi4 brick]# du -h file > *4.0K file** > * > Hope this helps, > Ravi > > > Pranith? > > > > [1] http://termbin.com/ka9x > > [2] > > http://gluster.readthedocs.org/en/latest/Administrator%20Guide/arbiter-vo > > lumes-and-quorum/ _______________________________________________ > > Gluster-devel mailing list > > Gluster-devel@xxxxxxxxxxx > > http://www.gluster.org/mailman/listinfo/gluster-devel _______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
