>What network do you have? We have 2X10G bonded interfaces on each server. Thanks to Xavier for detailed explanation of EC details. On Sat, May 6, 2017 at 2:20 AM, Alastair Neil <ajneil.tech@xxxxxxxxx> wrote: > What network do you have? > > > On 5 May 2017 at 09:51, Serkan Çoban <cobanserkan@xxxxxxxxx> wrote: >> >> In our use case every node has 26 bricks. I am using 60 nodes, one 9PB >> volume with 16+4 EC configuration, each brick in a sub-volume is on >> different host. >> We put 15-20k 2GB files every day into 10-15 folders. So it is 1500K >> files/folder. Our gluster version is 3.7.11. >> Heal speed in this environment is 8-10MB/sec/brick. >> >> I did some tests for parallel self heal feature with version 3.9, two >> servers 26 bricks each, 8+2 and 16+4 EC configuration. >> This was a small test environment and the results are as I said 8+2 is >> 2x faster then 16+4 with parallel self heal threads set to 2/4. >> In 1-2 months our new servers arriving, I will do detailed tests for >> heal performance for 8+2 and 16+4 and inform you the results. >> >> >> On Fri, May 5, 2017 at 2:54 PM, Pranith Kumar Karampuri >> <pkarampu@xxxxxxxxxx> wrote: >> > >> > >> > On Fri, May 5, 2017 at 5:19 PM, Pranith Kumar Karampuri >> > <pkarampu@xxxxxxxxxx> wrote: >> >> >> >> >> >> >> >> On Fri, May 5, 2017 at 2:38 PM, Serkan Çoban <cobanserkan@xxxxxxxxx> >> >> wrote: >> >>> >> >>> It is the over all time, 8TB data disk healed 2x faster in 8+2 >> >>> configuration. >> >> >> >> >> >> Wow, that is counter intuitive for me. I will need to explore about >> >> this >> >> to find out why that could be. Thanks a lot for this feedback! >> > >> > >> > From memory I remember you said you have a lot of small files hosted on >> > the >> > volume, right? It could be because of the bug >> > https://review.gluster.org/17151 is fixing. That is the only reason I >> > could >> > guess right now. We will try to test this kind of case if you could give >> > us >> > a bit more details about average file-size/depth of directories etc to >> > simulate similar looking directory structure. >> > >> >> >> >> >> >>> >> >>> >> >>> On Fri, May 5, 2017 at 10:00 AM, Pranith Kumar Karampuri >> >>> <pkarampu@xxxxxxxxxx> wrote: >> >>> > >> >>> > >> >>> > On Fri, May 5, 2017 at 11:42 AM, Serkan Çoban >> >>> > <cobanserkan@xxxxxxxxx> >> >>> > wrote: >> >>> >> >> >>> >> Healing gets slower as you increase m in m+n configuration. >> >>> >> We are using 16+4 configuration without any problems other then >> >>> >> heal >> >>> >> speed. >> >>> >> I tested heal speed with 8+2 and 16+4 on 3.9.0 and see that heals >> >>> >> on >> >>> >> 8+2 is faster by 2x. >> >>> > >> >>> > >> >>> > As you increase number of nodes that are participating in an EC set >> >>> > number >> >>> > of parallel heals increase. Is the heal speed you saw improved per >> >>> > file >> >>> > or >> >>> > the over all time it took to heal the data? >> >>> > >> >>> >> >> >>> >> >> >>> >> >> >>> >> On Fri, May 5, 2017 at 9:04 AM, Ashish Pandey <aspandey@xxxxxxxxxx> >> >>> >> wrote: >> >>> >> > >> >>> >> > 8+2 and 8+3 configurations are not the limitation but just >> >>> >> > suggestions. >> >>> >> > You can create 16+3 volume without any issue. >> >>> >> > >> >>> >> > Ashish >> >>> >> > >> >>> >> > ________________________________ >> >>> >> > From: "Alastair Neil" <ajneil.tech@xxxxxxxxx> >> >>> >> > To: "gluster-users" <gluster-users@xxxxxxxxxxx> >> >>> >> > Sent: Friday, May 5, 2017 2:23:32 AM >> >>> >> > Subject: disperse volume brick counts limits in >> >>> >> > RHES >> >>> >> > >> >>> >> > >> >>> >> > Hi >> >>> >> > >> >>> >> > we are deploying a large (24node/45brick) cluster and noted that >> >>> >> > the >> >>> >> > RHES >> >>> >> > guidelines limit the number of data bricks in a disperse set to >> >>> >> > 8. >> >>> >> > Is >> >>> >> > there >> >>> >> > any reason for this. I am aware that you want this to be a power >> >>> >> > of >> >>> >> > 2, >> >>> >> > but >> >>> >> > as we have a large number of nodes we were planning on going with >> >>> >> > 16+3. >> >>> >> > Dropping to 8+2 or 8+3 will be a real waste for us. >> >>> >> > >> >>> >> > Thanks, >> >>> >> > >> >>> >> > >> >>> >> > Alastair >> >>> >> > >> >>> >> > >> >>> >> > _______________________________________________ >> >>> >> > Gluster-users mailing list >> >>> >> > Gluster-users@xxxxxxxxxxx >> >>> >> > http://lists.gluster.org/mailman/listinfo/gluster-users >> >>> >> > >> >>> >> > >> >>> >> > _______________________________________________ >> >>> >> > Gluster-users mailing list >> >>> >> > Gluster-users@xxxxxxxxxxx >> >>> >> > http://lists.gluster.org/mailman/listinfo/gluster-users >> >>> >> _______________________________________________ >> >>> >> Gluster-users mailing list >> >>> >> Gluster-users@xxxxxxxxxxx >> >>> >> http://lists.gluster.org/mailman/listinfo/gluster-users >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > -- >> >>> > Pranith >> >> >> >> >> >> >> >> >> >> -- >> >> Pranith >> > >> > >> > >> > >> > -- >> > Pranith >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users@xxxxxxxxxxx >> http://lists.gluster.org/mailman/listinfo/gluster-users > > _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://lists.gluster.org/mailman/listinfo/gluster-users
