Been there ... here is my 10cent advise a) Prepare for tomorrow b) Rest c) Think d) Plan e) act I am sure it will work for you when calmed Tech hints. ifconfig iface mtu 9000 or whatever your nic can afford Having a 100Mbit is not a good idea. I 've recently located a dual port 1Gbit nic on ebay for $15 USD Get them. And last but not least In case you happen to have a switch between the nodes make sure that you enable jumbo frames on it. Otherwise you are in DEEP Trouble. stat' ing a 120G file in my case takes miliseconds not even seconds. GL Harry. On 10/07/2013 02:01 ??, Allan Latham wrote: > Hi all > > Thanks to all those volunteers who are working to get gluster into a > state where it can be used for live work. > > I understand that you are giving your free time and I very much > appreciate it on this project and the many others we use for live > production work. > > There seems to be a problem with the way gluster is going. > For me it would be an ideal solution if it actually worked. > > I have a simple scenario and it just simply doesn't work. Reading over > the network when the file is available locally is plainly wrong. Our > application cannot take the performance hit nor the extra network traffic. > > I would suggest: > > 1. get a simple minimalist configuration working - 2 hosts and > replication only. > 2. make it bomb-proof. > 2a. it must cope with network failures, random reboots etc. > 2b. if it stops it has to auto-recover quickly. > 2c. if it can't it needs thorough documentation and adequate logs so a > reasonable sysop can rescue it. > 2d. it needs a fast validation scanner which verifies that data is where > it should be and is identical everywhere (md5sum). > 3. make it efficient (read local whenever possible - use rsync > techniques - remove scalability obstacles so it doesn't get > exponentially slower as more files are replicated) > 4. when that works expand to multiple hosts and clever distribution > techniques. > (repeat items 2 and 3 in the more complex environment) > > If it doesn't work rock solid in a simple scenario it will never work in > a large scale cluster. > > Until point 3 is reached I cannot use it - which is a great > disappointment for me as well as the good guys doing the development. > > Good luck and thanks again > > Allan > > > On 04/07/13 13:10, Allan Latham wrote: >> Hi all >> >> Does anyone use read-subvolume? >> >> Has anyone tested read-subvolume? >> >> Does read-subvolume work in such a way that if the file is present on >> the local node the local copy is used rather than a remote one? >> >> Alternatively is there any way to configure (or patch) gluster to always >> prefer the local file? >> >> I have read everything available and have found no answer. >> >> Unison works very well in our environment but is not real time and needs >> to be run every few minutes and/or be kicked off with inotify. >> >> If I could get gluster to always read the local copy it would be a much >> better drop in replacement. >> >> This is a small scale deployment not a massive cluster but I can imagine >> there are many potential users of gluster in this mode. It should beat >> unison and similar solutions in every way - but it doesn't because it is >> reading from the network even when it has a local up-to-date copy. This >> can't be intended behaviour. >> >> So what have I configured wrong? >> >> Thanks in advance >> >> Allan >> >> >> On 02/07/13 13:38, Allan Latham wrote: >>> Hi everyone >>> >>> I have installed 3.3.1-1 from the Debian repository you provide. >>> >>> I am using a simple 2 node cluster and running in replication mode. The >>> connection between the nodes is limited to 100MB/sec (that's bits not >>> bytes!). Usage will be mainly for read access and since there is always >>> a local copy available [ exactly 2 replicas on exactly 2 machines ] I >>> expect very fast read performance. Writes are low volume and very >>> infrequent - performance is not an issue. >>> >>> Almost everything works as I would expect. >>> >>> Write speed is limited to 10Mb (bytes) per second which is what I would >>> expect and is adequate for the application. >>> >>> But read speed is either super fast or 10Mb/sec. i.e. read operations >>> take place on the local copy or the remote seemingly at random. >>> >>> This not the 'small files problem'. I am aware that Gluster must use >>> network access for stat() etc. This is all about where the data comes >>> from on a read(). If I do an m5dum on a 200Mb file it takes either half >>> a second or 18 seconds. >>> >>> There is an option read-subvolume. >>> >>> I have tried to understand how this works from the documentation >>> available and from the few examples on the web. >>> >>> I have added the option using: >>> >>> gluster volume set X read-subvolume Y >>> >>> It has no effect even after stopping and starting the volume, >>> remounting, restarting gluster servers etc. >>> >>> What's more I fail to see how this option could ever work at all. The >>> configuration changes caused by the above command are rolled out to both >>> nodes - but what is right for one node is exactly the wrong >>> configuration for the other node. >>> >>> Configs attached are in /var/lib/glusterd/vols/shared except >>> glusterd.vol which is in /etc/glusterfs. >>> >>> Here is the output of the mount command filtered to just the glusterfs >>> mount: >>> >>> 10.255.255.1:/shared on /gluster/rw/shared type fuse.glusterfs >>> (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072) >>> >>> 10.255.255.1 is local to this host. >>> >>> I would be very thankful if someone can enlighten me. I am obviously >>> configuring this wrong. I may have missed something important. >>> >>> Best regards to all >>> >>> Allan >>> >>> >>> >>> _______________________________________________ >>> Gluster-users mailing list >>> Gluster-users at gluster.org >>> http://supercolony.gluster.org/mailman/listinfo/gluster-users >>> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> http://supercolony.gluster.org/mailman/listinfo/gluster-users >> > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://supercolony.gluster.org/mailman/listinfo/gluster-users
