Hello, After the important Bron's explanation, I have been running some preliminary tests using fio in order to raise errors. I had to start on simple small local Debian Linux 6.x Squeeze VMs before trying to stress a XenServer environment hosts, network and storage. Chose NFSv4 because of, presumably, better locking and delegations. Tried some of the fio ioengines. Until now, there were no errors. Please, verify if there are misconceptions or any errors. If not, will try on Debian Linux 6.x Squeeze VMs at production environment before setting up a cyrus. ==== server /etc/default/nfs-common add the following #AFM 20130326 NEED_IDMAPD=YES ==== server /etc/exports Add the following #AFM20130326 /exports 192.168.0.0/255.255.255.0(sync,rw,no_root_squash,no_subtree_check,crossmnt,fsid=0) invoke-rc.d portmap restart invoke-rc.d nfs-common restart invoke-rc.d nfs-kernel-server restart ==== client mkdir -p /mnt/nfsimports mount -t nfs4 -o timeo=900,retrans=3,intr,proto=tcp,lock,vers=4,actimeo=10 192.168.0.110:/ /mnt/nfsimports Only for tests, without security !!!! chmod -R 777 /mnt/nfsimports ==== fio tests /home/andremachado/fio_randomrw_1kthreads_nfsv4.txt ; random read write of data [random-rw] iodepth=64 rw=randrw #rw=randwrites rwmixwrite=90 #90% writes invalidate=1 #buffered=1 direct=1 #zfsfuse segfault fsync_on_close=1 size=4000m directory=/mnt/nfsimports numjobs=50 filesize=1k,75k runtime=1800 time_based verify=md5 verify_fatal=1 verify_async=10 #write_lat_log lockfile=readwrite fallocate=0 #cyrus, nfs use this? fadvise_hint=0 #cyrus, nfs use this? ##scramble_buffers=false #anti dedupe. not implemented in debian squeeze #ioengine=mmap #cyrus imap uses writev, write, fdatasync, mmap/memcpy, fclose ioengine=sync #ioengine=vsync #takes hours for finish, despite a 3 min time limit fdatasync=1 #cyrus number of writes before force fdatasync syscall #iomem=mmap #cyrus reads from mmap files do_verify=1 nrfiles=1 openfiles=1 file_service_type=random nice=19 prio=7 #highest 0, lowest 7 prioclass=2 #best effort priority #ioscheduler=deadline #squeeze linux does not support this at now unlink=1 #unlink files at end of job ==== run the test nice fio --output=/tmp/fio_output_nfs.log /home/andremachado/fio_randomrw_1kthreads_nfsv4.txt Some useful links: http://lists.andrew.cmu.edu/pipermail/info-cyrus/2013-March/thread.html ************************************************************ http://www.cyberciti.biz/faq/nfs4-server-debian-ubuntu-linux/ *********** http://www.cyberciti.biz/faq/howto-mount-nfs-file-system-over-busy-slow-network/ ******** http://wiki.debian.org/NFSTroubleshooting **** http://wiki.dovecot.org/NFS http://serverfault.com/questions/127615/how-does-nfs-read-cache-work-on-debian ***** http://www.redhat.com/resourcelibrary/whitepapers/nfsv4wp http://linux.derkeiler.com/Mailing-Lists/Debian/2012-12/msg01237.html http://nfsv4.bullopensource.org/doc/nfsv4.admin.frs.v03.php ***** http://linux-nfs.org/wiki/index.php/Cluster_Coherent_NFSv4_and_Delegations ************** http://storagegaga.wordpress.com/tag/nfsv4/ ********** http://lists.freebsd.org/pipermail/freebsd-stable/2006-March/023766.html use tcp http://dboptimizer.com/2012/07/16/nfs-versus-dnfs/ http://www.spinics.net/lists/fio/index.html fio mailing list http://www.spinics.net/lists/fio/msg01457.html http://www.dbi-services.com/index.php/blog/entry/flexible-io-simulating-database-like-io-activity-without-an-installed-database http://www.spinics.net/lists/fio/msg00813.html http://linux.die.net/man/1/fio http://heatware.net/linux-unix/create-large-many-number-files-thousands-millions/ http://stackoverflow.com/questions/13400312/linux-create-random-directory-file-hierarchy http://lists.debian.org/debian-user/2012/12/msg01253.html Regards Andre Felipe Machado http://www.techforce.com.br
---- Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
