Have you looked at CPU load on the server? Maybe the difference is just the CPU-power that NFS consumes and decryption is already CPU-Limited before. The low non-encrypted performance, compared to local performance, is also a bit fishy. Pure speculation, but maybe something is consuming CPU on the server or the CPU is slowed-down in some way. If this not a CPU-power issue, then the CPU should have significant idle percentage during encrypted accesses. Arno On Wed, Feb 08, 2012 at 03:04:57PM +0000, Mickael wrote: > >De?: Arno Wagner <arno@xxxxxxxxxxx> > >??: dm-crypt@xxxxxxxx > >Cc?: > >Envoy? le : Mardi 7 f?vrier 2012 9h33 > >Objet?: Re: Poor performances with nfs and Kernel 3.x > > > >O.k., no such obervation with 3.2.2 on client and server. I got > > > >root /gate/tmp>cat ttt | wcs > /dev/null > >read:? 2.147 GB [? 2147483648 B]? ? avg:? 69.274 MB/sec [? ? 31 sec] > > > >(wcs, a.k.a. wc-stream is a small tool I wrote to do real-time > >monitoring of pipeline throughput and byte count, sources > >below.) This is with some ramp-up and almost 100% CPU load > >on the (slower) server. > > > >I noticed though that nfsiostat is not the right tool to measure, as > >it gives you performance over the whole time the device has been > >mounted, i.e. the throughput number keeps being updated in > >real-time. > > > >Maybe re-run with wcs as shown above (compile instructions are in > >the header of wcs.c) and also monitor CPU usage on the server > >while this is rrunning, e.g. with "top". I am not saying you > >imagine the issue, but lets be sure the measurement is good. > > > >If you do not want to compile anything, you could also use > >something like "time cat ttt > /dev/null" and calculate > >throughput manually. > > > >Arno > > > >----- CUTed ----- > > > > > Hello Arno, > Here are the new tests with wcs: results are still the same. > > /tmp is the tmp directory on the client (7200 rpm HD) > /CRYPT is a crypted partition on the server (Green HD ~5600 rpm) > /SAV is a non-crypted partition on the server (Green HD ~5600 rpm) > CPU monitored with htop > > > *** Server with Kernel 3.0.0-15-server #26 > > * From client, using NFS on crypted partition: > > cat /tmp/testfile.avi | wcs > /mnt/CRYPT/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 41.476 MB/sec [??? 49 sec] > > cat /mnt/CRYPT/testfile.avi | wcs > /tmp/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 29.454 MB/sec [??? 69 sec]??? <--------------- > > ----------- > > * From client, using NFS on non-crypted partition > > cat /tmp/testfile.avi | wcs > /mnt/SAV/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 72.583 MB/sec [??? 28 sec] > > cat /mnt/SAV/testfile.avi | wcs > /tmp/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 65.559 MB/sec [??? 31 sec] > > ----------- > > > * From Server: > > cat /mnt/CRYPT/testfile.avi | wcs > /dev/null > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 72.583 MB/sec [??? 28 sec] > > cat /mnt/SAV/testfile.avi | wcs > /dev/null > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 67.744 MB/sec [??? 30 sec] > > ----------- > > cat /mnt/CRYPT/testfile.avi | wcs > /mnt/SAV/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 56.454 MB/sec [??? 36 sec] > > cat /mnt/SAV/testfile.avi | wcs > /mnt/CRYPT/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 54.928 MB/sec [??? 37 sec] > > > > =============================== > > *** Server with Kernel 2.6.38-13-server #52 > > * From client, using NFS on crypted partition: > > cat /tmp/testfile.avi | wcs > /mnt/CRYPT/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 54.928 MB/sec [??? 37 sec]??? CPU=100% > > cat /mnt/CRYPT/testfile.avi | wcs > /tmp/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 81.293 MB/sec [??? 25 sec]??? CPU=98%??? ??? <------------------- > > ----------- > > * From client, using NFS on non-crypted partition > > cat /tmp/testfile.avi | wcs > /mnt/SAV/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 63.510 MB/sec [??? 32 sec]??? CPU=50% > > cat /mnt/SAV/testfile.avi | wcs > /tmp/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 63.510 MB/sec [??? 32 sec]??? CPU=25% > > > > * From Server: > > cat /mnt/CRYPT/testfile.avi | wcs > /dev/null > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 84.680 MB/sec [??? 24 sec]??? CPU=98% > > cat /mnt/SAV/testfile.avi | wcs > /dev/null > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 67.744 MB/sec [??? 30 sec]??? CPU=25% > > ----------- > > cat /mnt/CRYPT/testfile.avi | wcs > /mnt/SAV/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 52.111 MB/sec [??? 39 sec]??? CPU=50%-100% > > cat /mnt/SAV/testfile.avi | wcs > /mnt/CRYPT/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 47.264 MB/sec [??? 43 sec]??? CPU=75%-100% > _______________________________________________ > dm-crypt mailing list > dm-crypt@xxxxxxxx > http://www.saout.de/mailman/listinfo/dm-crypt > -- Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@xxxxxxxxxxx GnuPG: ID: 1E25338F FP: 0C30 5782 9D93 F785 E79C 0296 797F 6B50 1E25 338F ---- One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision. -- Bertrand Russell _______________________________________________ dm-crypt mailing list dm-crypt@xxxxxxxx http://www.saout.de/mailman/listinfo/dm-crypt
