I'm getting mixed results but it is looking like ZFS have something to do with the problem afterall.
I tried it with dm-crypt -> XFS -> NFS and I'm not seeing the same lag issue. I say mixed results mainly because I've been using rsync to do the testing and for some reason rsync has trouble with this setup. It works but really slowly. Like around 3 MB/sec over a Gigabit link. If I use "cp -pR" then it goes at 60 MB/sec. I use rsync all the time and I've never seen this before so it took a while for me to realize that rsync was acting up. Anyone run into this before?
I'll bring this to the ZFSonLinux people and see what they say.
Thanks,
Steve
On Thu, Mar 20, 2014 at 3:16 PM, Milan Broz <gmazyland@xxxxxxxxx> wrote:
On 03/20/2014 03:48 PM, Stephen Cousins wrote:Hi,
> I am testing dm-crypt with a system that serves data over NFS. I'm
> finding that when serving the data over NFS4 it will go for a little
> while and then it will just stop copying and networking on the NFS
> server will stall out so that other connections to other parts of the
> storage (a non-dm-crypt volume) will become very laggy. Soon after
> canceling the copy the other connections resume back to normal.
>
> I have tried this on two different servers, one with a single disk
> with ZFS and another with a ZFS raidz2 pool and they both do the same
> thing. CPU doesn't seem to be an issue. Doing local copies is not a
> problem. That is, copying to/from these volumes made up of dm-crypted
> disks from other disks on the server have no problems.
>
> One system (the one with the RAID pool) is CentOS 6.4 and the other
> one is CentOS 6.5.
>
> I'm curious if anyone else has run into this before and if so, what
> can be done to make it work? I'm going to test to see if taking ZFS
> out of the equation helps. I haven't run into this with ZFS with
> disks without dm-crypt.
unfortunately the dmcrypt code in Centos 6.x is not what is upstream
kernel (mainly because of per cpu processing changes upstream).
(It would be interesting if it is reproducible with upstream kernel,
but I guess it is not easy to test...)
I am adding cc to dm-devel list, maybe someone there could have some idea
how to help here.
Milan
> <tel:%28207%29%20561-3574> Orono ME 04469
>
> Thanks,
>
> Steve
>
>
>
> -- ________________________________________________________________
> Steve Cousins Supercomputer Engineer/Administrator
> Advanced Computing Group University of Maine System 244
> Neville Hall (UMS Data Center) (207) 561-3574
> steve.cousins at maine.edu <http://maine.edu>
________________________________________________________________
Steve Cousins Supercomputer Engineer/Administrator
Advanced Computing Group University of Maine System
244 Neville Hall (UMS Data Center) (207) 561-3574
Orono ME 04469 steve.cousins at maine.edu
-- dm-devel mailing list dm-devel@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/dm-devel
