On Fri, 17 Jun 2011 12:20:08 -0500
Shirish Pargaonkar <shirishpargaonkar@xxxxxxxxx> wrote:
> On Fri, Jun 17, 2011 at 10:58 AM, Jeff Layton <jlayton@xxxxxxxxxx> wrote:
> > On Fri, 17 Jun 2011 10:40:21 -0500
> > Shirish Pargaonkar <shirishpargaonkar@xxxxxxxxx> wrote:
> >
> >> On Fri, Jun 17, 2011 at 9:13 AM, Jeff Layton <jlayton@xxxxxxxxxx> wrote:
> >> > On Fri, 17 Jun 2011 08:58:40 -0500
> >> > Shirish Pargaonkar <shirishpargaonkar@xxxxxxxxx> wrote:
> >> >
> >> >> On Fri, Jun 17, 2011 at 8:06 AM, Jeff Layton <jlayton@xxxxxxxxxx> wrote:
> >> >> > Hi Shirish,
> >> >> >
> >> >> > I've been working on some backports of some upstream patch series and
> >> >> > have run into what I think is a problem with the new crypto code. The
> >> >> > problem mainly seems to manifest itself as bad signatures in write
> >> >> > calls. This causes a win2k8 server (at least) to reject the call with
> >> >> > STATUS_ACCESS_DENIED and stop responding to other calls on the socket.
> >> >> >
> >> >> > I did a bisect of sorts, and got to this patch:
> >> >> >
> >> >> > commit ca83ce3d5b9ad321ee24f5870a77f0b21ac5a5de
> >> >> > Author: Jeff Layton <jlayton@xxxxxxxxxx>
> >> >> > Date: Tue Apr 12 09:13:44 2011 -0400
> >> >> >
> >> >> > cifs: don't allow mmap'ed pages to be dirtied while under writeback (try #3)
> >> >> >
> >> >> > My original thought was that something was altering these pages while
> >> >> > they were under writeback, but I did some instrumentation and found
> >> >> > that not to be the case. The signature is the same before and after
> >> >> > the send when this occurs. A key change in this patch is that when
> >> >> > signing is enabled, the code started using CIFSSMBWrite2(), which
> >> >> > marshals up the send buffer in an array of kvecs.
> >> >> >
> >> >> > That leads me to believe that the cifs_sign_smb2 codepath is busted.
> >> >> >
> >> >> > I'll see if I can come up with a testcase, but I'm not that familiar
> >> >> > with the kernel crypto code. Is this something you've seen in your
> >> >> > testing? Any immediate thoughts as to where the problem may be?
> >> >> >
> >> >> > --
> >> >> > Jeff Layton <jlayton@xxxxxxxxxx>
> >> >> >
> >> >>
> >> >
> >> > (fixing cc list since I goofed it earlier...)
> >> >
> >> >> Jeff, no I have not seen this. You think some iozone testing against
> >> >> a Windows server with the latest cifs code might expose this problem?
> >> >> I will try both Windows 2003 server and Windows 2008 server.
> >> >>
> >> >
> >> > I'm using fsstress against win2k8, and it seems to fail on the initial
> >> > write calls.
> >> >
> >> >> cifs_sign_smb and cifs_sign_smb2 do the same exact thing except that
> >> >> the messages that gets used in signature calculation are different in these
> >> >> routines.
> >> >>
> >> >> My initial thought was/is the same as yours, the content of message
> >> >> used in calculating signature is different at the server and client resulting
> >> >> in different signatures hence dropped smb connection by the server.
> >> >> But it is possible cifs_sign_smb2 and/or cifs_calc_signature2 have a bug!
> >> >>
> >> >
> >> > I did a test where I recalculated the signature after calling smb_sendv
> >> > and then compared it to the original signature and they matched, but
> >> > the server rejected it. I'm still trying to nail down the problem, so
> >> > the bug could be anywhere really.
> >> >
> >> > --
> >> > Jeff Layton <jlayton@xxxxxxxxxx>
> >> >
> >>
> >> Strange, iozone just hangs when I issue a command against a file on
> >> a Windows 2003 server
> >>
> >> Run began: Fri Jun 17 10:34:13 2011
> >>
> >> Auto Mode
> >> Command line used:
> >> /usr/src/iozone/iozone3_323/src/current/iozone -a -f /mnt/smb_a/nfile1
> >> Output is in Kbytes/sec
> >> Time Resolution = 0.000001 seconds.
> >> Processor cache size set to 1024 Kbytes.
> >> Processor cache line size set to 32 bytes.
> >> File stride size set to 17 * record size.
> >> random
> >> random bkwd record stride
> >> KB reclen write rewrite read reread read
> >> write read rewrite read fwrite frewrite fread freread
> >> 64 4
> >>
> >> The last packets 9n wireshark trace on the server is
> >>
> >> write andx response: status access denied
> >> locking andx request fid: xyz
> >>
> >> and then cifs client reconnects.
> >>
> >> (may be this is the dropping/locking smb connection because signature
> >> in the write andx request is invalid).
> >
> > Right, that's basically what I see. win2k8 just replies with status
> > access denied. I suspect that that's because the signature is incorrect.
> >
> > I've noticed too that it only happens with larger writes. Right now,
> > I'm trying to figure out the size where it breaks.
> >
> > --
> > Jeff Layton <jlayton@xxxxxxxxxx>
> >
>
> Jeff, no problems with the dd commands
>
> # ls -l /mnt/smb_a/largefle
> -rw-r--r-- 1 root root 16641 Jun 17 12:22 /mnt/smb_a/largefile
>
> # dd if=/tmp/largefile of/mnt/smb_a/largefile bs=16641 count=100
> 100+0 records in
> 100+0 records out
> 1664100 bytes (1.7 MB) copied, 0.0350296 s, 47.5 MB/s
>
> # ls -l /mnt/smb_a/largefle
> -rw-r--r-- 1 root root 1664100 Jun 17 12:22 /mnt/smb_a/largefile
>
> # dd if=/tmp/largefile of=/mnt/smb_a/largefile bs=16641 count=1000
> 1000+0 records in
> 1000+0 records out
> 16641000 bytes (17 MB) copied, 0.241208 s, 69.0 MB/s
>
> # ls -l /mnt/smb_a/largefile
> -rw-r--r-- 1 root root 16641000 Jun 17 12:23 /mnt/smb_a/largefile
Interesting, I've been able to consistently get this to fail with:
# dd if=/dev/urandom of=/mnt/win2k8/file bs=16641 count=1
...but if I write 16640 bytes, it works.
In any case, I've opened a bug at samba.org:
https://bugzilla.samba.org/show_bug.cgi?id=8245
...so we can track this problem. At this point, I'm still trying to
nail down what the problem actually is.
--
Jeff Layton <jlayton@xxxxxxxxxx>
--
To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
