Hi Tom,
Steve suggested I should ask you about this.
I have IWarp RDMA mostly working with my iteratorisation patches - certainly
better than without them, but I think that's mostly due to the patch that
Stefan Metzmacher so dislikes ("cifs: Fix problem with encrypted RDMA data
read").
However, fallocate doesn't work:
# rdma link add siw0 type siw netdev enp6s0 # andromeda, softIWarp
# mount //192.168.6.1/test /xfstest.test -o user=shares,pass=foobar,rdma
# fallocate -l 1M /xfstest.test/hello
fallocate: fallocate failed: Resource temporarily unavailable
Because smb3_simple_fallocate_write_range() calls SMB2_write(), which calls
cifs_send_recv() then compound_send_recv() and thence to smb_send_rqst().
smb_send_rqst() encrypts the buffer it is given and smbd_send() attempts to
shovel it to the server using Direct Data Placement - which I think might fail
because the data is encrypted.
In one run of the above commands, the data in the kvec array looked like:
fe534d42400001000000000009000a0000000000000000001600000000000000a01300000200
0000000000000000000000000000000000000000000000000000000000000000000000000000
before the smb_send_rqst() gets to ->init_transform_rq() and like:
98eddc1bc31da7c55c00341e4dc769fa4c8b2b0ecdacbad33eb31855ec162fa2458b8437edc7
88ee0a033c84aa857b65ab31ce553594d412719cc3daf925e873e80062ec16b97c855721a42d
after. The encrypted data is seen on the wire in DDP/RDMA packets.
Any thoughts as to how to fix this?
Does it need to pass a flag down to suppress the encryption or suppress the
use of direct data placement? Or should it perhaps go through something like
->write_iter()?
Note also that it encrypts the buffer in place and then
smb3_simple_fallocate_write_range() reuses the buffer multiple times without
clearing it.
I've pushed my cifs iteratorisation patches to:
https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=iov-cifs
I can post them by email a bit later.
David
