On 9/25/2022 9:13 PM, Namjae Jeon wrote:
2022-09-26 0:41 GMT+09:00, Tom Talpey <tom@xxxxxxxxxx>:
On 9/24/2022 11:40 PM, Namjae Jeon wrote:
2022-09-24 6:53 GMT+09:00, Tom Talpey <tom@xxxxxxxxxx>:
Reduce ksmbd smbdirect max segment send and receive size to 1364
to match protocol norms. Larger buffers are unnecessary and add
significant memory overhead.
Signed-off-by: Tom Talpey <tom@xxxxxxxxxx>
---
fs/ksmbd/transport_rdma.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/fs/ksmbd/transport_rdma.c b/fs/ksmbd/transport_rdma.c
index 494b8e5af4b3..0315bca3d53b 100644
--- a/fs/ksmbd/transport_rdma.c
+++ b/fs/ksmbd/transport_rdma.c
@@ -62,13 +62,13 @@ static int smb_direct_receive_credit_max = 255;
static int smb_direct_send_credit_target = 255;
/* The maximum single message size can be sent to remote peer */
-static int smb_direct_max_send_size = 8192;
+static int smb_direct_max_send_size = 1364;
/* The maximum fragmented upper-layer payload receive size supported
*/
static int smb_direct_max_fragmented_recv_size = 1024 * 1024;
/* The maximum single-message size which can be received */
-static int smb_direct_max_receive_size = 8192;
+static int smb_direct_max_receive_size = 1364;
Can I know what value windows server set to ?
I can see the following settings for them in MS-SMBD.pdf
Connection.MaxSendSize is set to 1364.
Connection.MaxReceiveSize is set to 8192.
Glad you asked, it's an interesting situation IMO.
In MS-SMBD, the following are documented as behavior notes:
Client-side (active connect):
Connection.MaxSendSize is set to 1364.
Connection.MaxReceiveSize is set to 8192.
Server-side (passive listen):
Connection.MaxSendSize is set to 1364.
Connection.MaxReceiveSize is set to 8192.
However, these are only the initial values. During SMBD
negotiation, the two sides adjust downward to the other's
maximum. Therefore, Windows connecting to Windows will use
1364 on both sides.
In cifs and ksmbd, the choices were messier:
Client-side smbdirect.c:
int smbd_max_send_size = 1364;
int smbd_max_receive_size = 8192;
Server-side transport_rdma.c:
static int smb_direct_max_send_size = 8192;
static int smb_direct_max_receive_size = 8192;
Therefore, peers connecting to ksmbd would typically end up
negotiating 1364 for send and 8192 for receive.
There is almost no good reason to use larger buffers. Because
RDMA is highly efficient, and the smbdirect protocol trivially
fragments longer messages, there is no significant performance
penalty.
And, because not many SMB3 messages require 8192 bytes over
smbdirect, it's a colossal waste of virtually contiguous kernel
memory to allocate 8192 to all receives.
By setting all four to the practical reality of 1364, it's a
consistent and efficient default, and aligns Linux smbdirect
with Windows.
Thanks for your detailed explanation! Agree to set both to 1364 by
default, Is there any usage to increase it? I wonder if users need any
configuration parameters to adjust them.
In my opinion, probably not. I give some reasons why large fragments
aren't always helpful just above. It's the same number of packets! Just
a question of whether SMBDirect or Ethernet does the fragmentation, and
the buffer management.
There might conceivably be a case for *smaller*, for example on IB when
it's cranked down to the minimum (256B) MTU. But it will work with this
default.
I'd say let's don't over-engineer it until we address the many other
issues in this code. Merging the two smbdirect implementations is much
more important than adding tweaky little knobs to both. MHO.
Tom.
Why does the specification describe setting it to 8192?
static int smb_direct_max_read_write_size = SMBD_DEFAULT_IOSIZE;
--
2.34.1
