On 2/8/23 7:15 AM, Wenjia Zhang wrote:
On 07.02.23 08:36, D. Wythe wrote:
From: "D. Wythe" <alibuda@xxxxxxxxxxxxxxxxx>
We know that all flows except confirm_rkey and delete_rkey are exclusive,
confirm/delete rkey flows can run concurrently (local and remote).
Although the protocol allows, all flows are actually mutually exclusive
in implementation, dues to waiting for LLC messages is in serial.
This aggravates the time for establishing or destroying a SMC-R
connections, connections have to be queued in smc_llc_wait.
We use rtokens or rkey to correlate a confirm/delete rkey message
with its response.
Before sending a message, we put context with rtokens or rkey into
wait queue. When a response message received, we wakeup the context
which with the same rtokens or rkey against the response message.
Signed-off-by: D. Wythe <alibuda@xxxxxxxxxxxxxxxxx>
---
net/smc/smc_llc.c | 174 +++++++++++++++++++++++++++++++++++++++++-------------
net/smc/smc_wr.c | 10 ----
net/smc/smc_wr.h | 10 ++++
3 files changed, 143 insertions(+), 51 deletions(-)
[...]
+static int smc_llc_rkey_response_wake_function(struct wait_queue_entry *wq_entry,
+ unsigned int mode, int sync, void *key)
+{
+ struct smc_llc_qentry *except, *incoming;
+ u8 except_llc_type;
+
+ /* not a rkey response */
+ if (!key)
+ return 0;
+
+ except = wq_entry->private;
+ incoming = key;
+
+ except_llc_type = except->msg.raw.hdr.common.llc_type;
+
+ /* except LLC MSG TYPE mismatch */
+ if (except_llc_type != incoming->msg.raw.hdr.common.llc_type)
+ return 0;
+
+ switch (except_llc_type) {
+ case SMC_LLC_CONFIRM_RKEY:
+ if (memcmp(except->msg.confirm_rkey.rtoken, incoming->msg.confirm_rkey.rtoken,
+ sizeof(struct smc_rmb_rtoken) *
+ except->msg.confirm_rkey.rtoken[0].num_rkeys))
+ return 0;
+ break;
+ case SMC_LLC_DELETE_RKEY:
+ if (memcmp(except->msg.delete_rkey.rkey, incoming->msg.delete_rkey.rkey,
+ sizeof(__be32) * except->msg.delete_rkey.num_rkeys))
+ return 0;
+ break;
+ default:
+ pr_warn("smc: invalid except llc msg %d.\n", except_llc_type);
+ return 0;
+ }
+
+ /* match, save hdr */
+ memcpy(&except->msg.raw.hdr, &incoming->msg.raw.hdr, sizeof(except->msg.raw.hdr));
+
+ wq_entry->private = except->private;
+ return woken_wake_function(wq_entry, mode, sync, NULL);
+}
+
s/except/expect/ ?
Just kind of confusing
[...]
Hi, Wenjia
Except is what I want to express.
It means that only the confirm and delete rkey can be processed in parallel. :-)
Thanks
D. Wythe
