On 2019-12-30 02:29, Jack Wang wrote:
> + * InfiniBand Transport Layer
Is RTRS an InfiniBand or an RDMA transport layer?
> +#define rtrs_prefix(obj) (obj->sessname)
Is it really worth it to introduce a macro for accessing a single member
of a single pointer?
> + * InfiniBand Transport Layer
Same question here: is RTRS an InfiniBand or an RDMA transport layer?
> +enum {
> + SERVICE_CON_QUEUE_DEPTH = 512,
What is a service connection?
> + /*
> + * With the current size of the tag allocated on the client, 4K
> + * is the maximum number of tags we can allocate. This number is
> + * also used on the client to allocate the IU for the user connection
> + * to receive the RDMA addresses from the server.
> + */
What does the word 'tag' mean in the context of the RTRS protocol?
> +struct rtrs_ib_dev;
What does the "rtrs_ib_dev" data structure represent? Additionally, I
think it's confusing that a single name has an "r" that refers to "RDMA"
and "ib" that refers to InfiniBand.
> +struct rtrs_ib_dev_pool {
> + struct mutex mutex;
> + struct list_head list;
> + enum ib_pd_flags pd_flags;
> + const struct rtrs_ib_dev_pool_ops *ops;
> +};
What is the purpose of an rtrs_ib_dev_pool and what does it contain?
> +struct rtrs_iu {
A comment that explains what the "iu" abbreviation stands for would be
welcome.
> +/**
> + * enum rtrs_msg_types - RTRS message types.
> + * @RTRS_MSG_INFO_REQ: Client additional info request to the server
> + * @RTRS_MSG_INFO_RSP: Server additional info response to the client
> + * @RTRS_MSG_WRITE: Client writes data per RDMA to server
> + * @RTRS_MSG_READ: Client requests data transfer from server
> + * @RTRS_MSG_RKEY_RSP: Server refreshed rkey for rbuf
> + */
What is "additional info" in this context?
> +/**
> + * struct rtrs_msg_conn_req - Client connection request to the server
> + * @magic: RTRS magic
> + * @version: RTRS protocol version
> + * @cid: Current connection id
> + * @cid_num: Number of connections per session
> + * @recon_cnt: Reconnections counter
> + * @sess_uuid: UUID of a session (path)
> + * @paths_uuid: UUID of a group of sessions (paths)
> + *
> + * NOTE: max size 56 bytes, see man rdma_connect().
> + */
> +struct rtrs_msg_conn_req {
> + u8 __cma_version; /* Is set to 0 by cma.c in case of
> + * AF_IB, do not touch that.
> + */
> + u8 __ip_version; /* On sender side that should be
> + * set to 0, or cma_save_ip_info()
> + * extract garbage and will fail.
> + */
The above two fields and the comments next to it look suspicious to me.
Does RTRS perhaps try to generate CMA-formatted messages without using
the CMA to format these messages?
> + u8 reserved[12];
Please leave out the reserved data. If future versions of the protocol
would need any of these bytes it is easy to add more data to this structure.
> +/**
> + * struct rtrs_msg_conn_rsp - Server connection response to the client
> + * @magic: RTRS magic
> + * @version: RTRS protocol version
> + * @errno: If rdma_accept() then 0, if rdma_reject() indicates error
> + * @queue_depth: max inflight messages (queue-depth) in this session
> + * @max_io_size: max io size server supports
> + * @max_hdr_size: max msg header size server supports
> + *
> + * NOTE: size is 56 bytes, max possible is 136 bytes, see man rdma_accept().
> + */
> +struct rtrs_msg_conn_rsp {
> + __le16 magic;
> + __le16 version;
> + __le16 errno;
> + __le16 queue_depth;
> + __le32 max_io_size;
> + __le32 max_hdr_size;
> + __le32 flags;
> + u8 reserved[36];
> +};
Same comment here: please leave out the "reserved[]" array. Sending a
bunch of zero-bytes at the end of a message over the wire is not useful.
> +static inline void rtrs_from_imm(u32 imm, u32 *type, u32 *payload)
> +{
> + *payload = (imm & MAX_IMM_PAYL_MASK);
> + *type = (imm >> MAX_IMM_PAYL_BITS);
> +}
Please do not use parentheses when not necessary. Such superfluous
parentheses namely hurt readability of the code.
> + type = (w_inval ? RTRS_IO_RSP_W_INV_IMM : RTRS_IO_RSP_IMM);
Same comment here: I think the parentheses can be left out from the
above statement.
> +static inline void rtrs_from_io_rsp_imm(u32 payload, u32 *msg_id, int *errno)
> +{
> + /* 9 bits for errno, 19 bits for msg_id */
> + *msg_id = (payload & 0x7ffff);
Are the parentheses in the above expression necessary?
> + *errno = -(int)((payload >> 19) & 0x1ff);
Is the '(int)' cast useful in the above expression? Can it be left out?
> +#define STAT_ATTR(type, stat, print, reset) \
> +STAT_STORE_FUNC(type, stat, reset) \
> +STAT_SHOW_FUNC(type, stat, print) \
> +static struct kobj_attribute stat##_attr = \
> + __ATTR(stat, 0644, \
> + stat##_show, \
> + stat##_store)
Is the above use of __ATTR() perhaps an open-coded version of __ATTR_RW()?
Thanks,
Bart.
