On Tue, Jan 05, 2021 at 03:15:58AM +0000, liweihang wrote: > On 2020/12/31 18:39, Leon Romanovsky wrote: > > On Thu, Dec 31, 2020 at 09:36:27AM +0000, liweihang wrote: > >> On 2020/12/31 15:59, Leon Romanovsky wrote: > >>> On Tue, Dec 29, 2020 at 01:31:39PM +0000, liweihang wrote: > >>>> Hi all, > >>>> > >>>> We want to dump some hns driver-defined information that belongs to a > >>>> process to keep track of current memory usage. For example, there is > >>>> a ucontext-level(process-level) memory pool to store WQE which is > >>>> shared by a lot of QPs, we want to record and query which QPs are using > >>>> this pool and how much space each QP is using. > >>>> > >>>> rdmatool don't have a ucontext-level resource tracking currently, is it > >>>> ok to achieve that through debugfs? > >>>> > >>>> This may looks like: > >>>> > >>>> $ echo 1 > <dbgfs_dir>/hns_roce/hns_0/<pid>/qp > >>>> QPN Total(kB) SQ(kB) SGE(kB) RQ(kB) > >>>> 110 6400 256 2048 4096 > >>>> 118 6400 256 2048 0 > >>>> > >>>> Or should it be achieved in rdmatool? > >>> > >>> I think so, because PID != ucontext. Why can't it be presented as QP > >>> attribute? Can you please send "rdmatool" example? > >>> > >>> Thanks > >> > >> Hi Leon, > >> > >> Thanks for your response. If we can achieve it in rdmatool, it may > >> looks like: > >> > >> 1) We want to get some information of a ucontext (assuming that each > >> ucontext has an ID), for example, the size of a memory pool that belongs > >> to a ucontext as I mentioned above: > >> > >> $ rdma res show ucontext > >> uctx_id 1 pid 20 qp_buf_sz 6400 sq_buf_sz 256 sge_buf_sz 2048 rq_buf_sz 4096 > >> uctx_id 2 pid 20 qp_buf_sz 4800 sq_buf_sz 128 sge_buf_sz 2048 rq_buf_sz 2048 > > > > I have no problems to add "rdma res show ucontext" command, we just need > > to find what should be printed. > > > >> > >> 2) We want to know which ucontext a QP belongs to: > >> > >> $ rdma res show qp > >> link hns_0/1 lqpn 1 type GSI ... uctx_id 1 > >> link hns_0/1 lqpn 2 type RC ... uctx_id 1 > >> > >> So the question is, we don't have a ucontext-level restrack currently, and > >> there in no 'id' for each ucontext. > > > > We have IDs for every ucontext, it is called "ctxn" and because QP is > > not connected directly to ucontext, but through PDs, it is visible > > when you check PDs. > > > > [leonro@vm ~]$ ibv_rc_pingpong & > > [leonro@vm ~]$ rdma res show > > 0: ibp0s9: pd 4 cq 4 qp 4 cm_id 0 mr 1 ctx 1 > > [leonro@vm ~]$ rdma res show qp type RC > > link ibp0s9/1 lqpn 50 rqpn 0 type RC state INIT rq-psn 16777215 sq-psn 0 path-mig-state MIGRATED pdn 3 pid 479 comm ibv_rc_pingpong > > [leonro@vm ~]$ rdma res show pd pdn 3 > > ifindex 0 ifname ibp0s9 pdn 3 users 2 ctxn 0 pid 479 comm ibv_rc_pingpong > > ^^^^^^ > > > > Thanks > > > > OK, thank you, we will try to use rdmatool to achieve our goal. > > By the way, can we use debugfs to add some trace functions? > Debugfs can quickly get a large amount of information at a time, > it's more convenient than netlink in such situation, especially > for some vendor-defined funtions. If you are talking about connecting your driver to ftrace, so sure go for it. However for the data, please use rdmatool, we have raw option (-r) in the rdmatool that dumps in binary format your specific object. Thanks > > Thanks > Weihang
