> On Apr 26, 2016, at 12:45 PM, Steve Wise <swise@xxxxxxxxxxxxxxxxxxxxx> wrote: > > On 4/26/2016 9:57 AM, Chuck Lever wrote: >> >>> On Apr 26, 2016, at 10:13 AM, Steve Wise <swise@xxxxxxxxxxxxxxxxxxxxx> wrote: >>> >>> Hey Chuck, I'm testing this series on cxgb4. I'm running 'iozone -a -+d -I' on a share and watching the server stats. Are the starve numbers expected? >> >> Yes, unless you're seeing much higher numbers than >> you used to. >> >> >>> Every 5.0s: for s in /proc/sys/sunrpc/svc_rdma/rdma_* ; do echo -n "$(basename $s): "; cat $s; done Tue Apr 26 07:10:17 2016 >>> >>> rdma_stat_read: 379872 >>> rdma_stat_recv: 498144 >>> rdma_stat_rq_poll: 0 >>> rdma_stat_rq_prod: 0 >>> rdma_stat_rq_starve: 675564 >> >> This means work was enqueued on the svc_xprt, but by the >> time the upper layer invoked svc_rdma_recvfrom, the work >> was already handled by an earlier wake-up. >> >> I'm not exactly sure why this happens, but it seems to be >> normal (if suboptimal). >> >> >>> rdma_stat_sq_poll: 0 >>> rdma_stat_sq_prod: 0 >>> rdma_stat_sq_starve: 1748000 >> >> No SQ space to post a Send, so the caller is put to sleep. >> >> The server chronically underestimates the SQ depth, especially >> for FRWR. I haven't figured out a better way to estimate it. >> >> But it's generally harmless, as there is a mechanism to put >> callers to sleep until there is space on the SQ. >> > > > Thanks. > > With this iw_cxgb4 drain fix applied: > > [PATCH 3/3] iw_cxgb4: handing draining an idle qp > > http://www.spinics.net/lists/linux-rdma/msg34927.html > > The series tests good over cxgb4. > > Tested-by: Steve Wise <swise@xxxxxxxxxxxxxxxxxxxxx> Excellent. I will mark up my patch series with your Tested-by tag. -- Chuck Lever -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
