> -----Original Message----- > From: Jason Gunthorpe <jgg@xxxxxxxx> > Sent: Wednesday, March 18, 2020 12:35 PM > To: Wan, Kaike <kaike.wan@xxxxxxxxx> > Cc: Dalessandro, Dennis <dennis.dalessandro@xxxxxxxxx>; > dledford@xxxxxxxxxx; linux-rdma@xxxxxxxxxxxxxxx; Marciniszyn, Mike > <mike.marciniszyn@xxxxxxxxx> > Subject: Re: [PATCH for-next 3/3] IB/hfi1: Use the ibdev in hfi1_devdata as > the parent of cdev > > > > > > > > > The hfi1 driver dynammically allocates a struct device to > > > > represent the cdev in sysfs and devtmpfs (/dev/hfi1_x). On the > > > > other hand, the hfi1_devdata already contains a struct device in > > > > its ibdev field (hfi1_devdata.verbs_dev.rdi.ibdev.dev), and it is > > > > therefore possible to eliminate the dynamical allocation when > > > > creating the cdev. Since each device could be added to the sysfs > > > > only once and the function > > > > device_add() is already called for the ibdev in > > > > ib_register_device(), the function cdev_device_add() could not be > > > > used to create the cdev, even though the hfi1_devdata contains > > > > both cdev and ibdev in the same structure. > > > > > > > > This patch eliminates the dynamic allocation by creating the cdev > > > > first, setting up the ibdev, and then calling the > > > > ib_register_device() to add the device to sysfs and devtmpfs. > > > > > > What do the sysfs paths for the cdev look like now? > > > > ls -l /sys/dev/char/243:0 > > lrwxrwxrwx 1 root root 0 Mar 15 14:30 /sys/dev/char/243:0 -> > > ../../devices/pci0000:00/0000:00:02.0/0000:02:00.0/infiniband/hfi1_0 > > > > It points back to the IB device (hfi1_0 ). > > > > Before this change, it pointed back to a virtual device: > > > > ls /sys/dev/char/243:0 -l > > lrwxrwxrwx 1 root root 0 Mar 18 11:52 /sys/dev/char/243:0 -> > > ../../devices/virtual/hfi1_user/hfi1_0 > > Great, yes this looks right to me > > So this came up due to PSM having problems.. The right way for PSM to work > is now to find the hfi1_0 devices under /sys/class/hfi1_user/* and then map > then back to RDMA devices and the physical card by doing realpath and > learning the '/sys/pci0000:00/0000:00:02.0/0000:02:00.0/infiniband/XXX/' > path > > Yes? That is certainly one way to get the device info. Kaike
