On Thu, Mar 2, 2017 at 6:08 PM, Hefty, Sean <sean.hefty@xxxxxxxxx> wrote: >> Here I suggest to modify the implementation of ibv_get_device_list() so >> that >> consecutive calls will re-scan the sysfs in the same manner as done >> today in >> order to create a fresh ibv_device list each time. We will remove >> caching of >> devices that support plugout, while keeping the ibv_device cache for >> devices >> which do not support plugout. >> >> For this purpose, the ibv_get_device_list() device scanning logic >> should be >> separated from the libibverbs singleton initialization step. >> User can call ibv_open_device() while holding this list (see man pages) >> and >> once ibv_free_device_list() is called libibverbs can release the unused >> ibv_device objects. Later, on calls to ibv_close_device(), additional >> ibv_device object should be released. Currently, on >> ibv_free_device_list(), >> only the array is freed, while the ibv_device objects are never freed. >> libibverbs will maintain a ref_count for each verbs_device object. >> Increase >> verbs_device->ref_count for every ibv_get_device_list() or >> ibv_open_device(). >> Decrease it for every ibv_free_device_list() or ibv_close_device(). >> On decrease, if ref_count tested to be zero, libibverbs will call the >> provider >> library to release the 'strcut verbs_device' which it allocated. >> Each provider library should provide a function to release the >> verbs_device >> >> object: 'uninit_device(struct verbs_device* device)'. > > Based on the discussion so far, is there any disagreement with just the above functionality? Ignoring the event reporting aspect, the above makes sense to me as a reasonable first step. > > - Sean Summarize the basic hotplug functionality part we discussed so far: 1. calling ibv_get_device_list() will rescan sysfs and provide an updated list of ibv_device's 2. optimize by identifying same device incarnation with timestamp creation of /sys/class/infiniband/%ibdev%/hca_type 3. release struct verbs_device according to usage ref_count 4. implement 'uninit_device(struct verbs_device* device)' for all provider libraries to release plugout devices NOTE: consolidating all providers libraries to use the extendible struct verbs_device and move it into driver private area already merged by https://github.com/linux-rdma/rdma-core/pull/87 We'll prepare a PR with the above functionality for all providers. And come back with a separate suggestion for hotplug reporting mechanism. Alex -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
