On 2020/8/19 下午1:58, Parav Pandit wrote:
From: Yan Zhao <yan.y.zhao@xxxxxxxxx>
Sent: Wednesday, August 19, 2020 9:01 AM
On Tue, Aug 18, 2020 at 09:39:24AM +0000, Parav Pandit wrote:
Please refer to my previous email which has more example and details.
hi Parav,
the example is based on a new vdpa tool running over netlink, not based on
devlink, right?
Right.
For vfio migration compatibility, we have to deal with both mdev and physical
pci devices, I don't think it's a good idea to write a new tool for it, given we are
able to retrieve the same info from sysfs and there's already an mdevctl from
mdev attribute should be visible in the mdev's sysfs tree.
I do not propose to write a new mdev tool over netlink. I am sorry if I implied that with my suggestion of vdpa tool.
If underlying device is vdpa, mdev might be able to understand vdpa device and query from it and populate in mdev sysfs tree.
Note that vdpa is bus independent so it can't work now and the support
of mdev on top of vDPA have been rejected (and duplicated with vhost-vDPA).
Thanks
The vdpa tool I propose is usable even without mdevs.
vdpa tool's role is to create one or more vdpa devices and place on the "vdpa" bus which is the lowest layer here.
Additionally this tool let user query virtqueue stats, db stats.
When a user creates vdpa net device, user may need to configure features of the vdpa device such as VIRTIO_NET_F_MAC, default VIRTIO_NET_F_MTU.
These are vdpa level features, attributes. Mdev is layer above it.
Alex
(https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.
com%2Fmdevctl%2Fmdevctl&data=02%7C01%7Cparav%40nvidia.com%7C
0c2691d430304f5ea11308d843f2d84e%7C43083d15727340c1b7db39efd9ccc17
a%7C0%7C0%7C637334057571911357&sdata=KxH7PwxmKyy9JODut8BWr
LQyOBylW00%2Fyzc4rEvjUvA%3D&reserved=0).
Sorry for above link mangling. Our mail server is still transitioning due to company acquisition.
I am less familiar on below points to comment.
hi All,
could we decide that sysfs is the interface that every VFIO vendor driver needs
to provide in order to support vfio live migration, otherwise the userspace
management tool would not list the device into the compatible list?
if that's true, let's move to the standardizing of the sysfs interface.
(1) content
common part: (must)
- software_version: (in major.minor.bugfix scheme)
- device_api: vfio-pci or vfio-ccw ...
- type: mdev type for mdev device or
a signature for physical device which is a counterpart for
mdev type.
device api specific part: (must)
- pci id: pci id of mdev parent device or pci id of physical pci
device (device_api is vfio-pci)
- subchannel_type (device_api is vfio-ccw)
vendor driver specific part: (optional)
- aggregator
- chpid_type
- remote_url
NOTE: vendors are free to add attributes in this part with a restriction that this
attribute is able to be configured with the same name in sysfs too. e.g.
for aggregator, there must be a sysfs attribute in device node
/sys/devices/pci0000:00/0000:00:02.0/882cc4da-dede-11e7-9180-
078a62063ab1/intel_vgpu/aggregator,
so that the userspace tool is able to configure the target device according to
source device's aggregator attribute.
(2) where and structure
proposal 1:
|- [path to device]
|--- migration
| |--- self
| | |-software_version
| | |-device_api
| | |-type
| | |-[pci_id or subchannel_type]
| | |-<aggregator or chpid_type>
| |--- compatible
| | |-software_version
| | |-device_api
| | |-type
| | |-[pci_id or subchannel_type]
| | |-<aggregator or chpid_type>
multiple compatible is allowed.
attributes should be ASCII text files, preferably with only one value per file.
proposal 2: use bin_attribute.
|- [path to device]
|--- migration
| |--- self
| |--- compatible
so we can continue use multiline format. e.g.
cat compatible
software_version=0.1.0
device_api=vfio_pci
type=i915-GVTg_V5_{val1:int:1,2,4,8}
pci_id=80865963
aggregator={val1}/2
Thanks
Yan
