On Tue, 11 Jun 2019 10:19:29 -0400
Tony Krowiak <akrowiak@xxxxxxxxxxxxx> wrote:
> On 6/7/19 4:03 PM, Alex Williamson wrote:
> > On Fri, 7 Jun 2019 14:26:13 -0400
> > Tony Krowiak <akrowiak@xxxxxxxxxxxxx> wrote:
> >
> >> On 6/6/19 12:15 PM, Alex Williamson wrote:
> >>> On Thu, 6 Jun 2019 09:32:24 -0600
> >>> Alex Williamson <alex.williamson@xxxxxxxxxx> wrote:
> >>>
> >>>> On Thu, 6 Jun 2019 16:44:17 +0200
> >>>> Cornelia Huck <cohuck@xxxxxxxxxx> wrote:
> >>>>
> >>>>> Add a rough implementation for vfio-ap.
> >>>>>
> >>>>> Signed-off-by: Cornelia Huck <cohuck@xxxxxxxxxx>
> >>>>> ---
> >>>>> mdevctl.libexec | 25 ++++++++++++++++++++++
> >>>>> mdevctl.sbin | 56 ++++++++++++++++++++++++++++++++++++++++++++++++-
> >>>>> 2 files changed, 80 insertions(+), 1 deletion(-)
> >>>>>
> >>>>> diff --git a/mdevctl.libexec b/mdevctl.libexec
> >>>>> index 804166b5086d..cc0546142924 100755
> >>>>> --- a/mdevctl.libexec
> >>>>> +++ b/mdevctl.libexec
> >>>>> @@ -54,6 +54,19 @@ wait_for_supported_types () {
> >>>>> fi
> >>>>> }
> >>>>>
> >>>>> +# configure vfio-ap devices <config entry> <matrix attribute>
> >>>>> +configure_ap_devices() {
> >>>>> + list="`echo "${config[$1]}" | sed 's/,/ /'`"
> >>>>> + [ -z "$list" ] && return
> >>>>> + for a in $list; do
> >>>>> + echo "$a" > "$supported_types/${config[mdev_type]}/devices/$uuid/$2"
> >>>>> + if [ $? -ne 0 ]; then
> >>>>> + echo "Error writing '$a' to '$uuid/$2'" >&2
> >>>>> + exit 1
> >>>>> + fi
> >>>>> + done
> >>>>> +}
> >>>>> +
> >>>>> case ${1} in
> >>>>> start-mdev|stop-mdev)
> >>>>> if [ $# -ne 2 ]; then
> >>>>> @@ -148,6 +161,18 @@ case ${cmd} in
> >>>>> echo "Error creating mdev type ${config[mdev_type]} on $parent" >&2
> >>>>> exit 1
> >>>>> fi
> >>>>> +
> >>>>> + # some types may specify additional config data
> >>>>> + case ${config[mdev_type]} in
> >>>>> + vfio_ap-passthrough)
> >>>>
> >>>> I think this could have some application beyond ap too, I know NVIDIA
> >>>> GRID vGPUs do have some controls under the vendor hierarchy of the
> >>>> device, ex. setting the frame rate limiter. The implementation here is
> >>>> a bit rigid, we know a specific protocol for a specific mdev type, but
> >>>> for supporting arbitrary vendor options we'd really just want to try to
> >>>> apply whatever options are provided. If we didn't care about ordering,
> >>>> we could just look for keys for every file in the device's immediate
> >>>> sysfs hierarchy and apply any value we find, independent of the
> >>>> mdev_type, ex. intel_vgpu/foo=bar Thanks,
> >>>
> >>> For example:
> >>>
> >>> for key in find -P $mdev_base/$uuid/ \( -path
> >>> "$mdev_base/$uuid/power/*" -o -path $mdev_base/$uuid/uevent -o -path $mdev_base/$uuid/remove \) -prune -o -type f -print | sed -e "s|$mdev_base/$uuid/||g"); do
> >>> [ -z ${config[$key]} ] && continue
> >>> ... parse value(s) and iteratively apply to key
> >>> done
> >>>
> >>> The find is a little ugly to exclude stuff, maybe we just let people do
> >>> screwy stuff like specify remove=1 in their config. Also need to think
> >>> about whether we're imposing a delimiter to apply multiple values to a
> >>> key that conflicts with the attribute usage. Thanks,
> >>>
> >>> Alex
One thing that this does is limiting us to things that can be expressed
with "if you encounter key=value, take value (possibly decomposed) and
write it to <device>/key". A problem with this generic approach is that
the code cannot decide itself whether value should be decomposed (and
if yes, with which delimiter), or not. We also cannot cover any
configuration that does not fit this pattern; so I think we need both
generic (for flexibility, and easy extensibility), and explicitly
defined options to cover more complex cases.
[As an aside, how should we deal with duplicate key= entries? Not
allowed, last one wins, or all are written to the sysfs attribute?]
> >>
> >> I like the idea of looking for files in the device's immediate sysfs
> >> hierarchy, but maybe the find could exclude attributes that are
> >> not vendor defined.
> >
> > How would we know what attributes are vendor defined? The above `find`
> > strips out the power, uevent, and remove attributes, which for GVT-g
> > leaves only the vendor defined attributes[1], but I don't know how to
> > instead do a positive match of the vendor attributes without
> > unmaintainable lookup tables. This starts to get into the question of
> > how much do we want to (or need to) protect the user from themselves.
> > If we let the user specify a key=value of remove=1 and the device
> > immediately disappears, is that a bug or a feature? Thanks,
> >
> > Alex
>
> By vendor defined, I meant attributes that are not defined by the mdev
> framework, such as the 'remove' attribute.
And those defined by the base driver core like uevent, I guess.
> As far as whether allowing
> specification of remove-1, I'd have to play with that and see what all
> of the ramifications are.
It does feel a bit odd to me (why would you configure it if you
immediately want to remove it again?)
>
> Tony K
>
> >
> > [1] GVT-g doesn't actually have an writable attributes, so we'd also
> > minimally want to add a test to skip read-only attributes.
>
> Probably a good idea.
Agreed.
