Re: [vfio-users] Fwd: Proper way for systemd service to wait mdev gvt device initialization

[Erik Skultety <eskultet@xxxxxxxxxx>]

On Sun, May 26, 2019 at 09:28:36PM +0300, Alex Ivanov wrote:
> Could Intel fix that?

This is not tied only to the Intel driver. Like Alex said, mdev is not a
fundamental feature of the parent device. There was an attempt to fix this
behaviour in the mdev core driver last year [1], but was rejected upstream with
a recommendation that all applications should handle this on their own, so we
ended up introducing a wait in libvirt too. Personally, it has always felt
weird to me that fixing 1 thing at N places could be the right approach, but I
hope we'll get the change event at some point.

[1] https://lkml.org/lkml/2018/2/9/164

Regards,
Erik

>
> -------- Пересылаемое сообщение --------
> 20.05.2019, 15:18, "Andrei Borzenkov" <arvidjaar@xxxxxxxxx>:
>
> On Mon, May 20, 2019 at 10:08 AM Mantas Mikulėnas <grawity@xxxxxxxxx> wrote:
> >  On Sun, May 19, 2019 at 9:50 PM Alex Ivanov <gnidorah@xxxxx> wrote:
> >>  Hello.
> >>  What is the proper way to do that? I have a unit that creates gvt device in the system
> >>
> >>  ExecStart = "sh -c 'echo a297db4a-f4c2-11e6-90f6-d3b88d6c9525 > /sys/bus/pci/devices/0000:00:02.0/mdev_supported_types/i915-GVTg_V5_8/create'";
> >>  ExecStop = "sh -c 'echo 1 > /sys/bus/pci/devices/0000:00:02.0/a297db4a-f4c2-11e6-90f6-d3b88d6c9525/remove'";
> >
> >  Personally, I would use an udev rule:
> >
> >  ACTION=="add", SUBSYSTEM=="pci", ENV{PCI_SLOT_NAME}=="0000:00:02.0", ATTR{mdev_supported_types/i915-GVTg_V5_8/create}="a297db4a-f4c2-11e6-90f6-d3b88d6c9525"
>
> There is a race condition here, driver creates
> .../mdev_supported_types after it has registered device so udev may
> process event before directory is available.
>
> >  Though on the other hand, a service is a good choice if you want to `systemctl stop` it later on.
> >
> >  ACTION=="add", SUBSYSTEM=="pci", ENV{PCI_SLOT_NAME}=="0000:00:02.0", ENV{SYSTEMD_WANTS}+="create-gvt.service"
> >
> >>  Ideally I would to like to start this service when 0000:00:02.0 device appears in the system, but the problem is that /sys/bus/pci/devices/0000:00:02.0/mdev_supported_types/ tree is populated later, so my service will fail.
> >>
> >>  So the question what is the proper way to fix that.
> >
> >  If the driver doesn't populate its sysfs entries in time, maybe it at least generates 'change' uevents? (udevadm --monitor)
>
> I would tentatively say this is driver bug. This directory is created
> during initial device setup, not in response to some event later. From
> https://github.com/torvalds/linux/blob/master/Documentation/driver-model/device.txt:
>
> --><--
> As explained in Documentation/kobject.txt, device attributes must be
> created before the KOBJ_ADD uevent is generated.
> --><--
>
> Note that some drivers even disable KOBJ_ADD notification during
> device_register() and trigger it manually later, after sysfs layout is
> complete. I cannot evaluate whether this directory can be created and
> populated before device_register().
>
> >  If there are no uevents either, well, there's nothing you can do from systemd's side. (Other than making a script that loops repeatedly checking "is it there yet? is it there yet?")
>
> Should really be fixed on kernel side.
> -------- Конец пересылаемого сообщения --------
>
> _______________________________________________
> vfio-users mailing list
> vfio-users@xxxxxxxxxx
> https://www.redhat.com/mailman/listinfo/vfio-users
_______________________________________________
systemd-devel mailing list
systemd-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.freedesktop.org/mailman/listinfo/systemd-devel