Re: [vfio-users] Fwd: Proper way for systemd service to wait mdev gvt device initialization

[Alex Williamson <alex.williamson@xxxxxxxxxx>]

On Sun, 26 May 2019 21:28:36 +0300
Alex Ivanov <gnidorah@xxxxx> wrote:

> Could Intel fix that?

I won't claim that mdev-core is bug free in this area, but it's
probably worth noting that mdev support isn't necessarily a fundamental
feature of the parent device, it could theoretically be enabled much
later than the add uevent for the parent device itself.  We may instead
want to trigger a change uevent when it becomes available.

Also in this space, I've been working on a very preliminary mediated
device management and persistence utility:

https://github.com/awilliam/mdevctl

It works around this issue by simply waiting a short time for mdev
support to appear after the device is added.  This also could be much
more deterministic with a change uevent.  Perhaps it's useful for what
you're trying to achieve though.  Thanks,

Alex

> -------- Пересылаемое сообщение --------
> 20.05.2019, 15:18, "Andrei Borzenkov" <arvidjaar@xxxxxxxxx>:
> 
> On Mon, May 20, 2019 at 10:08 AM Mantas Mikulėnas <grawity@xxxxxxxxx> wrote:
> >  On Sun, May 19, 2019 at 9:50 PM Alex Ivanov <gnidorah@xxxxx> wrote:  
> >>  Hello.
> >>  What is the proper way to do that? I have a unit that creates gvt device in the system
> >>
> >>  ExecStart = "sh -c 'echo a297db4a-f4c2-11e6-90f6-d3b88d6c9525 > /sys/bus/pci/devices/0000:00:02.0/mdev_supported_types/i915-GVTg_V5_8/create'";
> >>  ExecStop = "sh -c 'echo 1 > /sys/bus/pci/devices/0000:00:02.0/a297db4a-f4c2-11e6-90f6-d3b88d6c9525/remove'";  
> >
> >  Personally, I would use an udev rule:
> >
> >  ACTION=="add", SUBSYSTEM=="pci", ENV{PCI_SLOT_NAME}=="0000:00:02.0", ATTR{mdev_supported_types/i915-GVTg_V5_8/create}="a297db4a-f4c2-11e6-90f6-d3b88d6c9525"  
> 
> There is a race condition here, driver creates
> .../mdev_supported_types after it has registered device so udev may
> process event before directory is available.
> 
> >  Though on the other hand, a service is a good choice if you want to `systemctl stop` it later on.
> >
> >  ACTION=="add", SUBSYSTEM=="pci", ENV{PCI_SLOT_NAME}=="0000:00:02.0", ENV{SYSTEMD_WANTS}+="create-gvt.service"
> >  
> >>  Ideally I would to like to start this service when 0000:00:02.0 device appears in the system, but the problem is that /sys/bus/pci/devices/0000:00:02.0/mdev_supported_types/ tree is populated later, so my service will fail.
> >>
> >>  So the question what is the proper way to fix that.  
> >
> >  If the driver doesn't populate its sysfs entries in time, maybe it at least generates 'change' uevents? (udevadm --monitor)  
> 
> I would tentatively say this is driver bug. This directory is created
> during initial device setup, not in response to some event later. From
> https://github.com/torvalds/linux/blob/master/Documentation/driver-model/device.txt:
> 
> --><--  
> As explained in Documentation/kobject.txt, device attributes must be
> created before the KOBJ_ADD uevent is generated.
> --><--  
> 
> Note that some drivers even disable KOBJ_ADD notification during
> device_register() and trigger it manually later, after sysfs layout is
> complete. I cannot evaluate whether this directory can be created and
> populated before device_register().
> 
> >  If there are no uevents either, well, there's nothing you can do from systemd's side. (Other than making a script that loops repeatedly checking "is it there yet? is it there yet?")  
> 
> Should really be fixed on kernel side.
> -------- Конец пересылаемого сообщения --------
> 
> _______________________________________________
> vfio-users mailing list
> vfio-users@xxxxxxxxxx
> https://www.redhat.com/mailman/listinfo/vfio-users

_______________________________________________
systemd-devel mailing list
systemd-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.freedesktop.org/mailman/listinfo/systemd-devel