On Mon, Nov 25, 2019 at 11:23:14AM +0000, Daniel P. Berrangé wrote: > On Mon, Nov 25, 2019 at 11:13:00AM +0000, Richard W.M. Jones wrote: > > On Fri, Nov 22, 2019 at 09:56:17AM +0000, Daniel P. Berrangé wrote: > > > On Thu, Nov 21, 2019 at 09:39:46PM -0500, Dusty Mabe wrote: > > > > > > > > > > > > On 11/21/19 6:20 AM, Daniel P. Berrangé wrote: > > > > > On Thu, Nov 21, 2019 at 11:07:24AM +0000, Richard W.M. Jones wrote: > > > > >> On Thu, Nov 21, 2019 at 10:34:14AM +0000, Daniel P. Berrangé wrote: > > > > >>> On Wed, Nov 20, 2019 at 08:18:01PM -0500, Dusty Mabe wrote: > > > > >>>> Basically in Fedora CoreOS we need a generic user data mechanism that works across > > > > >>>> platforms (x86_64, aarch64, ppc64le, s390x) and doesn't have possible race conditions. > > > > >>>> Right now we're using `-fw_cfg` but it's not cross platform. We don't have an answer > > > > >>>> yet: https://github.com/coreos/ignition/issues/666#issuecomment-452835654 > > > > >>> > > > > >>> For platform portability you need to find some device that is common > > > > >>> across all platforms, and either disk or network are the only two > > > > >>> good options that exist today or for the forseeable future. If those > > > > >>> aren't acceptable then all we have left are platform specific options. > > > > >> > > > > >> While it's not a "good option that exists today", AF_VSOCK would be a > > > > >> good choice to settle on in the future. It's completely cross > > > > >> platform, available for Windows, and doesn't interfere with existing > > > > >> network or disk devices. > > > > >> > > > > >> Would needing virtio be a barrier? Our impl of AF_VSOCK runs over > > > > >> virtio, but there are other transports. > > > > > > > > > > From a cloud-init POV, I don't see virtio as a barrier. Defining an > > > > > AF_VSOCK data source for it should be quite straightforward really > > > > > and they already have so many data sources, it seems reasonable > > > > > that they'd accept one more. > > > > > > > > > > On the host side there's still the task of providing a metdata > > > > > service to expose the data, which is outside scope of virt-install. > > > > > > > > > > > > > Thanks for the fruitful conversation here regarding a cross platform data source > > > > that we could use. Is this worth us writing up into a request for an issue tracker > > > > somewhere where it could be discussed further? > > > > > > Possibly - depends if AF_VSOCK is viable or not for Ignition. > > > > > > Previously when we suggested use of a plain disk for Ignition, the issue > > > raised was that the /dev/sdXXX device nodes take a non-deterministic > > > amount of time to appear, and ignition doesn't know how long to wait > > > for them, because the disks might not exist at all in the first place. > > > > > > Using AF_VSOCK is going to have exactly this same problem, so personally > > > I can't see it can satisfy Ignition, where virtio-blk failed. > > > > Be interesting what exactly their concerns were. Obviously both PCI > > initialization and (especially) kernel disk enumeration can take a > > long time with guests that have a lot of PCI slots or disks. But > > that's what udev is for, right :-) Couldn't they just hook up their > > initialization to a udev rule? > > I mention it above - they don't know whether the user has even > provided the metadata in the first place. They don't want to wait > for a device that doesn't exist or they'll be waiting forever. If > you wait with a timeout, then it delays boot, and creates failure > conditions under high load. Maybe kernel command line option indicating "yes there really is a metadata socket/disk, please wait for it"? Are these Ignition images built specially for this purpose? Rich. > > > The main benefit I see of AF_VSOCK over virtio-blk disk / cdrom, is that > > > it is a live data channel so you are not restricted to providing data that > > > is fixed at boot time, it can be live updated on the fly. > > > > > > The only mechanism that can avoid the problem of waiting for some > > > device driver to initialize asynchonrously is to use a directly > > > mapped memory region that is immediately exposed by the core kenrel > > > functionality. This is what fw_cfg, SMBIOS, and/or kernel command > > > line args all achieve, and the other options not based on mapped > > > memory will all fail to achieve. > > > > > > Regards, > > > Daniel > > > > I added Stefan to CC because he might be interested in this use case too. > > > > Rich. > > > > -- > > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones > > Read my programming and virtualization blog: http://rwmj.wordpress.com > > Fedora Windows cross-compiler. Compile Windows programs, test, and > > build Windows installers. Over 100 libraries supported. > > http://fedoraproject.org/wiki/MinGW > > Regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v _______________________________________________ virt-tools-list mailing list virt-tools-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/virt-tools-list
