On Saturday, 23-11-2019 at 15:07 Cole Robinson wrote: > On 11/22/19 5:12 AM, Daniel P. Berrangé wrote: > > On Thu, Nov 21, 2019 at 07:44:38PM -0500, Cole Robinson wrote: > >> On 11/21/19 5:28 AM, Daniel P. Berrangé wrote: > >>> One option is to take the lazy approach and always enable cloud init > >>> if we are given a pre-built disk image. > >>> > >>> * If the disk image doesn't support cloud init, then its fairly > >>> harmless to provide cloud-init data. > >>> > >>> * If the disk image does support cloud init but has already been > >>> booted, and the user has marked cloud init as disable, then it > >>> is again harmless to provide cloud-init data > >>> > >>> * If the disk image does support cloud init but has already been > >>> booted, and the user left cloud init enabled so that it runs on > >>> every boot, then passing cloud init data is probably good thing. > >>> > >>> * If the disk image does support cloud init and has never been > >>> booted, then obviously providing cloud init data is good > >>> > >> > >> If we had a SMBIOS or -fw_cfg option, then maybe this is an option. But > >> I don't think I could stomach doing this by default with the current > >> 'cdrom' approach, surely there would be complications, and at minimum > >> complaints about a needless attached CDROM device. > > > > Yeah, I can understand this concern. > > > >> I think the 'cdrom' case is good enough for a first crack at this and > >> anything else can build on that initial support. I think the command > >> line is flexible enough that in the future we could have a --cloud-init > >> method=SMBIOS/... for users to explicitly choose what injection method > >> they want, but otherwise virt-install will choose a default, which may > >> be different depending on the arch. > > > > That sounds reasonable. > > > >>>> * Do you have an opinion of what behavior bare '--cloud-init' should give? > >>> > >>> My feeling is that a bare --cloud-init should inject the current user's > >>> SSH identity and/or authorized keys list. > >>> > >>> ie either we grab $HOME/.ssh/id_rsa.pub, or we grab all the keys from > >>> the SSH agent connection (ie ssh-add -l) > >>> > >>> Given that virt-intsall is run as root, there probably isn't a > >>> /root/.ssh/id_rsa.pub file for most people. Grabbing the ssh-agent > >>> authorized keys is probably best default approach. > >> > >> How to handle ssh keys kinda makes my head spin. I've shrunk the CC list > >> on this mail to just the folks that mentioned ssh in some way in their > >> replies > >> > >> If I'm thinking of the ideal interactive virt-install UI, it would be: > >> > >> $ virt-install --name MYVM --disk Fedora-Cloud-...qcow2 --cloud-init > >> INFO: SSH keys added to MYVM > >> INFO: Connecting to MYVM with: ssh fedora@xxxxxxxxxxxxxxx > >> ... > >> [fedora@localhost:~] $ > >> > >> > >> If all the pieces come together, that's pretty nice. But there's a lot > >> of missing pieces needed for that, and some unclear bits: > >> > >> * What is the cloud-init user name? Passing in an ssh key gives access > >> to the default cloud-init configured username, which is different > >> depending on the distro (might be fedora, ubuntu, etc). To properly > >> handle this, we either need libosinfo distro detection, or use > >> libguestfs to fish the default user out of /etc/cloud/cloud.cfg in the > >> VM. And root account is typically locked, and ssh access disabled, and > >> even if we could hack things to give root ssh access it seems like going > >> against the grain WRT cloud images. > > > > IIRC, you can pass an SSH key for root separately, but not sure how > > widely supported that is. Sticking to just providing 1 SSH key for > > the default unprivileged user likely suffices. > > > >> So best I can think of for an immediate solution for SSH is to print > >> something explanatory: > >> > >> INFO: you can ssh to the VM using the distro's default username. This is > >> usually one of: fedora, ubuntu, ... > >> > >> Or something along those lines. Maybe we can tie that into --os-variant > >> as well so the user can inform us what distro it is. Still it will take > >> some work. > > > > This is a bit of a tangent, but this makes me wonder if qcow2 format > > could benefit from having support for arbitrary user metadata fields > > > > People building cloud images could then tag their images with useful > > things including vendor, product name, product version, whether it > > supports cloud-init or ignition, what the default user account name > > is, suggested min ram, and so on. > > > > Of course people might say this is all stuff that's within scope > > of external metadata like OVF, but I've rarely, if ever, seen anyone > > say anything nice about OVF. Also there's a certain convenience in > > being able to directly pass around a bootable disk image, rather than > > a disk image + separate metadata file, or worse a ZIP file containing > > both which you have to unpack. > > > > Having some simple metadata in the qcow2 file would simplify life I > > think. If someone really wanted to use OVF, they could stuff the OVF > > XML descriptor into a qcow2 metdata field. > > > >> * Where to get ssh keys from? You mention ssh-add -l which is nice, I > >> didn't know about that. There's the issue with running virt-install > >> under sudo, but not sure there's anything we can do about that to make > >> it 'just work'. Sounds like `ssh-add -l` is the way though for some sort > >> of ssh-key=auto mode > > > > Yeah, sudo breaks it, as does "su -". The ssh agent only gets > > forwarded if you're accessing the virt host over SSH. This is > > the case for me usually, as my virt servers are separate from > > my laptop. > > > > I don't see much of an alternative to ssh-agent though, besides > > just defining that virt-install will grab a key from a well known > > place like $HOME/.config/virt-install/ssh-injection-keys > > > >> * What to do if we can't find any ssh-keys to add? I guess explicitly > >> fail, and print a descriptive error: We didn't find any SSH keys to add. > >> Maybe you are running under sudo. You can manually specify a key with > >> XXX or generate root password with YYY. (something better than that of > >> course) > > > > Depends whether we want to also auto-generate root and/or user passwords > > by default, or only work with SSH keys by default. > > > > My personal preference is to only deal with SSH keys by defaults, as > > passwords are just terrible in general. This would mean if we can't > > find a SSH key, we should abort the install & require an explicit > > setting. > > > >> * virt-install support for launching an ssh session as the UI. This is > >> probably worth doing regardless. I'd like to add an explicit > >> --autoconsole none|graphical|text anyways, adding ssh to that seems > >> reasonable. Requires waiting for VM IP address to pop up, but that > >> shouldn't be too bad to implement. The main problem for the --cloud-init > >> case is that I don't know if it makes sense to use this if we are just > >> guessing about the what the default username is? > > > > We would need to solve the default username problem reliably > > for this to be usable, otherwise it'll be a neverending source of > > bug reports. > > > >> So, to get that ideal UI, or at least point the user in the right > >> direction, we need to: > >> > >> * determine the default user name for the disk image, which we don't > >> have a solution for presently > >> * successfully inject a relevant ssh key, which may not be available > >> like under sudo > >> > >> So there's missing non-trivial work, and there's ways it can fail. > >> > >> The current UI, with root-password-generate=on (plus some work to > >> default to '--console text' when --cloud-init is passed), is more > >> reliable as it should work in all situations, but even when it works > >> it's not as friendly as the working ssh case (need to copy the printed > >> password and paste it into a terminal). And it doesn't sound like it's > >> what people expect the default to be. And giving root access seems to be > >> against the spirit of the cloud images, but maybe that's just for ssh access > > > > Personally I think from a security POV we should do our best to > > discourage & avoid use of passwords, at least for server based > > installs. > > > > IOW, if using the JEOS profile then we should prefer SSH keys. > > Essentially my view is that passwords are just there as a fallback > > for disaster recovery when you have to boot single user mode to > > fix a broken system. Even then we don't really need that for > > VMs, because virt-rescue provides a backdoor into any broken > > VM. > > > > The key place where passwords are needed are when we are doing > > graphical installs. The user connects to the graphical console > > with VNC/SPICE, and password is the only option there really. > > > > Okay after mulling it over I think I've come full circle :) My plan: > bare --cloud-init stays as is: generate a root password and set it to > immediately expire. Anything ssh needs to be explicitly opt in. Never > plan to trigger any --cloud-init behavior automatically, require it to > be opt in. > > Here's my thinking: the ssh case is too complicated to be the default. > We've covered it in previous mails; there's many ways it may not 'just > work'. It might be what some or most users want out of using cloud > images with virt-install/virt-manager, but those users likely 1) already > have some cloud-init or cloud image specific knowledge, and 2) have a > good understanding of ssh keys. So, I'm comfortable letting those users > figure out themselves that they need --cloud-init ssh... or similar. > > The root-password-generate=on case, combined with using `--autoconsole > text` to give terminal login access to the VM is the best mix I can come > up with of simple and reliable. It should always work, and doesn't take > much special explanation besides 'this is your root password, copy it > into the login prompt', and then set a new password on first login. I > understand that we shouldn't really be encouraging password auth but I > think it's unavoidable here as a 'please just let me log in to this disk > image' hammer > > Another benefit of doing this as the sorta recommended --cloud-init > option for virt-install is that it will be closer to what I think we > should provide in virt-manager UI: some checkbox like 'This is a cloud > image' that gives an option to set a custom root password. The user can > log in to the imported VM easy enough through the graphical console. > > I was making a big deal out of this bare '--cloud-init' option because I > was presuming that some day we may want to use it by default if we > detect we are passed a cloud image. But I've changed my mind a bit. Even > if today we could perfectly detect if the user provided disk image is a > distro cloud image, I think all we should do for virt-install is print a > warning like: WARNING: distro cloud image detected. you may not be able > to log in without specifying --cloud-init options. try FOO or BAR...`. > This way the user always has to explicitly request some kind of > --cloud-init interaction. I think this is the only way avoid a lot of > ambiguity, and possibly avoid undesirable behavior in some subtle > scenarios. > > The one case I could see us using some --cloud-init values by default, > is if we use libosinfo to point us to a distros's cloud image which we > download on demand. Say something like 'virt-install --install > fedora31,variant=cloud`. In that case we know we are dealing with a > cloud image, which distro, we know the image is pristine, so it would > make sense to default to --cloud-init as well. > > But in the end, bare `--cloud-init` will still be the recommendation for > someone who justs wants to log in to that distro disk image they > downloaded. If the user is cloud-init savy and knows they want to get > ssh keys into the VM, they likely know enough to figure out how to make > it work with the explicit options we will provide. > > > Here's a finer grained task list. But for those who have read this far, > I'm not planning on working on this in the near term, probably not > before the end of the year at least > > * Add --autoconsole none|default|text|graphical > * Any --cloud-init usage implies --autoconsole text. > * Add --cloud-init ssh-key=agent or similar: the ssh-add -l behavior. If > ssh-add -l fails, print some hints and fail > * Add `--cloud-init none` as future proof incase we ever choose to make > --cloud-init implied for some usecases > * make sure root-password-generate=on doesn't pause output unless run > interactively > * test a bunch of different distro cloud images, make sure our settings work > * man page document it all > * possibly virt-manager UI for it +1 > * stretch: possibly virt-install --install fedora30,variant=cloud install > * stretch: libosinfo track default cloud image usernames. may come in > handy at some point > > Thanks, > Cole > > _______________________________________________ > virt-tools-list mailing list > virt-tools-list@xxxxxxxxxx > https://www.redhat.com/mailman/listinfo/virt-tools-list _______________________________________________ virt-tools-list mailing list virt-tools-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/virt-tools-list
