On 01/16/2009 02:47 PM, David Cantrell wrote: > Steffen Maier wrote: >> On 12/02/2008 07:36 PM, Jeremy Katz wrote: >>> On Tue, 2008-12-02 at 07:52 -1000, David Cantrell wrote: >> On s390, udev may also automatically load (network) driver modules >> (except for some hacking required for LCS and CTC). > > I still don't understand the hacking aspect of setting up an LCS or CTC > device. Currently, modaliases for both LCS and CTC devices are in the module cu3088 which both lcs and ctc depend on. I.e. udev loads the dependent module cu3088 but not the necessary driver on top. We're looking at this and see if/how we can fix that, so udev would just work out of the box. Otherwise we could use some udev rule to pull in lcs/ctc based on the control unit model, as soon as some device with control unit type 3088 gets sensed by the common-I/O layer. See also RHIT 116546 and the hack that resulted for RHEL (same problem persists with CTC though): > --- modprobe.conf.dist.orig > +++ modprobe.conf.dist > @@ -162,3 +162,5 @@ > install eth1394 /bin/true > > install snd-emu10k1 /sbin/modprobe --ignore-install snd-emu10k1 && > /sbin/modprobe snd-emu10k1-synth > + > +install cu3088 /sbin/modprobe --ignore-install cu3088; /sbin/modprobe lcs >> After loading driver modules, the probing as we are used to on other >> platforms does not happen in the same way and _no_ (network) device >> kernel structures are allocated at this point. We must not touch >> devices which might be meant for another LPAR/VM. Hence, only the user >> can decide which one (or few) of all the available devices should be >> made available to the local Linux instance. Only then, a (network) >> device gets allocated and the remaining configuration (e.g. IP on >> layer 3) may very well be done as on any other hardware platform. > > Right, which is what's happening now. The direction I'd like to see is > getting rid of the current way we find devices and set them up. Is > there any good reason that it's still all manual on s390? Yes, as mentioned, it is prohibitive to set all visible devices online and allocate device structures in terms of time and memory. Also, this might lead to interference between guests competing for the same full set of devices. > Can this information not be provided to us by the kernel? Which information? The kernel has no knowledge of which devices the user wants to use. > The whole point of > linuxrc.s390 is to collect parameters so that we can load the modules > with the right parameters. Sorry for nitpicking, but it's not parameters for module loading. Device driver modules for s390 nowadays mostly don't use load parameters any more. The parameters linuxrc.s390 collects are applied to sysfs after module loading, since setting a device online is a dynamic process. > And this isn't a problem that we can solve in anaconda, it's a kernel > thing. If the kernel can provide that information to us, we can really > improve linuxrc.s390 and possibly do away with it entirely. Ah, OK, now I can see where you think this should be going. You want all devices to be accessible through sysfs. We cannot afford to set all devices online to be in the required state. So what could the kernel do to help here? The only thing I could think of from the top of my head was some shadow view presented by the kernel to the user space, e.g. via sysfs. In this view all devices look as if they were accessible but in fact they are not yet online and no device kernel structures have been allocated. Problem 1 here is that this requires a good load of heuristic with network devices where subchannels need to be grouped before they make up a potential network device (that's the way the NIC firmware works and we won't change that). I compare such a heuristic to the policy of routing algorithms which is also implemented by a daemon in user space and not in the kernel, where only the mechanics reside, i.e. the packet forwarding. Hence, such a heuristic should reside in user space. Now, if something tries to really work with such a shadow device, the kernel would have to group it, set it online, allocate a real device structure. Problem 2: I'm already ignoring the load of parameters the user might want to configure before setting the device online (because that is how the NIC firmware works). How would we support that if the kernel does magic on his own? Events that trigger something in user space? Then we'd better do all in user space right from the start. > I've only been working on s390 for a few years, but what I've seen > regarding setting up a VM is very redundant. Set up your devices in > z/VM, then set them up again once we boot up the installer. You only tell VM which devices are available in a guest, that's different from setting devices online in the guest. It is not redundant. >> Yet, I don't think the steps for an actual allocation of a network >> device on s390 should be integrated into NetworkManager. Those steps >> are nothing more than a little pre-init we need on s390 to get our >> devices going and available for e.g. HAL/NetworkManager. It's best to >> code the pre-init at one place in the anaconda package and independent >> of any of anaconda's dependencies such as NetworkManager since they >> apparently change over the course of time. > > The reason we mentioned hal and NetworkManager is that anaconda is using > those pieces now, so the correct location to fix things is always the > dependent component whenever possible. The more special case device > setup knowledge we put in to linuxrc.s390, the more difficult it becomes > to maintain. While we probably can't eliminate linuxrc.s390, it would > be nice to expand hal and other dependent components to perform tasks > that are more appropriate to them. Rather than crawling /sys in > linuxrc.s390, let hal do it (as an example). We can then ask hal for > the information. Maintenance of that information is now in one location > rather than in anaconda AND in hal. Our current pre-init in linuxrc.s390 sets up devices so that they can be used with hal and friends without even having to change the latter. Maintenance of that information is already in one place, namely anaconda. Your suggestion just moves the maintenance to another place and brings a lot of unsolved issues we currently don't even have: A solution that suggests itself, would be to set all visible devices online => hal and friends will explode with thousands of devices. As mentioned above, there are even more reasons why we MUST NOT set all devices online just in case we need them later on. Side node: cio_ignore. The common-I/O layer senses devices on s390 so they appear in sysfs. Even that might take too long with thousands of devices (tens of minutes), so there is a device blacklist where you can stop common-I/O from even sensing certain devices. And all this even happens long before device kernel structures could be allocated and hal gets anything to see. Hope you understand my point, that even long before hal there might already be show stoppers, so the later we would try to set all of them online in this device setup process the bigger the explosion will be. Just like described for the kernel variant above, hal and friends could alternatively be extended to provide a shadow view of all visible devices without actually setting them online and without actual kernel device structures. Only when a user wants to use one of those devices, hal would have to set it online on short notice. That's a lot of code to get this working reliably. Each time hal gets replaced by something else, you would have to recode all this. To me, this sounds even worse than the kernel solution. All my solutions assume that hal does not have any interactive component where the user may choose devices to use. I hope this constraint is valid. At the risk of repeating myself: All this is no problem, since after our little pre-init helper the devices are there and ready to be used as on all other platforms. Pre-init is good. >> Also, NetworkManager relies on DHCP for e.g. auto-configuring network >> layer addresses. We cannot assume to have a DHCP server for Linux >> guests on System z environments. Therefore, we need a way to manually >> configure network addresses in any case. And that is what we already >> have in linuxrc.s390 and if I understood it correctly, NetworkManager >> may just use the pre-configured devices and everything would just work >> out fine. > > That's not entirely true. NetworkManager does have static IPv4 support > now, but better than that it comes with nm-system-settings. If you look > through loader in rawhide, you'll see that we write out > /etc/sysconfig/network-scripts/ifcfg-DEVICE files and nm-system-settings > manages those and speaks to NetworkManager for us. You're right, that's what I meant. Sorry for my vague wording due to missing knowledge. Meanwhile I've seen the new anaconda code and with our pre-init in linuxrc.s390 it should just work. If parameters from the UI or parm file get handed to the loader and the loader writes such an ifcfg, then the use of the network device happens transparently as you described and we're good. >> Above said holds not only true for network devices but also for >> e.g. mass storage such as DASD or FCP on s390. [snip] > We definitely have other technologies that require 'pre-init' before we > can work with them. Things like iSCSI, for instance, which is strange > magical storage technology and fairly complex in terms of wrapping your > head around. So does s390 and the more I think about it the more I like it. It's comparable to the Unix approach on the command line where you have small tools, each one doing some specific thing. Then you combine them to achieve bigger things. On s390 we combine the pre-init of linuxrc.s390 with the existing loader and anaconda. > I think we're looking at this the wrong way. Not necessarily init.c > gaining the capabilities of linuxrc.s390, but loader gaining them when > appropriate. That's where we do everything else. So far, I've been assuming loader requires a curses capable terminal since it uses newt for its TUI, right? We cannot assume to have such a terminal on s390. Hence, it is impossible to do interactive pre-inits in loader. Or did I get something wrong here? > Additionally, rawhide might lose init soon as we are working on > integrating upstart to replace anaconda's special init process. Now I learnt something new. So in a nutshell, upstart seems to be a more flexible replacement for sysV-init and it is used since F9 in the installed system. Unfortunately, I could neither find any information on what are the implications using upstart in an installer nor do I have any clue what this means. Could you please point me to some more information? >>>> Even if we keep the linuxrc.s390 script, >>>> reducing it to bare minimum and turning things over to the real init >>>> when we are done are better than what we have now. >> After digging into F10's init.c (as a more recent/rawhidish version) >> and finding some code duplicates between init.c and linuxrc.s390, I >> really like that idea and am willing to give it a try. [snip] >> Anyway, the idea is as follows: Reducing code duplicates and thus >> improving maintenance by doing the common environment initialization >> in init.c. [snip] >> At the point where init.c would fork the loader, we would statically >> replace this with an #ifdef and instead fork a stripped down >> linuxrc.s390 which only contains the s390-specific device pre-init >> including the necessary UI. [snip] > I think this is a reasonable thing to try. Cool, so I'll give that a try. Might take some time, though. I'll post an untested prototype for review. I defer upstart conversion to a 2nd step and make it working with the current init first. Steffen Linux on System z Development IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martin Jetter Geschäftsführung: Erich Baier Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294 _______________________________________________ Anaconda-devel-list mailing list Anaconda-devel-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/anaconda-devel-list
