Steffen Maier wrote: > On 12/02/2008 07:36 PM, Jeremy Katz wrote: >> On Tue, 2008-12-02 at 07:52 -1000, David Cantrell wrote: >>> Jeremy Katz wrote: > >>>> But taking a step back to some more general thoughts on linuxrc.s390, is >>>> there an effort to actually get it using the same things we use >>>> everywhere else? > >>>> and then manual network configuration >>>> instead of using NetworkManager. > > AFAIK, NetworkManager is able to manage network devices such as ethX > if they already exist. On most platforms this is the case after > loading the correct driver kernel module which can be triggered by > udev with the help of modaliases. Correct. > In the following, I'd like to point out why s390 looks different. I > only work on s390 since about a year now. Prior to that, I worked with > Linux on x86 for more than a decade. Therefore, I can see the > differences from both "sides" and hopefully shed some light on s390's > specialties without much bias. > > On s390, udev may also automatically load (network) driver modules > (except for some hacking required for LCS and CTC). I still don't understand the hacking aspect of setting up an LCS or CTC device. > After loading driver modules, the probing as we are used to on other > platforms does not happen in the same way and _no_ (network) device > kernel structures are allocated at this point. We must not touch > devices which might be meant for another LPAR/VM. Hence, only the user > can decide which one (or few) of all the available devices should be > made available to the local Linux instance. Only then, a (network) > device gets allocated and the remaining configuration (e.g. IP on > layer 3) may very well be done as on any other hardware platform. Right, which is what's happening now. The direction I'd like to see is getting rid of the current way we find devices and set them up. Is there any good reason that it's still all manual on s390? Can this information not be provided to us by the kernel? The whole point of linuxrc.s390 is to collect parameters so that we can load the modules with the right parameters. It seems so outdated. Why can't the system provide that information for us? And this isn't a problem that we can solve in anaconda, it's a kernel thing. If the kernel can provide that information to us, we can really improve linuxrc.s390 and possibly do away with it entirely. I've only been working on s390 for a few years, but what I've seen regarding setting up a VM is very redundant. Set up your devices in z/VM, then set them up again once we boot up the installer. Perhaps it's just my environment and/or lack of s390 knowledge. > Yet, I don't think the steps for an actual allocation of a network > device on s390 should be integrated into NetworkManager. Those steps > are nothing more than a little pre-init we need on s390 to get our > devices going and available for e.g. HAL/NetworkManager. It's best to > code the pre-init at one place in the anaconda package and independent > of any of anaconda's dependencies such as NetworkManager since they > apparently change over the course of time. The reason we mentioned hal and NetworkManager is that anaconda is using those pieces now, so the correct location to fix things is always the dependent component whenever possible. The more special case device setup knowledge we put in to linuxrc.s390, the more difficult it becomes to maintain. While we probably can't eliminate linuxrc.s390, it would be nice to expand hal and other dependent components to perform tasks that are more appropriate to them. Rather than crawling /sys in linuxrc.s390, let hal do it (as an example). We can then ask hal for the information. Maintenance of that information is now in one location rather than in anaconda AND in hal. > Also, NetworkManager relies on DHCP for e.g. auto-configuring network > layer addresses. We cannot assume to have a DHCP server for Linux > guests on System z environments. Therefore, we need a way to manually > configure network addresses in any case. And that is what we already > have in linuxrc.s390 and if I understood it correctly, NetworkManager > may just use the pre-configured devices and everything would just work > out fine. That's not entirely true. NetworkManager does have static IPv4 support now, but better than that it comes with nm-system-settings. If you look through loader in rawhide, you'll see that we write out /etc/sysconfig/network-scripts/ifcfg-DEVICE files and nm-system-settings manages those and speaks to NetworkManager for us. > Above said holds not only true for network devices but also for > e.g. mass storage such as DASD or FCP on s390. S390 might currently be > the only platform requiring such a pre-initialization of devices due > to the fact that there might be thousands of them and allocating a > kernel device structure for each of them just in case is prohibitively > expensive in terms of memory and probing time. I think the huge > difference in number of devices is what s390 currently separates from > other platforms and should be accepted this way. As other platforms > grow they might sooner or later just get into the exact same situation > where they can no longer afford to init and allocate each detected > device just in case. Then they may make use of the current "special" > handling of s390 and platforms will behave similar again. We definitely have other technologies that require 'pre-init' before we can work with them. Things like iSCSI, for instance, which is strange magical storage technology and fairly complex in terms of wrapping your head around. The fact that s390 essentially provides two types of devices to VMs: disks and network interfaces (apologies to Hans Picht, but I really haven't worked with the tape device), I really don't see why the kernel can't be involved for a lot of this pre-initialization. Maybe not all of it, but some. Again, my knowledge of the architecture is limited and every time I talk to you guys, I learn something knew. But I think if we push in the direction of the kernel doing more rather than linuxrc.s390 doing more, it will make our life easier. I do understand the s390 is special, but rather than taking that at face value, it's try top reduce linuxrc.s390 to the bare minimum that needs to happen. >>>> As we've moved more of these bits to >>>> be common, we really probably need to sit down and rework the entire way >>>> that the script works. The goal of it is that it takes the place of >>>> loader/init.c with a few notable and important differences >>>> * Prompting for information on things like network setup if you don't >>>> set it in the parm file. This pretty much has to be "custom" for s390 >>>> and doing it with shell is as easy as anything else >>>> * Getting modules loaded. We don't do this until in the loader on other >>>> arches, but we need to do it to get the network up. But we really >>>> probably want to let udev do it as much as we can >>>> * Starting the network. Which we do with spawning NetworkManager in the >>>> "normal" case >>> This mirrors a conversation I've had with IBM recently. Personally, I'd >>> like linuxrc.s390 to go entirely. It's a maintenance nightmare and >>> init.c can just grow to include s390 code where needed. > > Honestly, I fear that much of the necessary s390 code in init.c will > remain so special that still nobody wants to maintain it. We would > wind up in the same situation we already have with > linuxrc.s390. Therefore, a _full_ integration of linuxrc.s390 into > init.c does not seem worth it. I think we're looking at this the wrong way. Not necessarily init.c gaining the capabilities of linuxrc.s390, but loader gaining them when appropriate. That's where we do everything else. Additionally, rawhide might lose init soon as we are working on integrating upstart to replace anaconda's special init process. >>> Not only is >>> there a different init for the s390 initrd, the initrd itself is >>> significantly different from the other architectures. > > I see that point. > >> Yep. And now that we've bit the bullet and accept a "larger" initrd for >> most cases, there are people that would actually really like the ssh >> functionality that we have on s390. > > OK, this functionality might indeed be interesting for all platforms > and may become non-s390-specific if it was in init.c. In fact, we > would make s390 behave like other platforms by making all platforms > behave like s390 already does. It brings us closer to having almost > the same initrd on s390 as on other platforms. Be aware though, that > loader may then no longer be init's child and they would have to > communicate with each other by means of IPC as is the case for > linuxrc.s390 already. If we used the same IPC interface for all > platforms, we had no problem and everything was fine. But this is a > major change affecting all platforms which requires testing on all > platforms, of course. The less special case stuff we have for s390, the better. If we move to upstart in anaconda, life becomes easier as well. Especially if loader takes over doing what linuxrc.s390 has to do. >>> IBM is very interested in reworking the linuxrc.s390 system as well. >>> For 5.x, they have a nice rewrite that improves the input loops and >>> module loading. For rawhide, I am at least wanting to get away from >>> having linuxrc.s390 at all. Even if we keep the linuxrc.s390 script, >>> reducing it to bare minimum and turning things over to the real init >>> when we are done are better than what we have now. > > After digging into F10's init.c (as a more recent/rawhidish version) > and finding some code duplicates between init.c and linuxrc.s390, I > really like that idea and am willing to give it a try. However, I'm > still unsure how to test it. Maybe I can use the preliminary F9 build > on s390. Speaking of Fedora on s390, who is heading that up? We should really have that set up as a downstream arch now that pulls in each rawhide build that gets submitted. Or however it can happen. Working with F-9 is uselsss now. > Anyway, the idea is as follows: Reducing code duplicates and thus > improving maintenance by doing the common environment initialization > in init.c. An example is the function createDevices which also exists > in linuxrc.s390 doing the exactly same thing as in init.c, namely > manually creating some basic device nodes in /dev before firing up > udevd. > > At the point where init.c would fork the loader, we would statically > replace this with an #ifdef and instead fork a stripped down > linuxrc.s390 which only contains the s390-specific device pre-init > including the necessary UI. Of course it needs some more glue to > support the IPC between init and the loader, which gets started later > on when the user logs on via ssh. > > What do you think? I think this is a reasonable thing to try. Keep an eye out for patches that may take us to upstart. Rather than putting things in init.c, might go for loader.c instead. >>> s390 is such a >>> special case anyway though, it'll take some more planning. >>> >>> Right now, I am happy with IBM improving the init system we have now. >>> They are far more knowledgeable about their platform than I am. >> 100% agreed, and to be honest, that was kind of what I was getting at >> without wanting to explicitly call them out and say "hey, come to the >> party and let's work to make this suck less" :) > > We're already there and working on that. But maybe we have a slightly > different focus. We try to improve usability and actively contribute > to Fedora on s390 to provide a solid basis for RHEL6. We approach the > latter by getting it to work gradually in small steps without breaking > existing functionality and if necessary reuse working code from RHEL5. > > Please correct me if I'm wrong, but your focus seems to be on > improving maintenance by treating all platforms the same way. > This is basically a good thing. Yes, that's one aspect. The other is actually improving anaconda by rewriting/reworking/implementing large parts. > I think we can achieve the best of both worlds by following our > approach until we got something working and after that there's still > the chance to redesign bit by bit if there's need. The big advantage > is that we continually had a working code-base which is very important > for getting Fedora on s390 in the long run IMHO. Correct. Another thing to keep in mind is that large projects like this that we want to do on the Fedora side become basically impossible to backport to a RHEL release. But that's ok, because the functionality will land in the next major RHEL release. -- David Cantrell <dcantrell@xxxxxxxxxx> Red Hat / Honolulu, HI _______________________________________________ Anaconda-devel-list mailing list Anaconda-devel-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/anaconda-devel-list
