On Sun, 2004-11-28 at 20:49 -0500, David Zeuthen wrote: Awesome! So... is no-one else going to reply to this email? Various comments inline below... > Hey, > > So, I've looked a bit more into the booting process and how to optimize > it. Mostly based on the discussion triggered by Owen's boot poster > challenge, here > > http://www.redhat.com/archives/fedora-devel-list/2004-November/msg00447.html > > and also some experiments that I did - basically replacing rhgb with gdm > as described here > > http://www.redhat.com/archives/fedora-desktop-list/2004-November/msg00066.html > > What I've done is a bit crude - I've replaced init(1) with a shell > script based on /etc/rc.d/rc.sysinit and tried to optimize specifically > for my system: IBM Thinkpad T41 laptop with a Pentium M processor at > 1600MHz along with 512MB of RAM. If I understand, you're also optimising for a specific user of that system. Is it worth splitting the readahead into a system-wide list of files (enough to get to the login screen), followed by a per-user list for logging in as a user? To what extent will the files needed to get to a usable desktop vary between Alice and Bob? > > The results are pretty good I think, here is the general time line made > with a wallclock > > 00: exit grub; start booting the kernel > 04: kernel prints audit() > 11: initrd is mounted; Red Hat nash visible > mount / ro (normal initrd procedure) > 13: start bootchart logging; start readahead of approx 193MB files > sleep until readahead is complete > 24: readahead done; now > create /dev and modprobe (in background) > mount / rw, enable swap > start xfs > startx as user davidz in background > start messagebus > start hald > start acpid > start NetworkManager > 32: X claims the display > 34: GNOME desktop banner > 40: GNOME desktop is usable (Nautilus desktop, panel fully populated) > > Here is a bootchart made with the bootchart software from Ziga Mahkovec: > > http://people.redhat.com/davidz/bootchart.png > You may notice that I also start firefox after login and it starts very > very fast - that's because readahead loads all files used by Firefox - > in earlier experiments I've also added files from OpenOffice.org to > readahead and that meant I could start up OpenOffice.org Writer in about > three seconds. More below. > > I've made the following observations > > 1. The kernel patch, linux-2.6.3-printopen.patch, wasn't really working > well for me - it reported far to few files - instead I added a > printk() to fs/namei.c:link_path_walk() > (disclaimer: I don't know much about the kernel so there may be a > better solution than this). > > 2. The data captured from link_path_walk() was massaged into a list > of unique files to preload and sorted on sectors. > > 3. While capturing the data link_path_walk() and before processing > I went through all the menus in the GNOME desktop (to make sure > their icon and desktop files would be added to the list) as well as > loading Firefox. The list contains 5189 unique files - 231 of these > from my home directory - 103 of these from gconf in my home > directory and 302 from gconf in /etc. 2267 were .png files and > 814 of them were .desktop files. 488 files had ".so" in their name. > There was a total of 193MB of files (which says something about > the footprint of the GNOME desktop on Fedora :-/) > > 4. Doing the readahead really helped the time from startx till a > usable desktop - less than 10 seconds! > > 5. Doing readahead on the 5189 files took about 45 seconds on my > system, mostly because the files were scattered around the disk. > Since I had a spare partition 17GB partition, I did this: > a. format spare partition as ext3 > b. copy all readahead files to spare partition (193MB) > c. copy rest of files from main partition to spare partition > (about 9GB) > Now the readahead is down to 11 seconds which averages out to > be 18MB/s. On the other hand, I can still see (using fileblock) > that the files in the readahead is still scattered out and hdparm > says I should be able to get 33.87 MB/sec with no seeks. > > 6. I made a hack to cache /dev (a dev.tar file) and the list of modules > to load. This could be used in production if the kernel could give > us basically a hash value for the kobject hierarchy representing > the hardware (perhaps even a 'tree /sys |md5sum' would suffice). > This shaved some seconds of as well. > > 7. A number of things was started in parallel - I found that doing > readahead while running modprobe wasn't helping anything; in fact > it contributed negatively to performance (a bit to my surprise, I > guess because the kernel was busy). > > 8. readahead on the right files is A Good Thing(tm). Booting my system > without readahead on the partition with the readahead files scattered > took 58 seconds (compared to 39 with readahead on the optimized > partition) > > http://people.redhat.com/davidz/bootchart-without-readahead-scattered.png > > and without readahead on on the optimized partition it took 43 > seconds > > http://people.redhat.com/davidz/bootchart-without-readahead-nonscattered.png > > again compared to 39 seconds. As an added bonus, the readahead > makes sure that e.g Firefox loads fast; all .png and .desktop files > are in place for when using the menus. As mentioned, one could put > very big apps like e.g. OO.o in the readahead set. > > So, I think these numbers are good and there's still some room for > improvement; e.g. it takes ten seconds from grub to when the initrd is > mounted - surely the kernel can boot faster? It's after all 25% of the > time spent from grub until I have usable desktop. > > The bad thing is that this approach is highly specific to my system (and > thus why I'm not posting an RPM with it :-), however I think it clearly > shows where improvements should be made; here are some random thoughts > > a. We should keep track of files being loaded and maintain the > readahead fileset as appropriate. printk() doesn't seem like the > right solution; perhaps a system daemon using inotify or the > kernel events layer is the road ahead? This would enable us to > readahead the KDE stuff if the user is e.g. using KDE a lot. > > b. ext3 should support operations for moving blocks around; e.g. > optimize around the readahead fileset - when idle the system should > rearrange the files to facilitate faster booting > > c. the start_udev and kmodule process could be cached as I did above > > d. The whole init(1) procedure seems dated; perhaps something more > modern built on top of D-BUS is the right choice - SystemServices > by Seth Nickell comes to mind [1]. Ideally services to be started > would have dependencies such as 1) don't start the gdm service > before /usr/bin/gdm is available; 2) the SSH service would only > be active when NetworkManager says there is a network connection; > /usr from LABEL=/usr would only be mounted when there is a volume > with that label and so forth. Also, such a system would of course > have support for LSB init scripts. > (This is probably a whole project on it's own so I'm omitting > detailed thinking on it for now) Hopefully this could also allow us to make system-config-services look a lot slicker. I've never liked the way it has random text spewage for each service's status - some kind of widgetry would satisfy my eye-candy cravings. > > Thanks a lot to Ziga Mahkovec for the bootchart software - it's been > very useful. > > Have fun, > David > > [1] : http://www.osnews.com/story.php?news_id=4711 > http://www.gnome.org/~seth/blog/2003/Sep/27 >
