splashutils-1.1.9.6 and beyond

With splashutils-1.1.9.6 out in the wild, and with >1.0 mask removed from package.mask, I thought it’s a good time to write a few words about what has been going on in the project for the last few weeks, and what is likely to follow.

As you can see in the changelog, there have been quite a few changes, fixes and updates in 1.1.9.5 and 1.1.9.6. Among the most important of these are: the new splash_manager script, merged MNG code and the new packaging
scheme.

Starting with 1.1.9.6, splashutils will be released in two versions — ‘full’ and ‘lite’. The ‘full’ version is just a continuation of what I’ve been doing for the last months — ie. a tarball which includes the core splashutils code and the bundled libraries which are built and linked against klibc. The ‘lite’ tarball contains only the core splashutils code, and thus is much smaller. media-gfx/splashutils will be using the lite tarball from now on, to save users’ bandwidth.

The MNG code is just a stub for now. A lot of important functions are already implemented, but it’s not usable from an user’s standpoint. This will change in 1.1.9.7 or 1.1.9.8. Due to problems with threads with klibc and large initramfs images, animations will only be supported in the splash daemon, and not in the kernel helper (which means that you won’t be able to see any animations at the stage at which ‘Initializing the kernel’ is displayed on the screen).

The splash_manager is a handy utility which can be used for all kind of theme-related tasks, such as theme setting, switching, testing and listing. It should make life easier for all those people who don’t have the time or will to read the documentation to learn how the whole thing works or how exactly should it be configured (make no mistake, it’s not an invitation to completely ignore the docs!). I’ve done some testing, but I’m sure there are still bugs. If you happen to find any of them, please report it on our Bugzilla. The script is undocumented, but it has a long help message, which I hope will prove to be helpful.

Lan Party

Having finally finished my high school final exams, I decided to take a break from the more serious stuff. A group of my friends also wanted to do something for fun, and so we organized a LAN Party. It lasted 2 full days and I really enjoyed it, even though I suck at almost any kind of computer game (at least the ones we played), with the exception of maybe sdlroids ;) (too bad there’s no multiplayer mode).

With 10 participants, we not only played games but also exchanged a lot of data. It was nice to see stuff flowing at 10 MB/s — a speed I virtually never see in the LAN owned by my ISP that I’m connected to at home.

We had both Linux and Windows machines in the network and everything worked without any significant problems. Gaming on Linux wasn’t troublesome or difficult. In fact, some Windows systems turned out to be harder to configure to make everything work.

The LAN Party began with two Gentoo machines, and ended with three, which I think is a good statistic :) A dual-boot system joined the Linux team after I helped a friend solve a little problem with his network card, which effectively finished his installation of a basic Gentoo system.

It was the first LAN party organized among my classmates, and I certainly hope that it won’t be the last one :)

splashutils-1.1.9.4 released

More than 24h ago I released splashutils-1.1.9.4. The new version includes many bugfixes and some minor speed enhancements. Have a look at the ChangeLog for a full list of changes. 1.1.9.4 was delayed by almost a week, all because of a speed-up idea I had. Instead of preparing things for a release, I was modifying the code and testing the performance of the splash daemon. The result? A 10% speed-up and the code turned into a mess. Since 10% doesn’t justify making splashutils even more difficult to maintain and potentially introducing new bugs, I decided to drop the idea for now. Fortunately, I’ve already come up with a solution that should allow me to make the code faster without making a mess of it. If all goes well, splashutils-1.1.9.5 will be 10% faster (not that anyone would actually notice that — it will translate into a 0.2s faster boot-up in common configurations..).

And speaking of 1.1.9.5, I thought I should elaborate on what’s in the development roadmap for the next few versions. 1.1.9.5 will be a bugfix-only release. I hope to get it out in a few days, provided that I’m able to fix all the important bugs in time. Whether 1.1.9.6 will be bugfix-only or not depends on how many unresolved bugs are left after 1.1.9.5 :) After that, I plan to get MNG support merged into the main source tree. This will make it possible to have animated themes, which sure will look nice if used properly. What will happen next is unknown — I’ll probably have a look at my old gensplash to-do lists and try to find things that haven’t already been implemented in splashutils-1.1.9.x.

The future of gensplash

For those who didn’t figure it out yet, let me break the news: my previous blog entry was an April Fool’s Day joke. The gensplash project is not dead, fbsplash is not going to be removed from gentoo-sources and neither fbsplash nor splashutils will be abandoned. In fact, the development is going quite well, and fbsplash+splashutils is gaining popularity, being used not only in Gentoo but also in a few other distros. The word is it will also be added to software-suspend 2. So the project is anything but dead :)

However, as people say, there is a grain of truth in every story and in every legend. And our case is not very different. While the gensplash project will not be discontinued, some things will have to be changed. Originally, the whole project was divided into two stages — stage 1 being fbsplash+splashutils and stage 2 — the final ‘gensplash’ program with all the fancy features. It was supposed to use the DirectFB library and XML config files. The way I see it now, neither of these was a good idea. DirectFB has proven to be difficult to use in the harsh conditions of the initial boot stages, and it increased the number of dependencies heavily. XML config files were something I came up with because I was too lazy to write a new config file parser from scratch ;)

As it turned out, it is possible to include virtually all gensplash features in splashutils, keeping the deps as minimal as possible and making the whole thing easy to manage. Support for icons and TTF fonts is already in place. So is support for the ‘daemon’ mode. Support for MNG animations will be coming soon. The current config file format seems to be flexible enough to handle new features as they get added to splashutils. The code handles various framebuffer formats quite well. And all this is possible without DirectFB, XML and other things that would make splashutils bloated.

So, to summarize, what is the meaning of all this? Well, there will most likely be no ‘gensplash’ app. The features that were planned for it will be added to splashutils, and care will be taken to keep the code as fast, lean and clean as possible. With the ‘release early, release often’ attitude that I have adopted now, splashutils 1.2.0 should be the equivalent of ‘gensplash stage 2′.

Is there a grain of truth in the statements about porting vesafb-tng to amd64/pcc and me helping the GeNToo project with the development of a graphics subsystem? Well, I’ll leave these two for people to figure out themselves :)

The gensplash project cancelled

After some 8 months without a single release of the “stage 2″ version of gensplash, the time has come to cancel the project. The probability of ever completing it has been getting lower and lower with every passing week. It was about a month ago that I realized that something has to be done. Struggling with various technical problems with the current, unreleased gensplash code, I decided that it would best to simply cancel the project.

Does it mean that Gentoo will be left without a ‘bootsplash’ solution? One of the reasons it took me a month to come to the final conclusion is that I always wanted to provide an alternative solution. Obviously, being left with fbsplash and splashutils is not an option — without the gensplash project kicking, they will probably soon be abandoned. One alternative was going back to bootsplash. While it doesn’t provide all the eye candy that I once promised to deliver, it has proven stable over the years and it’s still widely supported. Its code is fast, clean, and easy to debug. Having the JPG decoder in the kernel, which has once been thought of as a disadvantage, seems to be a good solution. Neither developers, nor users seem to mind adding a few bytes to the kernel image. After almost making the decision to switch back to bootsplash, another idea came to my mind. I recalled a project called ‘rhgb’, which is a ‘bootsplash’ solution used in Fedora and Red Hat systems. After doing some initial tests and considering both its advantages and disadvantages, I came to the conslusion that this is the perfect solution for Gentoo. True, it might lengthen the boot process by some 10s or so, and it won’t make it possible to display images during early boot (ie. when the kernel is still initializing its subsystems), but these are a low price to pay for the new features. I’m sure no one will mind watching a black screen for a little longer during the initial boot stages. With rhgb, we will be starting an X server somewhere during the ’sysinit’ runlevel. Thanks to the capabilities of X and GTK, it will be possible to use rhgb to handle the
verbose mode. This means that fbsplash will be removed from the newer gentoo
kernels.

Thanks to the flexible splash initscripts introduced a few months ago, the move to rhgb should be complete within as little as one week. Please stay tuned for more updates.

The precious development time saved thanks to this move will be used for more interesting projects, such as porting vesafb-tng to amd64 and ppc. I am also looking forward to working on graphics support for the GeNToo project. If all goes well, rhgb will be supported in GeNToo as well.

splashutils-1.1.9.3 released

I’ve just released splashutils-1.1.9.3, hopefully the most stable version of the 1.1.9.x series :) It includes the usual amount of bugfixes, some enhancements (among which is better compile-time configurability; the ebuilds don’t take advantage of it yet, sorry) and some new features (font stuff — font styles, text positioning, variables evaluation, etc).

This is also the first 1.1.9.x release that has a full, up-to-date documentation. The previous versions lacked some descriptions of the new features. If you want to design a cool new theme which will use some of them, now might be a good time to start the work :)

Speaking of documentation, I’ve also written a new document describing the splash initscripts used in Gentoo. This was requested by several people maintaining support for splashutils in other distros, but I guess Gentoo users might find the new doc interesting as well, should they ever want to learn how the pretty boot-time stuff works on the inside.

If you want to help with testing, please do echo ">=media-gfx/splashutils-1.0" >> /etc/portage/package.unmask and re-merge splashutils. Suggestions and bug reports are appreciated.

The splash and breaking initscripts

Yesterday I spent a good few hours on chasing a bug related to the new splashutils. For quite some time my qmail and svscan have been misbehaving on my system. svscan seemed to start properly, but when I logged in, the process was dead and had to be manually restarted. After that everything worked fine. It turns out this kind of problems is caused by adding CONSOLE=/dev/tty1 to the kernel command line. So far I have discovered two services which cause trouble — hddtemp and svscan. I’ve opened bugs for both of them (#86550, #86562) and added info about the issue to the gensplash troubleshooting section on my devsite. Hopefully this will save others from the trouble I had with fixing this problem.

Changes in splashutils

As you might have noticed, some two weeks ago I released a new version of splashutils (1.1.9), which introduces many new features and some rather large changes in the way the splash works.

There have been a few factors that pushed me towards this release. First of all, people seem to like it when stuff is moved from the kernel into the userspace. Thus, starting with fbsplash-0.9.2, what little silent mode code there was in the kernel, has been removed. The truth is, the silent mode can easily be handled by userspace, with no help from the kernel. The verbose mode on the other hand can’t be moved to the userspace, unless one is willing to write a complete userspace console. That’s why fbsplash is there to stay, at least for some time. The silent image is now displayed on its own tty. This allows the user to switch back and forth between verbose and silent mode. It also automatically solves problems with consolefont, provided it doesn’t touch the silent tty.

Second thing that made me realize that some changes are necessary is a simple test I have performed. Try doing for ((i=0;i<65535;i=$i+1000)) do splash_util -c paint -m s -t livecd-2005.0 --progress=$i ; done. It’s slow! A lot slower than I suspected it would be. It turns out that loading and unpacking the background picture every time it is to be repainted isn’t such a good idea, even if the whole process is repeated only a few times during boot. Fortunately, a solution has existed long before the problem was discovered. It’s called the ’splash daemon’ and was originally planned for gensplash. The splash daemon code has now been integrated into splashutils. The new mode of operation allows us to avoid the constant reloading and unpacking that was taking place with previous versions of splashutils.

The third reason for introducing the changes was a feeling that gensplash is moving far too slow. I started the project more than half a year ago and since then there hasn’t been a single release. Sure, there is some progress as far as the code is concerned but it’s still nowhere near a final version. Taking splashutils as a base, I was able to implement many of the features planned for gensplash in just a few days. At least now it will be possible to test them and see them in action.

I realize most of these new features is undocumented yet, and thus hard to take advantage of, but this will be taken care of soon. I plan to include full documentation in the next release of splashutils.

Physics workshop

During the last week (Mar, 14th-18th) I took part in a physics workshop organized by the Polish Children’s Fund (don’t let the name fool you!) as part of its “educational enrichment and support programme”. The workshop took place at the Institute of Theoretical Physics of the Warsaw University.

There were three people in my working group (Marcin Bieda, Marcin Grzybowski and myself) in which we were working on “simulating the effects of large masses within the framework of general relativity” under the guidance of Mikołaj Korzyński.

For me, the workshop was a lot of fun and a source of new knowledge and experiences. The result of our 5-day work is a program that can simulate the optical effects in the vicinity of a massive, sperically symmetric and non-rotating body (we’re using the Schwarzchild metric). At this point, the code needs some clean-ups and possibly some new features as well (support for relativistic aberration and gravitational redshift, rendering directly to a X window, generating animations, etc). As soon as the cleanups are complete, I’ll publish the code in a new science-related section of my devsite. I’ve been planning on creating such a section for quite some time and I guess this program, along with a bunch of other ones, will be a good way of starting it.

The workshop has been my first chance to use general relativity in a practical project. I have been interested in the topic for quite some time before, but I never really got into applying it to anything. Now that I have a program to play with, I hope exploring this subject will be easier and more fun.

I’m currently reading “A First Course in General Relativity” (Bernard F. Schutz), “Gravitation” (Kip S. Thorne, et al) and I’m waiting to get my hands on “Relativistic Astrophysics” (Marek Demiański). If you happen to know any other good textbooks on general relativity, I would be grateful for sharing information about them with me.

Making Gentoo boot even faster

Gentoo has supposedly one of the fastest boot-ups in the Linux world. I say “supposedly”, as at the moment I don’t really have anything I could compare it with and I have to refer to what has been posted in bug #69579. As fast and as advanced as our rc-scripts might be, there is always some room for improvement.

I have been looking through the Gentoo Bugzilla, searching for bugs about boot speed improvement. Fortunately there is the meta-bug mentioned in the previous paragraph which makes things easier to track. I haven’t had much time to play with the various modifications proposed there, but I’ve done some preliminary analyses and tests.

As you can see in comment #25, I don’t have much faith in running rc-scripts in parallel. Not that this not an useful feature - it certainly is, especially when you’re doing things like waiting for DHCP-provided network configuration or mounting some network filesystems. But, on my system running these scripts in parallel saves me not much more than 1 second. And it breaks the cool new boot icons I’ve been coding into splashutils for the past few days. The odds are definitely against setting RC_PARALLEL_STARTUP to “yes” :)

As far as the other changes are concerned, I’m not going to comment most of them just yet, I still need to do some tests. While I doubt the speed-up introduced by them will be dramatic, I think it’s worthwhile to integrate any optimizations that don’t break anything. After all, a large number of small speed-ups might very well result in a noticeably faster boot.

However, what I’m really looking for is a relatively huge speed increase (I’m talking 100% or more here). These things, if available, usually come at a price. Well, I would be willing to pay that price, even if getting the speed-up would mean resorting to some dirty tricks and hacks. What got my attention in these matters is the idea of using software suspend to make things faster.

The original idea proposed in one of the comments of our meta-bug was to make a snapshot of the system state at a late stage of the boot process and then use it to resume the system after reboot. While it sounds quite easy, it’s in fact pretty tricky on the technical side. The main problem I can see at this point is getting the in-kernel data structs in sync with the filesystems and thus not breaking anything and not sending any data from the hdd straight to /dev/null. I have yet to think whether this is even practically possible (comments, anyone?).

Since rewriting half of the software suspend code to make this new ideas work is not what I originally had in mind, I began looking for another possible solutions. What I have come up with so far, is making a snapshot of the system instead the normal shutdown sequence, with some potential “cleaning up” (ie. killing processes we wouldn’t want to run after we “boot up” with our snapshot) before doing so. This idea appears to work - I have already written some code and successfully tested it a few times. The speed improvement is huge - huge enough to make me notice how slow waiting for the BIOS to finish the POST is. My normal boot sequence takes some 48 seconds (from GRUB to the login prompt). With the new software suspend modifications in place, it’s down to 5-7 secs. I call that a promising result :)

The code still needs MUCH MORE testing and extending, but I think it’s something worth spending time on. Some may argue that this is cheating - we are not going through the normal boot sequence after all. On one hand they are right, but on the other - if the end result is the same, who cares what is really going on? ;)