Netboot image creation for SPARC machines

If your SPARC machine has a dead CD/DVD ROM drive or no CD/DVD ROM drive at all or if, for various reasons, you wish to build a netboot image from scratch the following paragraphs should help you.

Concepts

Basically, a netboot image is the sum of :

• Some bootstrap code (loader)
• An uncompressed kernel image
• An optional ramdisk / initrd image.

When the machine is powered-on, a Power On Self Test (POST) of the machine is done then, if nothing has been found defective Open Boot checks what the variable boot-device look like and take appropriate actions. If the machine has been set to boot via the network several situations can happen :

• if Open Boot has no WAN boot (Open Boot prior 4.17.1 ?) support and no DHCP support, a RARP request is emitted on the network to get an IP address from a RARP server then the machine starts a TFTP session on the machine who provided the IP and request to download a file named after the provided IP address (e.g. 192.168.4.101 => C0A80465).

• if Open Boot has been set to boot from the network using a DHCP request (boot-device is set to net:dhcp or boot net:dhcp has been entered at the Open Boot prompt), the machine will grab its network parameters using a DHCP request then starts to download the file according the DHCP response (filename and TFTP server), or by default adopt the same behavior than above.

• if Open Boot has been setup to perform a WAN boot, it uses what is contained in the variable network-boot-arguments to determine what to download from which location and with what networking parameters.

Once the netboot image has been downloaded, Open Boot transfers the execution the netboot image loader, which, at its turn will transfer to the Linux kernel which will run /sbin/init (or whatever else specified through the kernel command line). On most of the Funtoo machines in the world, /sbin/init will launch Open RC at the end do various things.

Short story:

OpenBoot -> Netboot loader -> Linux kernel -> /sbin/init -> Open RC -> Login on a tty (agetty+login) or whatever else suitable

The process is very similar to what happens when booting the machine from its hard drive, nothing magic here. The form varies, but the philosophy remains the same.

Limitations

According to various sources on the internet and experience returns, it looks like that :

• sun4u machines (e.g. UltraSPARC I/II/III/IV servers and workstations) do not initialize the processor MMU and thus, this requires the whole netboot image to fit within 4 megabytes. What happens if this rule is enforced? Simple: as the buffer where the netboot image is loaded into acts as a 4 megabytes ring-buffer, any byte exceeding the limit will overwrite the previously netboot image loaded data leading to the so famous "Fast Data Access MMU Miss" a couple of seconds after the machine started to download the netboot image.

• sun4v machines (e.g. UltraSPARC T1 and later) initialize their MMU and thus, are not subject to the above limitation. Pay attention to the startup messages when you power on the machine you will see something like "Init MMU init" (assuming you are connected to the virtual console from ALOM):

...
0:0>VBSC selecting POST MIN Testing.
0:0>POST enabling threads: ffffffff
0:0>VBSC setting verbosity level 2
0:0>Start Selftest.....
0:0>Master CPU Tests Basic....Done
0:0>Init MMU.....
...

Fortunately for us : SPARC machines relies on a very small set of hardware from an model to another (modulo extension cards).
Unfortunately for us : 2.6 kernels, tends to be big even with an aggressive strip down of features and aggressive size optimizations.

Producing the netboot image

Kernel configuration

It is highly suggested to use modules as much as possible. Of course don't forget to:

• pull in the correct modules and firmware to handle your network adapter :-)
• Activate initrd/initramfs support
• Put the filesystem used for your RAM disk built-in (suggested: ext2, do not use cramfs unless you want a readonly filesystem).
• Activate RAM Disk support (1 RAM disk is sufficient, for the size increase the suggested 4 Mbytes to at least 32 Mbytes)

Newer kernels (e.g. 2.6.38) also have the ability to handle LZMA 2 compressed RAM disk images (check "Embedded System" in "General Preferences").

Preparing a ramdisk image

You have to strategies :

1. An initramfs image which is, for the record, a compressed cpio archive of a live filesystem (the way presented here)
2. A minimalistic disk image created through dd and a loopback device

Structure creation

First, you need to create a bare minimal file system structure in a working directory of you choice, although /tmp is used here, anything left at of your choice can be used. You will need :

/bin /dev /dev/pts /etc /sbin /lib /usr/bin /usr/sbin /tmp

~# cd /tmp
tmp# mkdir -p bin dev dev/pts etc sbin usr/bin usr/sbin tmp 

Second, copy your /etc/passwd (shadow password are used, so only the logins are stored here) and /etc/groups :

tmp# cp /etc/passwd /etc/group etc

Although it not mandatory required to copy those two files, it will, however help BusyBox' mdev.

Mandatory shared object files

In this example dynamically linked binaries are used so the shared objects files their refer to must be copied. As of writing you must copy the following files from /lib to the lib directory located in your working space

1. libc-2.11.2.so
2. libc.so.6 (symlink to libc-2.11.2.so)
3. ld-2.11.2.so
4. ld-linux.so.2 (symlink to ld-2.11.2.so)
5. libblkid.so.1.1.0
6. libblkid.so.1 (symlink to libblkid.so.1.1.0)

tmp # cd /lib
lib # cp libc-2.11.2.so libc-2.11.2.so ld-2.11.2.so ld-linux.so.2 libblkid.so.1.1.0 ..... /tmp/lib

You can safely ignore libpam and libnss (if you don't copy the later, you will have to refer the remote hosts by their IP addresses only as libnss handles names resolution facilities). libpam will be unused because BusyBox won't be compiled with Pluggable Authentication Modules support.

Adding extra mandatory libs

If you add extra binaries to your netboot RAM disk, your are likely due to copy extra shared objects comming for /lib (and/or /usr/lib). To know the dependencies of your binary program, just run ldd on it :

# ldd /sbin/fdisk
        libblkid.so.1 => /lib/libblkid.so.1 (0xf7f68000)
        libc.so.6 => /lib/libc.so.6 (0xf7de8000)
        libuuid.so.1 => /lib/libuuid.so.1 (0xf7dd0000)
        /lib/ld-linux.so.2 (0xf7fa4000)

If you see the output "VDSO" do not search for the file existence, VDO stands for Virtual Dynamic Shared Object (this virtual shared object is provided by the Linux kernel itself through the remapping of some its code in the userland).

Gluing things together

The root.img is seated, the kernel is rebuilt, now the most exciting part begins. You have several options to cook the cake:

1. Use the command tilo which comes with sys-boot/silo
2. Use the facilities provided with the Linux kernel source code

At date of writing (February 2011) tilo has a bug that makes one the internal commands it uses to segfault when building over-sized images (> 4 Mbytes), so let's use the Linux kernel black magic. Simply :

• Go in your /usr/src/linux (Again: simply go there, do not change anything or rebuild a brand new kernel)
• Do the following incantation :

# make tftpboot.img

Various things will be checked and after a couple of seconds a file named tftpboot.img is produced in arch/sparc/boot.

What to do next?

Simply copy the freshly produced tftpboot.img in the right directory of your TFTP server and, if you use symlinks to refer the image, make sure they points on the right file.

That's it!

Strategy suggestion

As netboot images are required to be as compact as possible a 3 phases strategy is suggested:

1. Build a minimalist netboot (you barely need your network card driver, an IP stack, plus the required filesystems for step 2)
2. This minimalist netboot will grab another ramdisk image (using NFS for example), eventually unpack it and put it in the VFS (using Union FS for example) which include everything you need like kernel modules you IDE/SCSI/SAS controllers or additional filesystems
3. Once everything is in place, just exec what you need (i.e. Open RC, a set of homebrew scripts...)