Netboot image creation for SPARC machines

If your SPARC machine has a dead CD/DVD ROM drive or no CD/DVD ROM drive at all or if, for various reasons, you wish to build a netboot image from scratch the following paragraphs should help you.

Concepts

Basically, a netboot image is the sum of :

• Some bootstrap code (loader)
• An uncompressed kernel image
• An optional ramdisk / initrd image.

When the machine is powered-on, a Power On Self Test (POST) of the machine is done then, if nothing has been found defective Open Boot checks what the variable boot-device look like and take appropriate actions. If the machine has been set to boot via the network several situations can happen :

• if Open Boot has no WAN boot (Open Boot prior 4.17.1 ?) support and no DHCP support, a RARP request is emitted on the network to get an IP address from a RARP server then the machine starts a TFTP session on the machine who provided the IP and request to download a file named after the provided IP address (e.g. 192.168.4.101 => C0A80465).

• if Open Boot has been set to boot from the network using a DHCP request (boot-device is set to net:dhcp or boot net:dhcp has been entered at the Open Boot prompt), the machine will grab its network parameters using a DHCP request then starts to download the file according the DHCP response (filename and TFTP server), or by default adopt the same behavior than above.

• if Open Boot has been setup to perform a WAN boot, it uses what is contained in the variable network-boot-arguments to determine what to download from which location and with what networking parameters.

Once the netboot image has been downloaded, Open Boot transfers the execution the netboot image loader, which, at its turn will transfer to the Linux kernel which will run /sbin/init (or whatever else specified through the kernel command line). On most of the Funtoo machines in the world, /sbin/init will launch Open RC at the end do various things.

Short story:

OpenBoot -> Netboot loader -> Linux kernel -> /sbin/init -> Open RC -> Login on a tty (agetty+login) or whatever else suitable

The process is very similar to what happens when booting the machine from its hard drive, nothing magic here. The form varies, but the philosophy remains the same.

Limitations

According to various sources on the internet and experience returns, it looks like that :

• sun4u machines (e.g. UltraSPARC I/II/III/IV servers and workstations) do not initialize the processor MMU and thus, this requires the whole netboot image to fit within 4 megabytes. What happens if this rule is enforced? Simple: as the buffer where the netboot image is loaded into acts as a 4 megabytes ring-buffer, any byte exceeding the limit will overwrite the previously netboot image loaded data leading to the so famous "Fast Data Access MMU Miss" a couple of seconds after the machine started to download the netboot image.

• sun4v machines (e.g. UltraSPARC T1 and later) initialize their MMU and thus, are not subject to the above limitation. Pay attention to the startup messages when you power on the machine you will see something like "Init MMU init" (assuming you are connected to the virtual console from ALOM):

...
0:0>VBSC selecting POST MIN Testing.
0:0>POST enabling threads: ffffffff
0:0>VBSC setting verbosity level 2
0:0>Start Selftest.....
0:0>Master CPU Tests Basic....Done
0:0>Init MMU.....
...

Fortunately for us : SPARC machines relies on a very small set of hardware from an model to another (modulo extension cards).
Unfortunately for us : 2.6 kernels, tends to be big even with an aggressive strip down of features and aggressive size optimizations.

Producing the netboot image

Kernel configuration

It is highly suggested to use modules as much as possible. Of course don't forget to:

• pull in the correct modules and firmware to handle your network adapter :-)
• Activate initrd/initramfs support
• Put the filesystem used for your RAM disk built-in (suggested: ext2, do not use cramfs unless you want a readonly filesystem).
• Activate RAM Disk support (1 RAM disk is sufficient, for the size increase the suggested 4 Mbytes to at least 32 Mbytes)

Newer kernels (e.g. 2.6.38) also have the ability to handle LZMA 2 compressed RAM disk images (check "Embedded System" in "General Preferences").

Preparing a ramdisk image

You have to strategies :

1. An initramfs image which is, for the record, a compressed cpio archive of a live filesystem (the way presented here)
2. A minimalistic disk image created through dd and a loopback device

Structure creation

First, you need to create a bare minimal file system structure in a working directory of you choice, although /tmp/initramfs-image is used here, anything left at of your choice can be used. You will need :

• /bin
• /dev
• /dev/pts
• /etc
• /sbin
• /lib
• /usr/bin
• /usr/sbin
• /tmp

~ # cd /tmp
tmp # mkdir initramfs-image
tmp # cd initramfs-image
initramfs-image # mkdir -p bin dev dev/pts etc sbin usr/bin usr/sbin tmp 

Second, copy your /etc/passwd (shadow password are used, so only the logins are stored here) and /etc/groups :

initramfs-image # cp /etc/passwd /etc/group etc

Although it not mandatory required to copy those two files, it will, however help BusyBox' mdev.

Mandatory shared object files coming from /lib

In this example dynamically linked binaries are used so the shared objects files their refer to must be copied. As of writing you must copy the following files from /lib to the lib directory located in your working space

• libc-2.11.2.so
• libc.so.6 (symlink to libc-2.11.2.so)
• ld-2.11.2.so
• ld-linux.so.2 (symlink to ld-2.11.2.so)
• libblkid.so.1.1.0
• libblkid.so.1 (symlink to libblkid.so.1.1.0)

tmp # cd /lib
lib # cp libc-2.11.2.so libc-2.11.2.so ld-2.11.2.so ld-linux.so.2 libblkid.so.1.1.0 ..... /tmp/initramfs-image/lib

You can safely ignore libpam and libnss (if you don't copy the later, you will have to refer the remote hosts by their IP addresses only as libnss handles names resolution facilities). libpam will be unused because BusyBox won't be compiled with Pluggable Authentication Modules support.

Configuring and emerging BusyBox

BusyBox is a Swiss Knife that regroup in a single compact binary more than 30 well-known utilities (grep, ls, fdisk....). BusyBox does not mimic 100% of the original tools functionalities but supports the most used ones, as such it fits will memory constrained environments such as embedded systems or everywhere else where every byte of code loaded in memory matters. As such it fits very well to a netboot image.

A recommended configuration for BusyBox can be found in the talk section of this page. Just copy the recommended settings in /etc/portage/savedconfig/sys-apps/busybox-1.18.3 (assuming sys-apps/busybox-1.18.3 is the version available on your system).A few discussionw on various points of suggested BusyBox configuration:

• No "Standalone shell" (CONFIG_FEATURE_SH_STANDALONE): if this option would be set, BusyBox will give the priority to the internal applets having the same name than the external commands (standalone scripts or binaries) you would add to your minimalist environment. A way to circumvent would be to launch the external commands with their absolute path but it is not a convenient way to use from a final user point of view.

• No support for ext2 utilities: As of writing, the BusyBox ext2 filesystem creation applet has an unwanted behaviour that makes it overwrite the first 1024 bytes of a partition. Creating an ext2 filesystem on the first partition of a hard drive starting at cylinder 0 (required if you want to boot your machine from it) with this applet would simply overwrite the Sun disklabel on the drive, thus killing the partition table on it.

• No support for ReiserFS: As of writing, the BusyBox ReiserFS is experimental.

• No support for fdisk: As of writing, the BusyBox fdisk applet is unable to generate proper Sun disklabels, thus making OpenBoot protest if you try too make to boot from the hard drive.

• No support for daemons (like http, telnetd, dhcpd, ftpd, ntpd, etc) : this is a client only environment.

Adding extra mandatory libs

If you add extra binaries to your netboot RAM disk, your are likely due to copy extra shared objects comming for /lib (and/or /usr/lib). To know the dependencies of your binary program, just run ldd on it :

# ldd /sbin/fdisk
        libblkid.so.1 => /lib/libblkid.so.1 (0xf7f68000)
        libc.so.6 => /lib/libc.so.6 (0xf7de8000)
        libuuid.so.1 => /lib/libuuid.so.1 (0xf7dd0000)
        /lib/ld-linux.so.2 (0xf7fa4000)

If you see the output "VDSO" do not search for the file existence, VDSO stands for Virtual Dynamic Shared Object (this virtual shared object is provided by the Linux kernel itself through the remapping of some its code in the userland).

Creating the initramfs image

Just create a cpio archive and compress it (make sure you have support for xz archive in the kernel) :

tmp # cd initramfs-image
initramfs-image # cpio * > ../root.img
tmp # xz --check=none --lzma2=dict=1MiB root.img
tmp # cp root.img.xz /usr/src/root.img

If you use the XZ archive format (this one has the best compression ratio), you must use :

• A CRC32 or no CRC at all (like here)
• A reasonable dictionary size (kernel documentation talks about a dictionary having no more than 1 Mbyte in size, see Documentation/xz.txt).

In all cases you must also :

• Use root.img as the final filename
• /usr/src as location to put root.img

At this point the initramfs image is ready for the final step.

Gluing things together

The root.img is seated, the kernel is rebuilt, now the most exciting part begins: generate the final netboot image. You have several options to get the job being done:

1. Use the command tilo which comes with sys-boot/silo
2. Use the facilities provided with the Linux kernel source code

At date of writing (February 2011) tilo has a bug that makes one the internal commands it uses to segfault when building over-sized images (> 4 Mbytes), so let's use the Linux kernel black magic. Simply :

• Go in your /usr/src/linux (Again: simply go there, do not change anything or rebuild a brand new kernel)
• Do the following incantation :

# make tftpboot.img

Various things will be checked and after a couple of seconds a file named tftpboot.img is produced in arch/sparc/boot.

What to do next?

Simply copy the freshly produced tftpboot.img in the right directory of your TFTP server and, if you use symlinks to refer the image, make sure they points on the right file.

That's it!

Strategy suggestion

As netboot images are required to be as compact as possible a 3 phases strategy is suggested:

1. Build a minimalist netboot (you barely need your network card driver, an IP stack, plus the required filesystems for step 2)
2. This minimalist netboot will grab another ramdisk image (using NFS for example), eventually unpack it and put it in the VFS (using Union FS for example) which include everything you need like kernel modules you IDE/SCSI/SAS controllers or additional filesystems
3. Once everything is in place, just exec what you need (i.e. Open RC, a set of homebrew scripts...)