Difference between pages "Funtoo Linux Kernels" and "Project Bootstrap"

(Difference between pages)
(Kernel Features and Stability)
 
(I wrote more on the ordering.)
 
Line 1: Line 1:
This Section will give you an overview of kernels used in funtoo.
+
Project Bootstrap seeks to create a minimal portage tree to better understand dependencies between those packages, create packages reflecting such dependencies, and eventually have a cleaner stage build.
  
Funtoo Linux provides a number of new kernels for your use. This is the official page for all Funtoo Linux kernel information.  
+
The repository for this project may be found at https://github.com/brantgurga/project-bootstrap.
  
Some points of interest:
+
= Dependencies =
 +
There are two kinds of dependencies that Portage currently supports. There is <code>RDEPEND</code> when specifies a runtime dependency. A runtime dependency can be merged after a package. For example, if foo.ebuild contains:
  
* Most Funtoo Linux kernels support the handy ''<code>[[#Binary USE|binary]]</code>'' USE flag, described below.
+
{{code|foo.ebuild|
* Funtoo Linux offers quality kernels from other Linux Distributions, like ''<code>ubuntu-server</code>'' and ''<code>debian-sources</code>''.
+
<pre>RDEPEND="baz/bar"</pre>}}
* A detailed [[#Kernel Features and Stability|Kernel Features and Stability]] table can be found below.
+
* Advanced users may want to take a look at [[Additional Kernel Resources]].
+
* There is a quick'n dirty howto to compile your own [[kernel]] with initramfs the funtoo way.
+
  
== Overview of Kernels ==
+
This means that Portage can use this order:
 +
# foo
 +
# bar
  
=== sysrescue-std-sources --> {{Package|sys-kernel/sysrescue-std-sources}} ===
+
On the other hand, <code>DEPEND</code> specifies a dependency that needs to exist beforehand. Consequently, if foo.ebuild contains:
  
This kernel is from the [http://www.sysresccd.org SystemRescueCD project], and is based on Fedora 14/15, plus some other patches related to booting from a live CD. It is a quality kernel, and is generally pretty stable. It is not suitable for production servers but is a good choice for Funtoo Linux desktops. Earlier,the [[Funtoo Linux Installation]] Guide recommended this kernel for general users, but now 'debian-sources' is recommended. Note however,  that by design all audio functions are removed from SystemRescue,  ie no sound when using this kernel.
+
{{code|foo.ebuild|
 +
<pre>DEPEND="baz/bar"</pre>}}
  
=== vanilla-sources --> {{Package|sys-kernel/vanilla-sources}} ===
+
This means that Portage must use the order:
 +
# bar
 +
# foo
  
This will install the "vanilla" (unmodified) Linux kernel sources. Current recommended version is 3.x. Funtoo Linux fully supports Linux 3.x. The advantages of this kernel include recent improvements to [[Linux Containers]], a very modern networking stack with lots of bug fixes, and high reliability for desktops and servers. The downside is that this kernel must be manually configured by the user and does not have built-in ''<code>genkernel</code>'' support via the ''<code>binary</code>'' USE flag at this time.
+
When we talk about bootstrapping, Portage is used with the <code>ROOT</code> variable to merge packages to a new root location. The problem is that something you need to build a package such as the compiler does not yet exist in the <code>ROOT</code> location. Consequently, you can't do something like:
  
=== gentoo-sources --> {{Package|sys-kernel/gentoo-sources}} ===
+
{{code|binutils.ebuild|
 +
<pre>DEPEND="sys-devel/gcc"</pre>}}
  
This kernel tree is based on stable kernels from [https://www.kernel.org/ kernel.org] with genpatches applied [http://dev.gentoo.org/~mpagano/genpatches/about.htm genpatches].
+
In order to build binutils, you need a compiler, but to build the compiler, at least for the x86 and x86-64 targets, you need binutils. So what do you do about that situation? Do you install binutils first (somehow). Or do you install gcc first (somehow)? I propose a two-part approach to handle this:
Gentoo patchset aims to support the entire range of Gentoo-supported architectures. List of available genpatched kernels: [http://dev.gentoo.org/~mpagano/genpatches/kernels.htm genpatches-kernels]
+
  
=== openvz-rhel6-stable --> {{Package|sys-kernel/openvz-rhel6-stable}} ===
+
== New Dependency Type ==
 +
The first aspect is having a new dependency type that Portage would support. When doing a <code>ROOT</code> merge which a normal merge can actually be considered a special case with <code>ROOT=/</code>, you care about dependencies in the <code>ROOT=/</code> environment. In the Autoconf terminology, these are <code>CHOST</code> dependencies. You also care about what is installed in the build environment. These are <code>CBUILD</code> dependencies to use the Autoconf terminology. The current <code>DEPEND</code> mechanism tries to address both which isn't really all that elegant. I therefore propose a <code>HOSTDEPEND</code> dependency type.
  
This is a RHEL6-based kernel with OpenVZ support. This kernel is now the preferred kernel for production OpenVZ deployments. It requires gcc-4.4.5 to build, which it will use automatically without the user needing to use ''<code>gcc-config</code>''. We use this version of gcc since this is the version of gcc used by Red Hat to build this kernel.
+
You actually also would have a <code>TARGET</code> dependency as well. For example, to build a compiler to target x86-64 on an x86 machine, you have a dependency on a binutils that can target x86-64. To build a package that is *hosted* in that *build* environment, you also have a dependency on a *hosted* compiler that *targets* that same *build* architecture. However, it is my thought that it is unnecessary to add a new dependency type for these, and the USE flags or a different package *might* address those types of dependencies. I have not experimented enough to know for sure though.
  
=== openvz-rhel5-stable --> {{Package|sys-kernel/openvz-rhel5-stable}} ===
+
== New Package Type ==
 +
Assuming host dependencies and build environment dependencies are addressed, there is now the actual bootstrapping problem. The end goal is to have a final product that has no influence from the host environment. For example, it has already been noted that gcc has a dependency on binutils. However, if you build binutils first, it gets linked with object files from the host environment. To address situation, I propose that we create a chain of toolchain packages such as:
 +
# bootstrap-binutils: binutils built with host tools
 +
# bootstrap-gcc: gcc built with host tools and utilizing bootstrap-binutils
 +
# binutils: binutils built with bootstrap-gcc so that it does not use the object files of the host environment
 +
# gcc: gcc built with bootstrap-gcc and utilizing binutils
  
This kernel is based on the latest Red Hat Enterprise Linux 5.6 kernel, and contains additional OpenVZ (virtual containers) patches from the [[OpenVZ on Funtoo Linux|OpenVZ]] project. It is a very stable and reliable kernel, and is recommended for use in production environments. The only major downside to this kernel is that it is based on Linux 2.6.18 -- some parts of the kernel are out-of-date, and it is not compatible with modern versions of udev. However, it is pretty trivial to downgrade udev to an earlier version on Funtoo Linux and this kernel has a track-record of being rock-solid. When stability is paramount, you put up with the udev downgrade, use this kernel, and can enjoy hundreds of days of uptime. For more information on how to use this kernel with Funtoo Linux, see the [[RHEL5 Kernel HOWTO]].
+
So bootstrap-binutils depends on the host environment, but binutils only depends on the build environment.
  
=== ubuntu-server --> {{Package|sys-kernel/ubuntu-server}} ===
+
= Toolchain =
 +
The basic toolchain consists of gcc and minimal dependencies. The simplest of the dependencies is the linux-headers package which is necessary for glibc. Glibc is probably a more complicated one since it can supply shared libraries as well as the mechanism for loading those shared libraries.
  
This is the kernel from Ubuntu Server. Version ''<code>2.6.32.32.62</code>'' is the same version used in Ubuntu Server 10.04 LTS, and version ''<code>2.6.35.28.50</code>'' is the one used in Ubuntu Server 10.10 (currently masked). In our testing of ''<code>2.6.32.32.62</code>'', it has been very reliable and offers very good performance. One exception, which is common among 2.6.32-based kernels, is that it's recommended that you emerge {{Package|net-misc/broadcom-netxtreme2}} if you have any Broadcom-based NICs, as the in-kernel drivers have compatibility issues with certain models. This kernel is a very good option if you want a relatively modern server kernel and do not need [[OpenVZ]] support. We use gcc-4.4.5 to build this kernel. It will use gcc-4.4.5 automatically, without requiring the user to use ''<code>gcc-config</code>''.
+
== Headers ==
 +
Most packages use libc routines such as <code>malloc</code> to allocate memory. However, libc needs to get that memory from somewhere. This is where the userspace interface supplied by the Linux kernel comes in. Libc uses interfaces described by the headers to implement the backend functionality of libc in different OS environments. This is how the same libc API can work on Linux as well as Hurd. glibc is doing the work of interfacing with the operating system kernel on the program's behalf so that most programs don't have to think about those differences if they are sticking with the standard C library.
  
=== debian-sources --> {{Package|sys-kernel/debian-sources}} ===
+
It used to be that glibc would get build using the headers in /usr/src/linux/include directly. However, the kernel folks were against this for various reasons. It then fell to distributions to create packages of sanitized headers with which to compile glibc. The problem here was different distributions doing different things and no standardized mechanism. Nowadays, within the kernel source, there is a mechanism to get the sanitized userspace headers installed through some variation of <tt>make headers_install</tt>. The nice thing is that since these are just header files, they don't depend on the host environment for object files. Consequently, these only need to be installed to the new ROOT once. They don't really need to be bootstrapped. Whether installed using host tools or ROOT tools, the headers are the same.
  
This is the Debian kernel. '''These ebuilds now support the ''<code>binary</code>'' USE flag.''' Daniel has added a special <tt>config-extract</tt> command which can be used to list all available official Debian kernel configurations, and generate them from the Debian files included with the kernel. This kernel has optional [[OpenVZ]] support, but it is much better to use <tt>openvz-rhel6-stable</tt> if you want a production-quality OpenVZ installation. For more information about how to use <tt>debian-sources</tt> and <tt>config-extract</tt>, see [[#Using Debian-Sources with Genkernel|Using debian-sources with Genkernel]] below.
+
== Glibc ==
 +
Glibc is a strange beast. Software written in C comes in two types. There is hosted C and unhosted C. Unhosted C is basically anything that doesn't use the standard C library. Hosted C is software that does use a C library. Things like the kernel use unhosted C since there is no libc available. The glibc package is interesting though because it provides libc so that is build in an unhosted environment. Kind of silly to link libc to libc, if that's even possible. However glibc also supplies programs like nscd as well as libraries for name resolution. So the libc portion of glibc doesn't have any build environment dependencies besides the headers. However, the extra glibc portions depend on the compiler's object files that provide the connection to a hosted environment. I am unsure at the moment if glibc does any magic to address this or if glibc is something that needs built at least twice to remove the influence of the host environment.
  
=== debian-sources-lts --> {{Package|sys-kernel/debian-sources-lts}} ===
+
== Binutils ==
 +
Binutils supplies the assemblers and linkers that gcc uses. It depends on gcc to be compiled though. It also depends on glibc to provide libc.
  
This is the Debian long-term stable kernel. '''These ebuilds now support the <tt>binary</tt> USE flag.''' Daniel has added a special <tt>config-extract</tt> command which can be used to list all available official Debian kernel configurations, and generate them from the Debian files included with the kernel.
+
== GCC ==
 +
GCC is interesting in that it had a bootstrapping built in, though this bootstrapping is unavailable is cross-compiling. It's unclear if the whole thing is rebuilt during that bootstrapping or just a certain part. While gcc supports many languages, for the purpose of bootstrapping a stage, handling C is all that matters.
  
== Binary USE ==
+
= Bootstrap Order =
 +
The goal of the bootstrap ordering is to eliminate build dependencies such as include files and linked libraries from being sought on the host's ROOT.
 +
# Headers
 +
# Glibc (depends on headers)
 +
# Binutils (depends on glibc)
 +
# GMP (depends on glibc)
 +
# MPFR (depends on gmp and glibc)
 +
# MPC (depends on glibc, gmp, and mpfr)
 +
# C-only GCC (depends on binutils, mpc, mpfr, mpc and Glibc)
  
Many of the kernel ebuilds in Funtoo Linux support the very useful <code>binary</code> USE flag. By enabling this USE flag and emerging the kernel, the ebuild will automatically build a binary kernel image, initramfs and kernel modules and install them to <code>/boot</code>. The binary kernel image and initramfs can be used to boot your Funtoo Linux system without requiring any additional configuration. This is a great way to get a Funtoo Linux system up and running quickly. Here's how to do it:
+
== Assumptions ==
 +
It is assumed that the host's tools work properly. The bootstrap toolchain ordering does not attempt to work around host tool issues. The host tools will need to be fixed if there are issues as a result of them in this phase. It does not scale well to try to do something like building binutils then glibc using the built binutils, then rebuilding binutils against glibc. As long as the host tools are working well enough, the influence of host tools is removed once the packages are rebuilt within the chroot environment.
  
<console>
+
== Further Thinking ==
###i## echo "sys-kernel/openvz-rhel5-stable binary" >> /etc/portage/package.use
+
In addition to the core toolchain, the tools necessary for operating Portage in a chroot environment need to be necessary. Consideration should also be made for running the test suites since it is critical to ensure that the core toolchain, Portage and its dependencies, and anything necessary for the chroot environment are working properly.
###i## emerge openvz-rhel5-stable
+
###i## nano -w /etc/boot.conf
+
###i## boot-update
+
</console>
+
  
More information can be found in the [[Funtoo Linux Installation]] Guide.
+
[[Category:Projects]]
 
+
== Funtoo Linux Genkernel ==
+
 
+
Funtoo Linux contains a forked/enhanced version of genkernel with the following new capabilities:
+
 
+
* genkernel can use a build directory that is separate from the kernel source directory. This is enabled using the new <tt>--build-dst</tt> option.
+
* <code>--build-src</code> is a new option that is equivalent to the <tt>--kerneldir</tt> option.
+
* <code>--fullname</code> can be used to specify the entire name of the kernel and initramfs images -- everything after <tt>kernel-</tt> and <tt>initramfs-</tt>.
+
* <code>--firmware-src</code> - a new option that works identically to <tt>--firmware-dir</tt>.
+
* <code>--firmware-dst</code> - a new capability - you can now define where genkernel installs firmware.
+
* Genkernel uses Funtoo Linux <code>lvm2</code> rather than building its own.
+
* Some compile fixes.
+
 
+
== Kernel Features and Stability Overview ==
+
 
+
{{Fancywarning|'''SPARC64''': All kernels beyond 3.9 series and before 3.14-rc8 are subject to a [http://www.spinics.net/lists/sparclinux/msg11805.html bug] that stalls the kernel on sun4v machines '''only'''. Those latter are machines provided with UltraSPARC T1 and later CPUs (e.g. SunFire T1000, SunFire T2000, SunFire T52x0/T54x0 series...), all sun4u machines (UltraSPARC IV and prior CPUs) are not subject to this problem and any kernel version is functional. }}
+
 
+
 
+
{| {{table}}
+
!Kernel Name
+
!Version
+
!USE flags
+
!Stability
+
!Extra Features
+
!Req'd udev
+
!Notes
+
|-
+
|<tt>{{Package|sys-kernel/vanilla-sources}}</tt>
+
|3.13.1
+
|N/A
+
|'''Excellent''' - recommended for desktops and servers.
+
|N/A
+
|Any
+
|Recommended for modern networking stack, hardware and [[Linux Containers]] support. This kernel must be manually configured by the user. New Features: [http://kernelnewbies.org/Linux_3.12 kernelnewbies.org/linux_3.12]  New Drivers: [http://kernelnewbies.org/Linux_3.12-DriversArch kernelnewbies/Linux_3.12-DriversArch]
+
|-
+
|<tt>{{Package|sys-kernel/gentoo-sources}}</tt>
+
|3.13.1
+
|N/A
+
|'''Excellent''' - recommended for desktops and workstations
+
|N/A
+
|Any
+
|Recommended for modern networking stack, hardware and [[Linux Containers]] support. This kernel must be manually configured by the user. New Features: [http://kernelnewbies.org/Linux_3.12 kernelnewbies.org/linux_3.12]  New Drivers: [http://kernelnewbies.org/Linux_3.12-DriversArch kernelnewbies/Linux_3.12-DriversArch]
+
|-
+
|<tt>{{Package|sys-kernel/sysrescue-std-sources}}</tt>
+
|3.0.21.302
+
|<tt>binary</tt>
+
|''Good'' - recommended for desktops
+
|N/A
+
|Any
+
|Nvidia card users: binary use flag installs nouveau drivers. Not compatible with nvidia-drivers.
+
|-
+
|<tt>{{Package|sys-kernel/openvz-rhel6-stable}}</tt>
+
|2.6.32.042.079.5
+
|<tt>binary</tt>
+
|'''Excellent''' - recommended for production servers
+
|N/A
+
|Any
+
|This kernel is built with gcc-4.4.5. <tt>emerge broadcom-netxtreme2</tt> for reliable BCM5709+ support (integrated NIC)
+
|-
+
|<tt>{{Package|sys-kernel/openvz-rhel5-stable}}</tt>
+
|2.6.18.028.095.1
+
|<tt>binary</tt>
+
|'''Excellent''' - recommended for production servers
+
|OpenVZ
+
|=sys-fs/udev-146*
+
|Broadcom <tt>bnx2</tt> driver module bundled with kernel appears to be OK. This kernel is built with gcc-4.1.2. Enabling the <tt>binary</tt> USE flag will cause gcc-4.1.2 to be emerged and used for building the kernel.
+
|-
+
|<tt>{{Package|sys-kernel/ubuntu-server}}</tt>
+
|2.6.32.32.62
+
|<tt>binary</tt>
+
|'''Excellent''' - recommended for production servers (still in extended testing)
+
| N/A
+
|Any
+
|This kernel is built with gcc-4.4.5. <tt>emerge broadcom-netxtreme2</tt> for reliable BCM5709+ support (integrated NIC)
+
|-
+
|<tt>{{Package|sys-kernel/ubuntu-server}}</tt>
+
|2.6.35.28.50
+
|<tt>binary</tt>
+
|''not yet tested''
+
| N/A
+
|Any
+
|This kernel is built with gcc-4.4.5. <tt>emerge broadcom-netxtreme2</tt> for reliable BCM5709+ support (integrated NIC)
+
|-
+
|<tt>{{Package|sys-kernel/debian-sources}}</tt>
+
|3.12.3
+
|<tt>openvz</tt>
+
|''Good'' - default kernel recommended by Funtoo
+
|OpenVZ (optional)
+
|Any
+
|See [[#Using debian-sources with Genkernel]], below.
+
|-
+
|}
+
 
+
== Using Debian-Sources with Genkernel ==
+
 
+
{{ fancyimportant|Debian-sources is now fully compatible with the ''binary'' USE flag and recommended for desktop users. The below example is valid for manual installation. At least 12G of ''/var/tmp'' required to build }}
+
 
+
 
+
This section describes how to build a binary kernel with ''<code>debian-sources</code>'' and ''<code>genkernel</code>'', and it also explains how to use Funtoo Linux's ''<code>config-extract</code>'' tool to list and create official Debian kernel configurations.
+
 
+
=== First step: emerging the required packages ===
+
 
+
The first step is to emerge:
+
 
+
# The Debian sources
+
# Genkernel itself
+
 
+
This is achieved by running the following:
+
 
+
<console>
+
###i## emerge -av sys-kernel/debian-sources sys-kernel/genkernel
+
</console>
+
 
+
Once the Debian kernel sources are deployed, you should find a directory named '''linux-debian-''version''''' (e.g. linux-debian-2.6.32.30) under '''<code>/usr/src</code>'''. Update your the '''<code>linux</code>''' symlink to point on this directory:
+
<console>
+
###i## cd /usr/src
+
###i## rm linux
+
###i## ln -s linux-debian-2.6.32.30 linux
+
</console>
+
 
+
Alternatively, emerge the debian-sources with the ''<code>symlink</code>'' USE flag.
+
 
+
=== Second step: Grabbing a configuration file ===
+
 
+
If is now time to download the kernel configuration file. For this tutorial we will use a configuration file for AMD64 (several others architectures like MIPS or SPARC64 are available.)  To view a complete list of available kernel configurations, type <code>./config-extract -l</code> '''in the Debian kernel source directory''':
+
 
+
<pre>
+
ninja1 linux-debian-2.6.32.30 # ./config-extract -l
+
 
+
====== standard featureset ======
+
 
+
      alpha: alpha-generic, alpha-legacy, alpha-smp
+
      amd64
+
      armel: iop32x, ixp4xx, kirkwood, orion5x, versatile
+
        hppa: parisc, parisc-smp, parisc64, parisc64-smp
+
        i386: 486, 686, 686-bigmem, amd64
+
        ia64: itanium, mckinley
+
        m68k: amiga, atari, bvme6000, mac, mvme147, mvme16x
+
        mips: 4kc-malta, 5kc-malta, r4k-ip22, r5k-ip32, sb1-bcm91250a, sb1a-bcm91480b
+
      mipsel: 4kc-malta, 5kc-malta, r5k-cobalt, sb1-bcm91250a, sb1a-bcm91480b
+
    powerpc: powerpc, powerpc-smp, powerpc64
+
        s390: s390x, s390x-tape
+
        sh4: sh7751r, sh7785lcr
+
      sparc: sparc64, sparc64-smp
+
    sparc64: sparc64, sparc64-smp
+
 
+
====== vserver featureset ======
+
 
+
      amd64
+
        i386: 686, 686-bigmem
+
        ia64: itanium, mckinley
+
    powerpc: powerpc, powerpc64
+
        s390
+
      sparc
+
    sparc64
+
 
+
====== xen featureset ======
+
 
+
      amd64
+
        i386
+
 
+
====== openvz featureset ======
+
 
+
      amd64
+
        i386
+
</pre>
+
 
+
Type <tt>config-extract -h</tt> for extended usage information:
+
 
+
<pre>
+
ninja1 linux-debian-2.6.32.30 # ./config-extract -h
+
This work is free software.
+
 
+
Copyright 2011 Funtoo Technologies. You can redistribute and/or modify it under
+
the terms of the GNU General Public License version 3 as published by the Free
+
Software Foundation. Alternatively you may (at your option) use any other
+
license that has been publicly approved for use with this program by Funtoo
+
Technologies (or its successors, if any.)
+
 
+
usage: config-extract [options] arch [featureset] [subarch]
+
 
+
  -h  --help        print this usage and exit
+
  -l  --list        list all available kernel configurations
+
  -o  --outfile    specify kernel config outfile --
+
                    defaults to .config in current directory
+
  [featureset]      defaults to "none" if not specified
+
  [subarch]        defaults to the only one available; otherwise required
+
 
+
This program was written by Daniel Robbins for Funtoo Linux, for the purpose of
+
easily and conveniently extracting Debian kernel configurations. To see a nice
+
list of all available kernel configurations, use the --list option.
+
 
+
Debian's kernel configs are specified internally in arch_featureset_flavor
+
format, such as: "amd64_openvz_amd64". The featureset typically describes an
+
optional kernel configuration such as "xen" or "openvz", while the flavor in
+
Debian terminology typically refers to the sub-architecture of the CPU.
+
 
+
When using this command, you must specify an arch. A featureset of "none" is
+
assumed unless you specify one, and by default this program will pick the only
+
available subarch if there is only one to choose from. If not, you will need to
+
pick one (and the program will remind you to do this.)
+
 
+
The kernel configuration will be written to ".config" in the current directory,
+
or the location you specified using the -o/--outfile option.
+
</pre>
+
 
+
Let's use <tt>config-extract</tt> to create a kernel configuration for an amd64 system:
+
 
+
<console>
+
# ##i##cd linux
+
# ##i##./config-extract amd64
+
Wrote amd64_none_amd64 kernel configuration to /usr/src/linux-debian-2.6.32.30/.config.
+
</console>
+
 
+
<tt>config-extract</tt> also allows you to extract special Debian featuresets, such as settings for Xen and [[OpenVZ]] kernels:
+
 
+
<console>
+
# ##i##./config-extract amd64 openvz
+
Wrote amd64_openvz_amd64 kernel configuration to /usr/src/linux-debian-2.6.32.30/.config.
+
</console>
+
 
+
'''It is necessary to name the kernel configuration file something other than ".config" to avoid errors with genkernel.'''
+
 
+
 
+
After using <tt>config-extract</tt>, run <tt>make oldconfig</tt> and accept all default options by hitting Enter at all prompts.
+
 
+
 
+
{{fancynote|if you are using the XFS file system as your root partition: Run <tt>make menuconfig</tt> and ensure that "File Systems --> XFS filesystem support" and "Library Routines --> CRC32c (Castagnoli, et al) Cyclic Redundancy-Check" are both set to * (and not [m]).}}
+
This is needed to ensure that your system can boot up correctly for kernel versions >= 3.10.11.
+
 
+
=== Third step: Building and installing the kernel ===
+
 
+
This is simply achieved by:
+
 
+
<console>
+
# ##i##genkernel --kernel-config=config-2.6.32-5-amd64 all
+
</console>
+
 
+
* --kernel-config: use the given configfile. If you only give a filename here, it is searched for in your current working dir. You can also use a relative or an absolute path leading to your configfile here (for example: "--kernel-config=/usr/src/linux/configfile").
+
* all: rebuild the kernel image and the initramfs ramdisk image (aside of kernel modules, the ramdisk image contains tools such as BusyBox and some generic startup scripts, depending on options you use on the command line several additional tools like lvm or raid volume management can be incorporated as well).
+
 
+
{{ fancyimportant|Unless explicitly stated via ''--no-clean'' or ''--no-mrproper'', Genkernel will do a '''make mrproper''' in the kernel source tree, thus cleaning a previous build '''and removing the previous kernel configuration file''' in it.
+
}}
+
 
+
If you use Genkernel to rebuild a Linux kernel on SPARC64, remember to either:
+
* Set '''sparc64-unknown-linux-gnu-''' in ''General setup --> Cross-compiler tool prefix''
+
* Put '''--kernel-cross-compile=sparc64-unknown-linux-gnu-''' on the Genkernel command line
+
 
+
Once the kernel has been compiled and the ram disk has been generated, the kernel image plus its companion files (initramfs image and System.map) are placed in the /boot directory. You can use your favourite tool to update your bootloader configuration files.
+
 
+
[[Category:Internals]]
+
[[Category:Funtoo features]]
+
[[Category:Kernel]]
+

Latest revision as of 03:04, 15 December 2010

Project Bootstrap seeks to create a minimal portage tree to better understand dependencies between those packages, create packages reflecting such dependencies, and eventually have a cleaner stage build.

The repository for this project may be found at https://github.com/brantgurga/project-bootstrap.

Dependencies

There are two kinds of dependencies that Portage currently supports. There is RDEPEND when specifies a runtime dependency. A runtime dependency can be merged after a package. For example, if foo.ebuild contains:

Code: foo.ebuild
RDEPEND="baz/bar"

This means that Portage can use this order:

  1. foo
  2. bar

On the other hand, DEPEND specifies a dependency that needs to exist beforehand. Consequently, if foo.ebuild contains:

Code: foo.ebuild
DEPEND="baz/bar"

This means that Portage must use the order:

  1. bar
  2. foo

When we talk about bootstrapping, Portage is used with the ROOT variable to merge packages to a new root location. The problem is that something you need to build a package such as the compiler does not yet exist in the ROOT location. Consequently, you can't do something like:

Code: binutils.ebuild
DEPEND="sys-devel/gcc"

In order to build binutils, you need a compiler, but to build the compiler, at least for the x86 and x86-64 targets, you need binutils. So what do you do about that situation? Do you install binutils first (somehow). Or do you install gcc first (somehow)? I propose a two-part approach to handle this:

New Dependency Type

The first aspect is having a new dependency type that Portage would support. When doing a ROOT merge which a normal merge can actually be considered a special case with ROOT=/, you care about dependencies in the ROOT=/ environment. In the Autoconf terminology, these are CHOST dependencies. You also care about what is installed in the build environment. These are CBUILD dependencies to use the Autoconf terminology. The current DEPEND mechanism tries to address both which isn't really all that elegant. I therefore propose a HOSTDEPEND dependency type.

You actually also would have a TARGET dependency as well. For example, to build a compiler to target x86-64 on an x86 machine, you have a dependency on a binutils that can target x86-64. To build a package that is *hosted* in that *build* environment, you also have a dependency on a *hosted* compiler that *targets* that same *build* architecture. However, it is my thought that it is unnecessary to add a new dependency type for these, and the USE flags or a different package *might* address those types of dependencies. I have not experimented enough to know for sure though.

New Package Type

Assuming host dependencies and build environment dependencies are addressed, there is now the actual bootstrapping problem. The end goal is to have a final product that has no influence from the host environment. For example, it has already been noted that gcc has a dependency on binutils. However, if you build binutils first, it gets linked with object files from the host environment. To address situation, I propose that we create a chain of toolchain packages such as:

  1. bootstrap-binutils: binutils built with host tools
  2. bootstrap-gcc: gcc built with host tools and utilizing bootstrap-binutils
  3. binutils: binutils built with bootstrap-gcc so that it does not use the object files of the host environment
  4. gcc: gcc built with bootstrap-gcc and utilizing binutils

So bootstrap-binutils depends on the host environment, but binutils only depends on the build environment.

Toolchain

The basic toolchain consists of gcc and minimal dependencies. The simplest of the dependencies is the linux-headers package which is necessary for glibc. Glibc is probably a more complicated one since it can supply shared libraries as well as the mechanism for loading those shared libraries.

Headers

Most packages use libc routines such as malloc to allocate memory. However, libc needs to get that memory from somewhere. This is where the userspace interface supplied by the Linux kernel comes in. Libc uses interfaces described by the headers to implement the backend functionality of libc in different OS environments. This is how the same libc API can work on Linux as well as Hurd. glibc is doing the work of interfacing with the operating system kernel on the program's behalf so that most programs don't have to think about those differences if they are sticking with the standard C library.

It used to be that glibc would get build using the headers in /usr/src/linux/include directly. However, the kernel folks were against this for various reasons. It then fell to distributions to create packages of sanitized headers with which to compile glibc. The problem here was different distributions doing different things and no standardized mechanism. Nowadays, within the kernel source, there is a mechanism to get the sanitized userspace headers installed through some variation of make headers_install. The nice thing is that since these are just header files, they don't depend on the host environment for object files. Consequently, these only need to be installed to the new ROOT once. They don't really need to be bootstrapped. Whether installed using host tools or ROOT tools, the headers are the same.

Glibc

Glibc is a strange beast. Software written in C comes in two types. There is hosted C and unhosted C. Unhosted C is basically anything that doesn't use the standard C library. Hosted C is software that does use a C library. Things like the kernel use unhosted C since there is no libc available. The glibc package is interesting though because it provides libc so that is build in an unhosted environment. Kind of silly to link libc to libc, if that's even possible. However glibc also supplies programs like nscd as well as libraries for name resolution. So the libc portion of glibc doesn't have any build environment dependencies besides the headers. However, the extra glibc portions depend on the compiler's object files that provide the connection to a hosted environment. I am unsure at the moment if glibc does any magic to address this or if glibc is something that needs built at least twice to remove the influence of the host environment.

Binutils

Binutils supplies the assemblers and linkers that gcc uses. It depends on gcc to be compiled though. It also depends on glibc to provide libc.

GCC

GCC is interesting in that it had a bootstrapping built in, though this bootstrapping is unavailable is cross-compiling. It's unclear if the whole thing is rebuilt during that bootstrapping or just a certain part. While gcc supports many languages, for the purpose of bootstrapping a stage, handling C is all that matters.

Bootstrap Order

The goal of the bootstrap ordering is to eliminate build dependencies such as include files and linked libraries from being sought on the host's ROOT.

  1. Headers
  2. Glibc (depends on headers)
  3. Binutils (depends on glibc)
  4. GMP (depends on glibc)
  5. MPFR (depends on gmp and glibc)
  6. MPC (depends on glibc, gmp, and mpfr)
  7. C-only GCC (depends on binutils, mpc, mpfr, mpc and Glibc)

Assumptions

It is assumed that the host's tools work properly. The bootstrap toolchain ordering does not attempt to work around host tool issues. The host tools will need to be fixed if there are issues as a result of them in this phase. It does not scale well to try to do something like building binutils then glibc using the built binutils, then rebuilding binutils against glibc. As long as the host tools are working well enough, the influence of host tools is removed once the packages are rebuilt within the chroot environment.

Further Thinking

In addition to the core toolchain, the tools necessary for operating Portage in a chroot environment need to be necessary. Consideration should also be made for running the test suites since it is critical to ensure that the core toolchain, Portage and its dependencies, and anything necessary for the chroot environment are working properly.