Difference between revisions of "Building a Kernel from Source"

m (kernel sources)
m
(6 intermediate revisions by the same user not shown)
Line 4: Line 4:
 
We are showing here how an intermediate Linux user can use an alternative to the standard beginners "genkernel" approach,  to compile a custom kernel,  in a relatively still speedy and easy set up.
 
We are showing here how an intermediate Linux user can use an alternative to the standard beginners "genkernel" approach,  to compile a custom kernel,  in a relatively still speedy and easy set up.
  
'''minimum requirements'''
+
=== Minimum Requirements ===
 
You should understand the way how things work in a terminal and how to use an editor and tweak config files. This is crucial.
 
You should understand the way how things work in a terminal and how to use an editor and tweak config files. This is crucial.
 
You don't need much knowledge about the linux kernel and it's internals. Nevertheless, you have to know at least where the files are located, how they are used and what is the file logic behind the overall structure. Otherwise you should consider using a non source based linux distribution.
 
You don't need much knowledge about the linux kernel and it's internals. Nevertheless, you have to know at least where the files are located, how they are used and what is the file logic behind the overall structure. Otherwise you should consider using a non source based linux distribution.
 
If you are scared now, don't worry - we are going to build a kernel the Funtoo way and you will pick up everthing necessary to accomplish this challenge, step by step, so the next time you do it yourself and become a real Funtoo knight!
 
If you are scared now, don't worry - we are going to build a kernel the Funtoo way and you will pick up everthing necessary to accomplish this challenge, step by step, so the next time you do it yourself and become a real Funtoo knight!
  
'''assumptions'''
+
=== Assumptions ===
 
You start from an installed Funtoo system on the disk, or at least, you are on stage3 in a chrooted environment from a live cd, following somehow the Funto [[Installation (Tutorial)|Installation Tutorial]].
 
You start from an installed Funtoo system on the disk, or at least, you are on stage3 in a chrooted environment from a live cd, following somehow the Funto [[Installation (Tutorial)|Installation Tutorial]].
  
Line 27: Line 27:
 
From there you should have a running system booting nicely from your own build (just little bit bloated) kernel. This is more than you can expect from any other ready to go distribution.
 
From there you should have a running system booting nicely from your own build (just little bit bloated) kernel. This is more than you can expect from any other ready to go distribution.
  
'''Attention'''
+
{{Note}} We are using RedHat's dracut in order to build a nice initramfs (containing all the necessary tools and extra drivers our kernel might need to start the system). Although dracut is the way to go, more sophisticated and not as buggy as gentoo's genkernel approach, more and more funtoo geeks start using slashbeast's better-initramfs, which we will cover at the end of this howto! So after having set up a genkernel from debian or gentoo sources we are going to build a kernel with either (or both) dracut or/and better-initramfs. So gentoo sources with genkernel is always my backup if anything is not working correctly on my system. For the slightly more geeky approach with my own initram I am using pf-sources, ck-sources or any other more or less heavily patched sources.
We are using RedHat's dracut in order to build a nice initramfs (containing all the necessary tools and extra drivers our kernel might need to start the system).
+
 
+
Although dracut is the way to go, more sophisticated and not as buggy as gentoo's genkernel approach, more and more funtoo geeks start using slashbeast's better-initramfs, which we will cover at the end of this howto!
+
 
+
So after having set up a genkernel from debian or gentoo sources we are going to build a kernel with either (or both) dracut or/and better-initramfs.
+
So gentoo sources with genkernel is always my backup if anything is not working correctly on my system. For the slightly more geeky approach with my own initram I am using pf-sources, ck-sources or any other more or less heavily patched sources.
+
  
 
Let's go!
 
Let's go!
Line 57: Line 51:
 
I am going to use the sys-kernel/pf-sources now, as I already had the gentoo-sources installed.
 
I am going to use the sys-kernel/pf-sources now, as I already had the gentoo-sources installed.
  
== prerequisites ==
+
== Prerequisites ==
  
 
I don't know which tools you have already installed, so some information here might be redundant.
 
I don't know which tools you have already installed, so some information here might be redundant.
 
It doesn't harm to just copy and paste and do some steps again.
 
It doesn't harm to just copy and paste and do some steps again.
  
First, we look into our /etc/make.conf:
+
First, we look into our <code>/etc/make.conf</code>:
  
<pre>
+
<console>
 +
###i## nano /etc/make.conf
 
#These compiler flags are just tweaking (optimazation) and NOT necessary:
 
#These compiler flags are just tweaking (optimazation) and NOT necessary:
 
CFLAGS="-O2 -pipe -march=native -ftracer -fforce-addr"
 
CFLAGS="-O2 -pipe -march=native -ftracer -fforce-addr"
 
CXXFLAGS="${CFLAGS} -fpermissive -fomit-frame-pointer"
 
CXXFLAGS="${CFLAGS} -fpermissive -fomit-frame-pointer"
 
 
KDIR=/usr/src/linux
 
KDIR=/usr/src/linux
 
 
KERNEL="symlink build"
 
KERNEL="symlink build"
 
USE="$KERNEL ....here are your use flags...."
 
USE="$KERNEL ....here are your use flags...."
 
## These modules are available:
 
## These modules are available:
 
## DRACUT_MODULES="dracut_modules_biosdevname dracut_modules_btrfs dracut_modules_caps dracut_modules_crypt dracut_modules_crypt-gpg dracut_modules_dmraid dracut_modules_dmsquash-live dracut_modules_gensplash dracut_modules_iscsi dracut_modules_livenet dracut_modules_lvm dracut_modules_mdraid dracut_modules_multipath dracut_modules_nbd dracut_modules_nfs dracut_modules_plymouth dracut_modules_ssh-client dracut_modules_syslog"
 
## DRACUT_MODULES="dracut_modules_biosdevname dracut_modules_btrfs dracut_modules_caps dracut_modules_crypt dracut_modules_crypt-gpg dracut_modules_dmraid dracut_modules_dmsquash-live dracut_modules_gensplash dracut_modules_iscsi dracut_modules_livenet dracut_modules_lvm dracut_modules_mdraid dracut_modules_multipath dracut_modules_nbd dracut_modules_nfs dracut_modules_plymouth dracut_modules_ssh-client dracut_modules_syslog"
## We are using this ones for LVM / LUKS
+
## We will use these modules for LVM / LUKS:
 
DRACUT_MODULES="crypt lvm plymouth biosdevname dmraid crypt-gpg dmsquash-live ssh-client syslog"
 
DRACUT_MODULES="crypt lvm plymouth biosdevname dmraid crypt-gpg dmsquash-live ssh-client syslog"
</pre>
+
</console>
  
Now we are setting the package keywords:
+
Next, we set the package keywords:
<pre>
+
<console>
 
/etc/portage/package.use/dracut:
 
/etc/portage/package.use/dracut:
<console>
 
 
sys-kernel/dracut dm net device-mapper crypt lvm
 
sys-kernel/dracut dm net device-mapper crypt lvm
 
</console>
 
</console>
</pre>
+
 
If you don't have lvm over encrypted LUKS you probably just add the "net" keyword here, or "selinux".
+
{{Note}} If you don't have lvm over encrypted LUKS you probably just add the "net" keyword here, or "selinux".
  
  
 
After that we are going to build our packages:
 
After that we are going to build our packages:
 
<console>
 
<console>
# ##i##emerge -av app-portage/gentoolkit sys-kernel/pf-sources sys-kernel/dracut sys-boot/plymouth sys-boot/plymouth-openrc-plugin
+
###i## emerge -av app-portage/gentoolkit sys-kernel/pf-sources sys-kernel/dracut sys-boot/plymouth sys-boot/plymouth-openrc-plugin
 
</console>
 
</console>
  
 
Finished? Well, then let's go on and
 
Finished? Well, then let's go on and
  
== preparing the kernel ==
+
== Preparing the kernel ==
  
 
We go now to the sources directory and enter the following commands to update the kernel's  .config  file:
 
We go now to the sources directory and enter the following commands to update the kernel's  .config  file:
 
<console>
 
<console>
# ##i##cd /usr/src/linux/
+
###i## cd /usr/src/linux/
# ##i##make clean
+
###i## make clean
 
   CLEAN  .
 
   CLEAN  .
 
   CLEAN  arch/x86/kernel/acpi/realmode
 
   CLEAN  arch/x86/kernel/acpi/realmode
Line 124: Line 116:
 
   CLEAN  .tmp_versions
 
   CLEAN  .tmp_versions
 
   CLEAN  vmlinux System.map .tmp_kallsyms2.S .tmp_kallsyms1.o .tmp_kallsyms2.o .tmp_kallsyms1.S .tmp_vmlinux1 .tmp_vmlinux2 .tmp_System.map
 
   CLEAN  vmlinux System.map .tmp_kallsyms2.S .tmp_kallsyms1.o .tmp_kallsyms2.o .tmp_kallsyms1.S .tmp_vmlinux1 .tmp_vmlinux2 .tmp_System.map
# ##i##zcat /proc/config.gz > /usr/src/linux/.config
+
###i## zcat /proc/config.gz > /usr/src/linux/.config
 
</console>
 
</console>
  
 
<console>
 
<console>
# ##i##make localmodconfig
+
###i## make localmodconfig
 
</console>
 
</console>
  
Line 177: Line 169:
 
= Building the Kernel =
 
= Building the Kernel =
 
<console>
 
<console>
# ##i##make -j8  bzImage
+
##i### make -j8  bzImage
# ##i##make -j8 modules
+
##i### make -j8 modules
# ##i##make modules_install
+
##i### make modules_install
# ##i##make install
+
##i### make install
 
</console>
 
</console>
  

Revision as of 17:25, January 9, 2014

Setting up a proper kernel yourself - lean, mean and tailored to your hardware, is the challenge by which a linux user can graduate to becoming a Funtoo knight ;-)

Even though many of us are using enterprise-ready kernels in datacenters, there is almost nobody who hasn't at least considered building a kernel for his laptop / PC. We are showing here how an intermediate Linux user can use an alternative to the standard beginners "genkernel" approach, to compile a custom kernel, in a relatively still speedy and easy set up.

Minimum Requirements

You should understand the way how things work in a terminal and how to use an editor and tweak config files. This is crucial. You don't need much knowledge about the linux kernel and it's internals. Nevertheless, you have to know at least where the files are located, how they are used and what is the file logic behind the overall structure. Otherwise you should consider using a non source based linux distribution. If you are scared now, don't worry - we are going to build a kernel the Funtoo way and you will pick up everthing necessary to accomplish this challenge, step by step, so the next time you do it yourself and become a real Funtoo knight!

Assumptions

You start from an installed Funtoo system on the disk, or at least, you are on stage3 in a chrooted environment from a live cd, following somehow the Funto Installation Tutorial.

In this case we are building a kernel that is booting root in LVM over encrypted LUKS container. If you don't have this setup, don't worry, you just don't need all the modules, but everything else is similar.


Getting everything in order to start

First there is the decision which linux kernel sources we need. There are plenty of them in the repositories around, often it is not easy to distinguish between them.

I would always trust my distribution of choice and take what is has to offer - and funtoo has a lot to offer!

I really do recommend (especially if it is your first time) to build a debian-sourced genkernel like described in chapter 5 "Using Debian-Sources with Genkernel" in the Funtoo Kernels Tutorial.

From there you should have a running system booting nicely from your own build (just little bit bloated) kernel. This is more than you can expect from any other ready to go distribution.

Note

{{{1}}}

We are using RedHat's dracut in order to build a nice initramfs (containing all the necessary tools and extra drivers our kernel might need to start the system). Although dracut is the way to go, more sophisticated and not as buggy as gentoo's genkernel approach, more and more funtoo geeks start using slashbeast's better-initramfs, which we will cover at the end of this howto! So after having set up a genkernel from debian or gentoo sources we are going to build a kernel with either (or both) dracut or/and better-initramfs. So gentoo sources with genkernel is always my backup if anything is not working correctly on my system. For the slightly more geeky approach with my own initram I am using pf-sources, ck-sources or any other more or less heavily patched sources.

Let's go!

Kernel Sources

We are going to use the kernel sources from the funtoo git repository.

The source you use on your system is up to you and your needs. For a laptop or desktop system, we recommend the following:

  • sys-kernel/pf-sources
  • sys-kernel/ck-sources
  • sys-kernel/gentoo-sources
  • sys-kernel/git-sources
  • sys-kernel/sysrescue-std-sources
  • sys-kernel/debian-sources

Please, have a look in the ebuild description, look onto their homepage and take the one that suits you best! If you are unsure for now, use sys-kernel/gentoo-sources. That's always a safe bet for a general system.

It is not a problem to have various kernels installed parallel, so go on with any one of them.

I am going to use the sys-kernel/pf-sources now, as I already had the gentoo-sources installed.

Prerequisites

I don't know which tools you have already installed, so some information here might be redundant. It doesn't harm to just copy and paste and do some steps again.

First, we look into our /etc/make.conf:

# nano /etc/make.conf
#These compiler flags are just tweaking (optimazation) and NOT necessary:
CFLAGS="-O2 -pipe -march=native -ftracer -fforce-addr"
CXXFLAGS="${CFLAGS} -fpermissive -fomit-frame-pointer"
KDIR=/usr/src/linux
KERNEL="symlink build"
USE="$KERNEL ....here are your use flags...."
## These modules are available:
## DRACUT_MODULES="dracut_modules_biosdevname dracut_modules_btrfs dracut_modules_caps dracut_modules_crypt dracut_modules_crypt-gpg dracut_modules_dmraid dracut_modules_dmsquash-live dracut_modules_gensplash dracut_modules_iscsi dracut_modules_livenet dracut_modules_lvm dracut_modules_mdraid dracut_modules_multipath dracut_modules_nbd dracut_modules_nfs dracut_modules_plymouth dracut_modules_ssh-client dracut_modules_syslog"
## We will use these modules for LVM / LUKS:
DRACUT_MODULES="crypt lvm plymouth biosdevname dmraid crypt-gpg dmsquash-live ssh-client syslog"

Next, we set the package keywords:

/etc/portage/package.use/dracut:
sys-kernel/dracut dm net device-mapper crypt lvm

Note

{{{1}}}

If you don't have lvm over encrypted LUKS you probably just add the "net" keyword here, or "selinux".


After that we are going to build our packages:

# emerge -av app-portage/gentoolkit sys-kernel/pf-sources sys-kernel/dracut sys-boot/plymouth sys-boot/plymouth-openrc-plugin

Finished? Well, then let's go on and

Preparing the kernel

We go now to the sources directory and enter the following commands to update the kernel's .config file:

# cd /usr/src/linux/
# make clean
  CLEAN   .
  CLEAN   arch/x86/kernel/acpi/realmode
  CLEAN   arch/x86/kernel/cpu
  CLEAN   arch/x86/kernel
  CLEAN   arch/x86/vdso
  CLEAN   arch/x86/lib
  CLEAN   drivers/gpu/drm/radeon
  CLEAN   drivers/net/wan
  CLEAN   drivers/scsi/aic7xxx
  CLEAN   drivers/tty/vt
  CLEAN   drivers/video/logo
  CLEAN   firmware
  CLEAN   kernel
  CLEAN   lib/raid6
  CLEAN   lib
  CLEAN   security/apparmor
  CLEAN   security/selinux
  CLEAN   usr
  CLEAN   arch/x86/boot/compressed
  CLEAN   arch/x86/boot
  CLEAN   .tmp_versions
  CLEAN   vmlinux System.map .tmp_kallsyms2.S .tmp_kallsyms1.o .tmp_kallsyms2.o .tmp_kallsyms1.S .tmp_vmlinux1 .tmp_vmlinux2 .tmp_System.map
# zcat /proc/config.gz > /usr/src/linux/.config
# make localmodconfig

You will get some questions which you can answer mostly with either M (compiled as a module) or Y (compiled directly into the kernel).

Enable different security models (SECURITY) [Y/n/?] y
Enable the securityfs filesystem (SECURITYFS) [Y/?] y
Socket and Networking Security Hooks (SECURITY_NETWORK) [Y/?] y
Security hooks for pathname based access control (SECURITY_PATH) [Y/?] y
Low address space for LSM to protect from user allocation (LSM_MMAP_MIN_ADDR) [65536] 65536
NSA SELinux Support (SECURITY_SELINUX) [Y/n/?] y
  NSA SELinux boot parameter (SECURITY_SELINUX_BOOTPARAM) [N/y/?] n
  NSA SELinux runtime disable (SECURITY_SELINUX_DISABLE) [N/y/?] n
  NSA SELinux Development Support (SECURITY_SELINUX_DEVELOP) [Y/n/?] y
  NSA SELinux AVC Statistics (SECURITY_SELINUX_AVC_STATS) [Y/n/?] y
  NSA SELinux checkreqprot default value (SECURITY_SELINUX_CHECKREQPROT_VALUE) [1] 1
  NSA SELinux maximum supported policy format version (SECURITY_SELINUX_POLICYDB_VERSION_MAX) [Y/n/?] y
    NSA SELinux maximum supported policy format version value (SECURITY_SELINUX_POLICYDB_VERSION_MAX_VALUE) [19] 19
TOMOYO Linux Support (SECURITY_TOMOYO) [Y/n/?] y
  Default maximal count for learning mode (SECURITY_TOMOYO_MAX_ACCEPT_ENTRY) [2048] 2048
  Default maximal count for audit log (SECURITY_TOMOYO_MAX_AUDIT_LOG) [1024] 1024
  Activate without calling userspace policy loader. (SECURITY_TOMOYO_OMIT_USERSPACE_LOADER) [Y/n/?] y
AppArmor support (SECURITY_APPARMOR) [Y/n/?] y
  AppArmor boot parameter default value (SECURITY_APPARMOR_BOOTPARAM_VALUE) [1] 1
Integrity Measurement Architecture(IMA) (IMA) [Y/n/?] y
EVM support (EVM) [N/y/?] (NEW)
Default security module
  1. SELinux (DEFAULT_SECURITY_SELINUX)
  2. TOMOYO (DEFAULT_SECURITY_TOMOYO)
  3. AppArmor (DEFAULT_SECURITY_APPARMOR)
> 4. Unix Discretionary Access Controls (DEFAULT_SECURITY_DAC)
choice[1-4?]: 4
warning: (ACPI_HOTPLUG_CPU) selects ACPI_CONTAINER which has unmet direct dependencies (ACPI && EXPERIMENTAL)
warning: (MEDIA_TUNER) selects MEDIA_TUNER_TEA5761 which has unmet direct dependencies (MEDIA_SUPPORT && VIDEO_MEDIA && I2C && EXPERIMENTAL)
#
# configuration written to .config
#
warning: (GFS2_FS) selects DLM which has unmet direct dependencies (EXPERIMENTAL && INET && SYSFS && CONFIGFS_FS && (IPV6 || IPV6=n))
warning: (IMA) selects TCG_TPM which has unmet direct dependencies (HAS_IOMEM && EXPERIMENTAL)
warning: (MEDIA_TUNER) selects MEDIA_TUNER_TEA5761 which has unmet direct dependencies (MEDIA_SUPPORT && VIDEO_MEDIA && I2C && EXPERIMENTAL)
warning: (ACPI_HOTPLUG_CPU) selects ACPI_CONTAINER which has unmet direct dependencies (ACPI && EXPERIMENTAL)
root@[~src/linux] #

Now comes the most adventurous part!

Building the Kernel

# make -j8  bzImage
# make -j8 modules
# make modules_install
# make install


Building an initramfs or not?

The reason to build a kernel with an initramfs is mostly for interoperability (e.g. live-cd's) and special features like an included busybox, ssh, etc. But mostly, and that's why we are doing this here now, to have a proper kernel up and running quick'n dirty in a reasonable time without fighting hours and days until a more or less exotic hardware is perfectly run by the kernel. After having a proper basic kernel running with the help of an initramfs, I really recommend you to go a step further and build a true kernel with all features includes without an initramfs. But this could be pain in the ass and very time consuming - so we do it the funtoo way here - at least in the second example when we stick to better-initramfs instead of Red-Hat's dracut.

Option one: Initrd with dracut

To build the initrd we just execute

# dracut -f --fstab --xz /boot/initramfs-3.2.6-pf.img  3.2.6-pf

Generally, this really should be enough! If you experience booting problems like missing modules / drivers then just boot from the genkernel section and fix the initrd building. You can look into the man page to tweak the command a bit (e.g. --add-drivers "xz dm_crypt" etc...).

Ok let's go on and finish the taks, we are going to tell now grub how to boot off correctly!

root@[~src/linux] # nano /etc/boot.conf

boot {
        generate grub
        default "Funtoo Linux dracut"
        timeout 3
}

"Funtoo Linux genkernel" {
        kernel kernel-genkernel[-v]
        initrd initramfs-genkernel[-v]
        params = quiet rootfstype=ext4
        params += luks enc_root=/dev/sda3
        params += lvm root=/dev/mapper/vg-root
}

"Funtoo Linux dracut" {
        kernel vmlinuz[-v]
## this is the better-initramfs generated initrd
        initrd initramfs[-v].img
        params  = quiet rootfstype=ext4
        params += luks enc_root=/dev/sda3
        params += lvm root=/dev/mapper/vg-root
}

That's it almost!

Now write to the grub.cfg with the new handy boot-update script from funtoo:

root@[~src/linux] # boot-update -v
root@[~src/linux] # sudo boot-update -v

 boot-update 1.5.2 / Copyright 2009-2011 Funtoo Technologies

 [use option "-l" for license info, "-h" for help]

 * Generating config for grub...

 DEFAULT > Funtoo Linux - vmlinuz-3.2.6-pf
           Funtoo Linux genkernel - kernel-genkernel-x86_64-3.2.6-pf

 * Completed successfully.

Okay,... here you go..! :)

Reboot and see how it works!

Option two: using better-initramfs

Piotr's better-initramfs is another approach that is tiny, nice and shiny and seems to become more and more a favourite among funtoo'ers. The biggest plus is that, once built it is kernel version independant.

For using this you just do the following steps:

1. download sources
2. build kernel with "make bzImage"
3. download better-initramfs
4. run better-initramfs
5. adjust /etc/boot.conf

Here is how in detail:

Assuming you did install already a genkernel backup or at least you have a working bzImage + modules installed, we rush forward to step 3:

download better-initramfs

➜ # cd /usr/src/;
➜ # git clone https://github.com/slashbeast/better-initramfs.git

➜ # /src #  cd better-initramfs
➜ # better-initramfs git:(master) ls
AUTHORS    LICENSE   README.rst  bootstrap  output   sourceroot
ChangeLog  Makefile  TODO        examples   scripts

build the better-initramfs

➜ # better-initramfs git:(master) sudo bootstrap/bootstrap-all
Passwort: xxx 
# from here go and grab a coffee
➜ # sudo make prepare
➜ # sudo make image
➜ # sudo mv output/initramfs.cpio.gz /boot

adjust grub

Taking the above setup we edit the /etc/boot.conf as I installed genkernel first, and dracut after - you see this setup:

boot {
        generate grub
        default "Funtoo Linux"
        timeout 3
}

# Rootfs over lvm over luks
# /dev/sda3 - encrypted lvm's pv
# /dev/mapper/vg-root - rootfs's lv

"Funtoo Linux" {
        kernel bzImage[-v]
## this is the better-initramfs generated initrd
        initrd initramfs.cpio.gz
        params  = quiet rootfstype=ext4
        params += luks enc_root=/dev/sda3
        params += lvm root=/dev/mapper/vg-root
}

"Funtoo Linux dracut" {
        kernel vmlinuz[-v]
## this is the dracut generated initrd
        initrd initramfs[-v].img
        params  = quiet rootfstype=ext4
        params += luks enc_root=/dev/sda3
        params += lvm root=/dev/mapper/vg-root
}


"Funtoo Linux genkernel" {
        kernel kernel-genkernel[-v]
        initrd initramfs-genkernel[-v]
        params = quiet rootfstype=ext4
        params += luks enc_root=/dev/sda3
        params += lvm root=/dev/mapper/vg-root
}

Okay,... here you go..! :)

update the grub.cfg, then reboot and see how it works!

root@[~src/linux-3.2.6-pf] # boot-update -v     

 boot-update 1.5.2 / Copyright 2009-2011 Funtoo Technologies

 [use option "-l" for license info, "-h" for help]

 * Generating config for grub...

 DEFAULT > Funtoo Linux better-initramfs - vmlinuz-3.2.6-pf
           Funtoo Linux dracut - vmlinuz-3.2.6-pf
           Funtoo Linux genkernel - kernel-genkernel-x86_64-3.2.6-ck

 * Completed successfully.

root@[~src/linux-3.2.6-pf] #

root@[~src/linux-3.2.6-pf] # reboot

System going down for reboot!