Difference between pages "OpenStack Architecture" and "Building a Kernel from Source"

From Funtoo
(Difference between pages)
Jump to: navigation, search
 
m
 
Line 1: Line 1:
This page exists to document [http://www.openstack.org OpenStack] configuration.
+
Setting up a proper kernel yourself - lean, mean and tailored to your hardware,  is the challenge by which a linux user can graduate to becoming a Funtoo knight ;-)
  
Note that the current approach is to use devstack, which is not a good way to learn OpenStack. So much of this document will be about doing a devstack-like configuration for Funtoo.
+
Even though many of us are using enterprise-ready kernels in datacenters, there is almost nobody who hasn't at least considered building a kernel for his laptop / PC.
 +
We are showing here how an intermediate Linux user can use an alternative to the standard beginners "genkernel" approach,  to compile a custom kernel,  in a relatively still speedy and easy set up.
  
This document will split OpenStack configuration into each architectural component, describing configuration steps for each component separately.
+
=== Minimum Requirements ===
 +
You should understand the way how things work in a terminal and how to use an editor and tweak config files. This is crucial.
 +
You don't need much knowledge about the linux kernel and it's internals. Nevertheless, you have to know at least where the files are located, how they are used and what is the file logic behind the overall structure. Otherwise you should consider using a non source based linux distribution.
 +
If you are scared now, don't worry - we are going to build a kernel the Funtoo way and you will pick up everthing necessary to accomplish this challenge, step by step, so the next time you do it yourself and become a real Funtoo knight!
  
== SQL Database ==
+
=== Assumptions ===
 +
You start from an installed Funtoo system on the disk, or at least, you are on stage3 in a chrooted environment from a live cd, following somehow the Funto [[Installation (Tutorial)|Installation Tutorial]].
  
A number of OpenStack services use a SQL back-end for storing various bits of data.
+
In this case we are building a kernel that is booting root in LVM over encrypted LUKS container.
 +
If you don't have this setup, don't worry, you just don't need all the modules, but everything else is similar.
  
While DevStack uses MySQL for its SQL deployment, multiple database back-ends are actually supported thanks to [http://sqlalchemy.org SQLAlchemy] being used behind the scenes, which is a re-targetable Python database API. Thus, it should be possible to use Postgres, etc, by simply using different connection strings. A list of SQLAlchemy connection types can be found on [http://docs.sqlalchemy.org/en/latest/core/engines.html this SQLAlchemy documentation page.]
 
  
Using a single root database user account for all services is not a good policy for production deployment. Ideally, each service should have its own restricted user account with only the ability to access its own database.
+
= Getting everything in order to start =
  
Let's look at how each service is configured in regards to SQL:
+
First there is the decision which linux kernel sources we need.
 +
There are plenty of them in the repositories around, often it is not easy to distinguish between them.
  
=== nova ===
+
I would always trust my distribution of choice and take what is has to offer - and funtoo has a lot to offer!
  
Here's how to set up a MySQL database back-end for nova and tell nova to initialize its database tables:
+
I really do recommend (especially if it is your first time) to build a debian-sourced genkernel like described in chapter 5 "Using Debian-Sources with Genkernel" in the [[Funtoo_Linux_Kernels| Funtoo Kernels Tutorial]].
 +
 
 +
From there you should have a running system booting nicely from your own build (just little bit bloated) kernel. This is more than you can expect from any other ready to go distribution.
 +
 
 +
{{Note}} We are using RedHat's dracut in order to build a nice initramfs (containing all the necessary tools and extra drivers our kernel might need to start the system). Although dracut is the way to go, more sophisticated and not as buggy as gentoo's genkernel approach, more and more funtoo geeks start using slashbeast's better-initramfs, which we will cover at the end of this howto! So after having set up a genkernel from debian or gentoo sources we are going to build a kernel with either (or both) dracut or/and better-initramfs. So gentoo sources with genkernel is always my backup if anything is not working correctly on my system. For the slightly more geeky approach with my own initram I am using pf-sources, ck-sources or any other more or less heavily patched sources.
 +
 
 +
Let's go!
 +
 
 +
== Kernel Sources ==
 +
We are going to use the kernel sources from the funtoo git repository.
 +
 
 +
The source you use on your system is up to you and your needs.
 +
For a laptop or desktop system, we recommend the following:
 +
 
 +
* '''sys-kernel/pf-sources'''
 +
* '''sys-kernel/ck-sources'''
 +
* '''sys-kernel/gentoo-sources'''
 +
* '''sys-kernel/git-sources'''
 +
* '''sys-kernel/sysrescue-std-sources'''
 +
* '''sys-kernel/debian-sources'''
 +
 
 +
Please, have a look in the ebuild description, look onto their homepage and take the one that suits you best!
 +
If you are unsure for now, use sys-kernel/gentoo-sources. That's always a safe bet for a general system.
 +
 
 +
It is not a problem to have various kernels installed parallel, so go on with any one of them.
 +
 
 +
I am going to use the sys-kernel/pf-sources now, as I already had the gentoo-sources installed.
 +
 
 +
== Prerequisites ==
 +
 
 +
I don't know which tools you have already installed, so some information here might be redundant.
 +
It doesn't harm to just copy and paste and do some steps again.
 +
 
 +
First, we look into our <code>/etc/make.conf</code>:
  
 
<console>
 
<console>
mysql> ##i##create database nova character set latin1;
+
###i## nano /etc/make.conf
Query OK, 1 row affected (0.02 sec)
+
#These compiler flags are just tweaking (optimazation) and NOT necessary:
 +
CFLAGS="-O2 -pipe -march=native -ftracer -fforce-addr"
 +
CXXFLAGS="${CFLAGS} -fpermissive -fomit-frame-pointer"
 +
KDIR=/usr/src/linux
 +
KERNEL="symlink build"
 +
USE="$KERNEL ....here are your use flags...."
 +
## These modules are available:
 +
## DRACUT_MODULES="dracut_modules_biosdevname dracut_modules_btrfs dracut_modules_caps dracut_modules_crypt dracut_modules_crypt-gpg dracut_modules_dmraid dracut_modules_dmsquash-live dracut_modules_gensplash dracut_modules_iscsi dracut_modules_livenet dracut_modules_lvm dracut_modules_mdraid dracut_modules_multipath dracut_modules_nbd dracut_modules_nfs dracut_modules_plymouth dracut_modules_ssh-client dracut_modules_syslog"
 +
## We will use these modules for LVM / LUKS:
 +
DRACUT_MODULES="crypt lvm plymouth biosdevname dmraid crypt-gpg dmsquash-live ssh-client syslog"
 +
</console>
  
mysql> ##i##grant all privileges on nova.* to nova@localhost identified by 'foobar';
+
Next, we set the package keywords:
Query OK, 0 rows affected (0.00 sec)
+
<console>
 +
/etc/portage/package.use/dracut:
 +
sys-kernel/dracut dm net device-mapper crypt lvm
 
</console>
 
</console>
  
Now set the following connection string in <tt>/etc/nova/nova.conf</tt>:
+
{{Note}} If you don't have lvm over encrypted LUKS you probably just add the "net" keyword here, or "selinux".
  
<pre>
 
--sql_connection=mysql://nova:foobar@localhost/nova
 
</pre>
 
 
Note the use of the latin1 character set when we created the tables in MySQL. This is so the following command will not cause an error due to the default UTF-8 character set creating indexes that are too big for MySQL to handle:
 
  
 +
After that we are going to build our packages:
 
<console>
 
<console>
# ##i##nova-manage db sync
+
###i## emerge -av app-portage/gentoolkit sys-kernel/pf-sources sys-kernel/dracut sys-boot/plymouth sys-boot/plymouth-openrc-plugin
2012-03-02 21:31:14 DEBUG nova.utils [-] backend <module 'nova.db.sqlalchemy.migration' from '/usr/lib64/python2.7/site-packages/nova/db/sqlalchemy/migration.pyc'> from (pid=17779) __get_b
+
ackend /usr/lib64/python2.7/site-packages/nova/utils.py:602
+
 
</console>
 
</console>
  
After running the command above, you should now have all the relevant database tables created:
+
Finished? Well, then let's go on and
  
 +
== Preparing the kernel ==
 +
 +
We go now to the sources directory and enter the following commands to update the kernel's  .config  file:
 
<console>
 
<console>
xdev var # ##i##mysql -u root -p nova
+
###i## cd /usr/src/linux/
Enter password:
+
###i## make clean
Reading table information for completion of table and column names
+
  CLEAN  .
You can turn off this feature to get a quicker startup with -A
+
  CLEAN  arch/x86/kernel/acpi/realmode
 +
  CLEAN  arch/x86/kernel/cpu
 +
  CLEAN  arch/x86/kernel
 +
  CLEAN  arch/x86/vdso
 +
  CLEAN  arch/x86/lib
 +
  CLEAN  drivers/gpu/drm/radeon
 +
  CLEAN  drivers/net/wan
 +
  CLEAN  drivers/scsi/aic7xxx
 +
  CLEAN  drivers/tty/vt
 +
  CLEAN  drivers/video/logo
 +
  CLEAN  firmware
 +
  CLEAN  kernel
 +
  CLEAN  lib/raid6
 +
  CLEAN  lib
 +
  CLEAN  security/apparmor
 +
  CLEAN  security/selinux
 +
  CLEAN  usr
 +
  CLEAN  arch/x86/boot/compressed
 +
  CLEAN  arch/x86/boot
 +
  CLEAN  .tmp_versions
 +
  CLEAN  vmlinux System.map .tmp_kallsyms2.S .tmp_kallsyms1.o .tmp_kallsyms2.o .tmp_kallsyms1.S .tmp_vmlinux1 .tmp_vmlinux2 .tmp_System.map
 +
###i## zcat /proc/config.gz > /usr/src/linux/.config
 +
</console>
  
Welcome to the MySQL monitor.  Commands end with ; or \g.
+
<console>
Your MySQL connection id is 16
+
###i## make localmodconfig
Server version: 5.1.61-log Gentoo Linux mysql-5.1.61
+
</console>
  
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.
+
You will get some questions which you can answer mostly with either M (compiled as a module) or Y (compiled directly into the kernel).
 +
<pre>
 +
Enable different security models (SECURITY) [Y/n/?] y
 +
Enable the securityfs filesystem (SECURITYFS) [Y/?] y
 +
Socket and Networking Security Hooks (SECURITY_NETWORK) [Y/?] y
 +
Security hooks for pathname based access control (SECURITY_PATH) [Y/?] y
 +
Low address space for LSM to protect from user allocation (LSM_MMAP_MIN_ADDR) [65536] 65536
 +
NSA SELinux Support (SECURITY_SELINUX) [Y/n/?] y
 +
  NSA SELinux boot parameter (SECURITY_SELINUX_BOOTPARAM) [N/y/?] n
 +
  NSA SELinux runtime disable (SECURITY_SELINUX_DISABLE) [N/y/?] n
 +
  NSA SELinux Development Support (SECURITY_SELINUX_DEVELOP) [Y/n/?] y
 +
  NSA SELinux AVC Statistics (SECURITY_SELINUX_AVC_STATS) [Y/n/?] y
 +
  NSA SELinux checkreqprot default value (SECURITY_SELINUX_CHECKREQPROT_VALUE) [1] 1
 +
  NSA SELinux maximum supported policy format version (SECURITY_SELINUX_POLICYDB_VERSION_MAX) [Y/n/?] y
 +
    NSA SELinux maximum supported policy format version value (SECURITY_SELINUX_POLICYDB_VERSION_MAX_VALUE) [19] 19
 +
TOMOYO Linux Support (SECURITY_TOMOYO) [Y/n/?] y
 +
  Default maximal count for learning mode (SECURITY_TOMOYO_MAX_ACCEPT_ENTRY) [2048] 2048
 +
  Default maximal count for audit log (SECURITY_TOMOYO_MAX_AUDIT_LOG) [1024] 1024
 +
  Activate without calling userspace policy loader. (SECURITY_TOMOYO_OMIT_USERSPACE_LOADER) [Y/n/?] y
 +
AppArmor support (SECURITY_APPARMOR) [Y/n/?] y
 +
  AppArmor boot parameter default value (SECURITY_APPARMOR_BOOTPARAM_VALUE) [1] 1
 +
Integrity Measurement Architecture(IMA) (IMA) [Y/n/?] y
 +
EVM support (EVM) [N/y/?] (NEW)
 +
Default security module
 +
  1. SELinux (DEFAULT_SECURITY_SELINUX)
 +
  2. TOMOYO (DEFAULT_SECURITY_TOMOYO)
 +
  3. AppArmor (DEFAULT_SECURITY_APPARMOR)
 +
> 4. Unix Discretionary Access Controls (DEFAULT_SECURITY_DAC)
 +
choice[1-4?]: 4
 +
warning: (ACPI_HOTPLUG_CPU) selects ACPI_CONTAINER which has unmet direct dependencies (ACPI && EXPERIMENTAL)
 +
warning: (MEDIA_TUNER) selects MEDIA_TUNER_TEA5761 which has unmet direct dependencies (MEDIA_SUPPORT && VIDEO_MEDIA && I2C && EXPERIMENTAL)
 +
#
 +
# configuration written to .config
 +
#
 +
warning: (GFS2_FS) selects DLM which has unmet direct dependencies (EXPERIMENTAL && INET && SYSFS && CONFIGFS_FS && (IPV6 || IPV6=n))
 +
warning: (IMA) selects TCG_TPM which has unmet direct dependencies (HAS_IOMEM && EXPERIMENTAL)
 +
warning: (MEDIA_TUNER) selects MEDIA_TUNER_TEA5761 which has unmet direct dependencies (MEDIA_SUPPORT && VIDEO_MEDIA && I2C && EXPERIMENTAL)
 +
warning: (ACPI_HOTPLUG_CPU) selects ACPI_CONTAINER which has unmet direct dependencies (ACPI && EXPERIMENTAL)
 +
root@[~src/linux] #
  
Oracle is a registered trademark of Oracle Corporation and/or its
+
</pre>
affiliates. Other names may be trademarks of their respective
+
owners.
+
  
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
+
Now comes the most adventurous part!
  
mysql> ##i##show tables;
+
= Building the Kernel =
+-------------------------------------+
+
<console>
| Tables_in_nova                      |
+
##i### make -j8  bzImage
+-------------------------------------+
+
##i### make -j8 modules
| agent_builds                        |
+
##i### make modules_install
| aggregate_hosts                    |
+
##i### make install
| aggregate_metadata                  |
+
| aggregates                          |
+
| auth_tokens                        |
+
| block_device_mapping                |
+
| bw_usage_cache                      |
+
| certificates                        |
+
| compute_nodes                      |
+
| console_pools                      |
+
...
+
 
</console>
 
</console>
  
You have now validated that nova is connecting to your MySQL database correctly.
 
  
=== glance ===
+
'''Building an initramfs or not?'''
  
From glance.openstack.org:
+
The reason to build a kernel with an initramfs is mostly for interoperability (e.g. live-cd's) and special features like an included busybox, ssh, etc.  But mostly, and that's why we are doing this here now, to have a proper kernel up and running quick'n dirty in a reasonable time without fighting hours and days until a more or less exotic hardware is perfectly run by the kernel.
 +
After having a proper basic kernel running with the help of an initramfs, I really recommend you to go a step further and build a true kernel with all features includes without an initramfs. But this could be pain in the ass and very time consuming - so we do it the funtoo way here - at least in the second example when we stick to better-initramfs instead of Red-Hat's ''dracut''.
  
<blockquote>The Glance project provides services for discovering, registering, and retrieving virtual machine images. Glance has a RESTful API that allows querying of VM image metadata as well as retrieval of the actual image.</blockquote>
+
= Option one: Initrd with dracut =
  
Glance typically uses a MySQL database called <tt>glance</tt>, although the name is configurable in the connection string.
+
To build the initrd we just execute
  
SQL connection settings might be stored in a glance configuration file located at <tt>/opt/stack/glance/etc/glance-registry.conf</tt>. In the devstack installation process, <tt>/opt/stack/glance</tt> contains a git checkout of the glance software.
+
<console>
 +
# ##i##dracut -f --fstab --xz /boot/initramfs-3.2.6-pf.img  3.2.6-pf
 +
</console>
  
The SQL connection configuration string might look something like this:
+
Generally, this really should be enough!
 +
If you experience booting problems like missing modules / drivers then just boot from the genkernel section and fix the initrd building. You can look into the man page to tweak the command a bit (e.g. --add-drivers "xz dm_crypt" etc...).
 +
 
 +
Ok let's go on and finish the taks, we are going to tell now grub how to boot off correctly!
  
 
<pre>
 
<pre>
sql_connection = mysql://glance:yourpassword@192.168.206.130/glance
+
root@[~src/linux] # nano /etc/boot.conf
</pre>
+
  
More info on glance configuration is available [http://docs.openstack.org/diablo/openstack-compute/install/content/glance-registry-conf-file.html here].
+
boot {
 +
        generate grub
 +
        default "Funtoo Linux dracut"
 +
        timeout 3
 +
}
  
=== keystone ===
+
"Funtoo Linux genkernel" {
 +
        kernel kernel-genkernel[-v]
 +
        initrd initramfs-genkernel[-v]
 +
        params = quiet rootfstype=ext4
 +
        params += luks enc_root=/dev/sda3
 +
        params += lvm root=/dev/mapper/vg-root
 +
}
  
Keystone, the OpenStack identity service, also uses SQL. <tt>etc/keystone.conf</tt> keystone install/git repo directory is used to store the SQL configuration:
+
"Funtoo Linux dracut" {
 +
        kernel vmlinuz[-v]
 +
## this is the better-initramfs generated initrd
 +
        initrd initramfs[-v].img
 +
        params  = quiet rootfstype=ext4
 +
        params += luks enc_root=/dev/sda3
 +
        params += lvm root=/dev/mapper/vg-root
 +
}
  
<pre>
 
sql_connection = %SQL_CONN%
 
 
</pre>
 
</pre>
  
As everything else, the SQL connection string uses SQLAlchemy syntax.
+
That's it almost!
  
=== Quantum and Open VSwitch ===
+
Now write to the grub.cfg with the new handy boot-update script from funtoo:
 +
<console>
 +
root@[~src/linux] # boot-update -v
 +
</console>
 +
<pre>
 +
root@[~src/linux] # sudo boot-update -v
  
<blockquote>Quantum is an incubated OpenStack project to provide "network connectivity as a service" between interface devices (e.g., vNICs) managed by other Openstack services (e.g., nova).</blockquote>
+
boot-update 1.5.2 / Copyright 2009-2011 Funtoo Technologies
  
[http://openvswitch.org Open VSwitch] is described as:
+
[use option "-l" for license info, "-h" for help]
  
<blockquote>Open vSwitch is a production quality, multilayer virtual switch licensed under the open source Apache 2.0 license. It is designed to enable massive network automation through programmatic extension, while still supporting standard management interfaces and protocols (e.g. NetFlow, sFlow, SPAN, RSPAN, CLI, LACP, 802.1ag). In addition, it is designed to support distribution across multiple physical servers similar to VMware's vNetwork distributed vswitch or Cisco's Nexus 1000V. </blockquote>
+
* Generating config for grub...
  
There is an [http://openvswitch.org/openstack/documentation/ Open VSwitch Plug-in for OpenStack Quantum] which can be set up by DevStack. This plug-in uses SQL storage. The SQLAlchemy connection string is stored in (relative to git/install root) <tt>etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini</tt>. Similarly to everything but nova, the SQL connection string is stored in <tt>sql_connection =</tt> format.
+
DEFAULT > Funtoo Linux - vmlinuz-3.2.6-pf
 +
          Funtoo Linux genkernel - kernel-genkernel-x86_64-3.2.6-pf
  
If the plugin is enabled, the following settings are added to nova.conf:
+
* Completed successfully.
 +
</pre>
 +
 
 +
Okay,... here you go..! :)
 +
 
 +
Reboot and see how it works!
 +
 
 +
= Option two: using better-initramfs =
 +
 
 +
Piotr's better-initramfs is another approach that is tiny, nice and shiny and seems to become more and more a favourite among funtoo'ers. The biggest plus is that, once built it is kernel version independant.
  
 +
For using this you just do the following steps:
 
<pre>
 
<pre>
--libvirt_vif_type=ethernet
+
1. download sources
--libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtOpenVswitchDriver
+
2. build kernel with "make bzImage"
--linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver
+
3. download better-initramfs
--quantum_use_dhcp
+
4. run better-initramfs
 +
5. adjust /etc/boot.conf
 
</pre>
 
</pre>
  
=== Melange ===
+
Here is how in detail:
  
From the Melange site:
+
Assuming you did install already a genkernel backup or at least you have a working bzImage + modules installed, we rush forward to step 3:
  
<blockquote>
+
=== download better-initramfs ===
Melange is intended to provide network information services for use across OpenStack services. The initial focus will be on IP address management (IPAM) and address discovery (DHCP/dnsmasq) functionality. Melange is intended to be a standalone service with it's own API. However, the initial use case will be to decouple existing IP management and VM address discovery from Nova and support the existing Nova networking capabilities.
+
</blockquote>
+
  
Melange also uses a <tt>sql_connection =</tt> string stored in its <tt>etc/melange/melange.conf</tt> (relative to install/git root).
+
<console>
 +
➜ # cd /usr/src/;
 +
➜ # git clone https://github.com/slashbeast/better-initramfs.git
  
== RabbitMQ ==
+
➜ # /src #  cd better-initramfs
 +
➜ # better-initramfs git:(master) ls
 +
AUTHORS    LICENSE  README.rst  bootstrap  output  sourceroot
 +
ChangeLog  Makefile  TODO        examples  scripts
 +
</console>
  
RabbitMQ  is a reliable messaging framework used by OpenStack. Currently, it looks like only nova uses it. Nova is configured to connect to rabbitmq by setting the following lines in <tt>/etc/nova/nova.conf</tt>:
+
=== build the better-initramfs ===
 +
<console>
 +
➜ # better-initramfs git:(master) sudo bootstrap/bootstrap-all
 +
Passwort: xxx
 +
# from here go and grab a coffee
 +
➜ # sudo make prepare
 +
➜ # sudo make image
 +
➜ # sudo mv output/initramfs.cpio.gz /boot
 +
</console>
 +
 
 +
=== adjust grub ===
 +
 
 +
Taking the above setup we edit the /etc/boot.conf
 +
as I installed genkernel first, and dracut after - you see this setup:
  
 
<pre>
 
<pre>
--rabbit_host=$RABBIT_HOST
+
boot {
--rabbit_password=$RABBIT_PASSWORD
+
        generate grub
 +
        default "Funtoo Linux"
 +
        timeout 3
 +
}
 +
 
 +
# Rootfs over lvm over luks
 +
# /dev/sda3 - encrypted lvm's pv
 +
# /dev/mapper/vg-root - rootfs's lv
 +
 
 +
"Funtoo Linux" {
 +
        kernel bzImage[-v]
 +
## this is the better-initramfs generated initrd
 +
        initrd initramfs.cpio.gz
 +
        params  = quiet rootfstype=ext4
 +
        params += luks enc_root=/dev/sda3
 +
        params += lvm root=/dev/mapper/vg-root
 +
}
 +
 
 +
"Funtoo Linux dracut" {
 +
        kernel vmlinuz[-v]
 +
## this is the dracut generated initrd
 +
        initrd initramfs[-v].img
 +
        params  = quiet rootfstype=ext4
 +
        params += luks enc_root=/dev/sda3
 +
        params += lvm root=/dev/mapper/vg-root
 +
}
 +
 
 +
 
 +
"Funtoo Linux genkernel" {
 +
        kernel kernel-genkernel[-v]
 +
        initrd initramfs-genkernel[-v]
 +
        params = quiet rootfstype=ext4
 +
        params += luks enc_root=/dev/sda3
 +
        params += lvm root=/dev/mapper/vg-root
 +
}
 
</pre>
 
</pre>
 +
Okay,... here you go..! :)
  
Rabbit's password is configured using the following command, as root:
+
update the grub.cfg, then reboot and see how it works!
  
 
<console>
 
<console>
# ##i##rabbitmqctl change_password guest $RABBIT_PASSWORD
+
root@[~src/linux-3.2.6-pf] # boot-update -v   
</console>
+
  
I am not yet completely sure how RabbitMQ fits into the OpenStack architecture. It may be that the supporting services expect it to be running locally, and that Nova compute nodes need to hook into a Nova instance, which would typically be running remotely. (Thus the ability for DevStack to target a remote RabbitMQ host.)
+
boot-update 1.5.2 / Copyright 2009-2011 Funtoo Technologies
  
== Virtualization Technology ==
+
[use option "-l" for license info, "-h" for help]
  
DevStack defaults to configuring OpenStack to use libvirt with [[KVM]], and will fall back to basic [[QEMU]] support if the <tt>kvm</tt> kernel module is not available. It also has support for using libvirt with [[LXC]], in addition to using [[Xen]] Server directly (bypassing libvirt.)
+
* Generating config for grub...
 +
 
 +
DEFAULT > Funtoo Linux better-initramfs - vmlinuz-3.2.6-pf
 +
          Funtoo Linux dracut - vmlinuz-3.2.6-pf
 +
          Funtoo Linux genkernel - kernel-genkernel-x86_64-3.2.6-ck
 +
 
 +
* Completed successfully.
 +
 
 +
root@[~src/linux-3.2.6-pf] #
 +
 
 +
root@[~src/linux-3.2.6-pf] # reboot
 +
 
 +
System going down for reboot!
 +
 
 +
</console>
  
  
[[Category:Virtualization]]
+
[[Category:HOWTO]]
 
[[Category:Featured]]
 
[[Category:Featured]]
[[Category:OpenStack]]
+
[[Category:Kernel]]

Revision as of 17:25, 9 January 2014

Setting up a proper kernel yourself - lean, mean and tailored to your hardware, is the challenge by which a linux user can graduate to becoming a Funtoo knight ;-)

Even though many of us are using enterprise-ready kernels in datacenters, there is almost nobody who hasn't at least considered building a kernel for his laptop / PC. We are showing here how an intermediate Linux user can use an alternative to the standard beginners "genkernel" approach, to compile a custom kernel, in a relatively still speedy and easy set up.

Minimum Requirements

You should understand the way how things work in a terminal and how to use an editor and tweak config files. This is crucial. You don't need much knowledge about the linux kernel and it's internals. Nevertheless, you have to know at least where the files are located, how they are used and what is the file logic behind the overall structure. Otherwise you should consider using a non source based linux distribution. If you are scared now, don't worry - we are going to build a kernel the Funtoo way and you will pick up everthing necessary to accomplish this challenge, step by step, so the next time you do it yourself and become a real Funtoo knight!

Assumptions

You start from an installed Funtoo system on the disk, or at least, you are on stage3 in a chrooted environment from a live cd, following somehow the Funto Installation Tutorial.

In this case we are building a kernel that is booting root in LVM over encrypted LUKS container. If you don't have this setup, don't worry, you just don't need all the modules, but everything else is similar.


Getting everything in order to start

First there is the decision which linux kernel sources we need. There are plenty of them in the repositories around, often it is not easy to distinguish between them.

I would always trust my distribution of choice and take what is has to offer - and funtoo has a lot to offer!

I really do recommend (especially if it is your first time) to build a debian-sourced genkernel like described in chapter 5 "Using Debian-Sources with Genkernel" in the Funtoo Kernels Tutorial.

From there you should have a running system booting nicely from your own build (just little bit bloated) kernel. This is more than you can expect from any other ready to go distribution.

Note Note: We are using RedHat's dracut in order to build a nice initramfs (containing all the necessary tools and extra drivers our kernel might need to start the system). Although dracut is the way to go, more sophisticated and not as buggy as gentoo's genkernel approach, more and more funtoo geeks start using slashbeast's better-initramfs, which we will cover at the end of this howto! So after having set up a genkernel from debian or gentoo sources we are going to build a kernel with either (or both) dracut or/and better-initramfs. So gentoo sources with genkernel is always my backup if anything is not working correctly on my system. For the slightly more geeky approach with my own initram I am using pf-sources, ck-sources or any other more or less heavily patched sources.

Let's go!

Kernel Sources

We are going to use the kernel sources from the funtoo git repository.

The source you use on your system is up to you and your needs. For a laptop or desktop system, we recommend the following:

  • sys-kernel/pf-sources
  • sys-kernel/ck-sources
  • sys-kernel/gentoo-sources
  • sys-kernel/git-sources
  • sys-kernel/sysrescue-std-sources
  • sys-kernel/debian-sources

Please, have a look in the ebuild description, look onto their homepage and take the one that suits you best! If you are unsure for now, use sys-kernel/gentoo-sources. That's always a safe bet for a general system.

It is not a problem to have various kernels installed parallel, so go on with any one of them.

I am going to use the sys-kernel/pf-sources now, as I already had the gentoo-sources installed.

Prerequisites

I don't know which tools you have already installed, so some information here might be redundant. It doesn't harm to just copy and paste and do some steps again.

First, we look into our /etc/make.conf:

# nano /etc/make.conf
#These compiler flags are just tweaking (optimazation) and NOT necessary:
CFLAGS="-O2 -pipe -march=native -ftracer -fforce-addr"
CXXFLAGS="${CFLAGS} -fpermissive -fomit-frame-pointer"
KDIR=/usr/src/linux
KERNEL="symlink build"
USE="$KERNEL ....here are your use flags...."
## These modules are available:
## DRACUT_MODULES="dracut_modules_biosdevname dracut_modules_btrfs dracut_modules_caps dracut_modules_crypt dracut_modules_crypt-gpg dracut_modules_dmraid dracut_modules_dmsquash-live dracut_modules_gensplash dracut_modules_iscsi dracut_modules_livenet dracut_modules_lvm dracut_modules_mdraid dracut_modules_multipath dracut_modules_nbd dracut_modules_nfs dracut_modules_plymouth dracut_modules_ssh-client dracut_modules_syslog"
## We will use these modules for LVM / LUKS:
DRACUT_MODULES="crypt lvm plymouth biosdevname dmraid crypt-gpg dmsquash-live ssh-client syslog"

Next, we set the package keywords:

/etc/portage/package.use/dracut:
sys-kernel/dracut dm net device-mapper crypt lvm

Note Note: If you don't have lvm over encrypted LUKS you probably just add the "net" keyword here, or "selinux".


After that we are going to build our packages:

# emerge -av app-portage/gentoolkit sys-kernel/pf-sources sys-kernel/dracut sys-boot/plymouth sys-boot/plymouth-openrc-plugin

Finished? Well, then let's go on and

Preparing the kernel

We go now to the sources directory and enter the following commands to update the kernel's .config file:

# cd /usr/src/linux/
# make clean
  CLEAN   .
  CLEAN   arch/x86/kernel/acpi/realmode
  CLEAN   arch/x86/kernel/cpu
  CLEAN   arch/x86/kernel
  CLEAN   arch/x86/vdso
  CLEAN   arch/x86/lib
  CLEAN   drivers/gpu/drm/radeon
  CLEAN   drivers/net/wan
  CLEAN   drivers/scsi/aic7xxx
  CLEAN   drivers/tty/vt
  CLEAN   drivers/video/logo
  CLEAN   firmware
  CLEAN   kernel
  CLEAN   lib/raid6
  CLEAN   lib
  CLEAN   security/apparmor
  CLEAN   security/selinux
  CLEAN   usr
  CLEAN   arch/x86/boot/compressed
  CLEAN   arch/x86/boot
  CLEAN   .tmp_versions
  CLEAN   vmlinux System.map .tmp_kallsyms2.S .tmp_kallsyms1.o .tmp_kallsyms2.o .tmp_kallsyms1.S .tmp_vmlinux1 .tmp_vmlinux2 .tmp_System.map
# zcat /proc/config.gz > /usr/src/linux/.config
# make localmodconfig

You will get some questions which you can answer mostly with either M (compiled as a module) or Y (compiled directly into the kernel).

Enable different security models (SECURITY) [Y/n/?] y
Enable the securityfs filesystem (SECURITYFS) [Y/?] y
Socket and Networking Security Hooks (SECURITY_NETWORK) [Y/?] y
Security hooks for pathname based access control (SECURITY_PATH) [Y/?] y
Low address space for LSM to protect from user allocation (LSM_MMAP_MIN_ADDR) [65536] 65536
NSA SELinux Support (SECURITY_SELINUX) [Y/n/?] y
  NSA SELinux boot parameter (SECURITY_SELINUX_BOOTPARAM) [N/y/?] n
  NSA SELinux runtime disable (SECURITY_SELINUX_DISABLE) [N/y/?] n
  NSA SELinux Development Support (SECURITY_SELINUX_DEVELOP) [Y/n/?] y
  NSA SELinux AVC Statistics (SECURITY_SELINUX_AVC_STATS) [Y/n/?] y
  NSA SELinux checkreqprot default value (SECURITY_SELINUX_CHECKREQPROT_VALUE) [1] 1
  NSA SELinux maximum supported policy format version (SECURITY_SELINUX_POLICYDB_VERSION_MAX) [Y/n/?] y
    NSA SELinux maximum supported policy format version value (SECURITY_SELINUX_POLICYDB_VERSION_MAX_VALUE) [19] 19
TOMOYO Linux Support (SECURITY_TOMOYO) [Y/n/?] y
  Default maximal count for learning mode (SECURITY_TOMOYO_MAX_ACCEPT_ENTRY) [2048] 2048
  Default maximal count for audit log (SECURITY_TOMOYO_MAX_AUDIT_LOG) [1024] 1024
  Activate without calling userspace policy loader. (SECURITY_TOMOYO_OMIT_USERSPACE_LOADER) [Y/n/?] y
AppArmor support (SECURITY_APPARMOR) [Y/n/?] y
  AppArmor boot parameter default value (SECURITY_APPARMOR_BOOTPARAM_VALUE) [1] 1
Integrity Measurement Architecture(IMA) (IMA) [Y/n/?] y
EVM support (EVM) [N/y/?] (NEW)
Default security module
  1. SELinux (DEFAULT_SECURITY_SELINUX)
  2. TOMOYO (DEFAULT_SECURITY_TOMOYO)
  3. AppArmor (DEFAULT_SECURITY_APPARMOR)
> 4. Unix Discretionary Access Controls (DEFAULT_SECURITY_DAC)
choice[1-4?]: 4
warning: (ACPI_HOTPLUG_CPU) selects ACPI_CONTAINER which has unmet direct dependencies (ACPI && EXPERIMENTAL)
warning: (MEDIA_TUNER) selects MEDIA_TUNER_TEA5761 which has unmet direct dependencies (MEDIA_SUPPORT && VIDEO_MEDIA && I2C && EXPERIMENTAL)
#
# configuration written to .config
#
warning: (GFS2_FS) selects DLM which has unmet direct dependencies (EXPERIMENTAL && INET && SYSFS && CONFIGFS_FS && (IPV6 || IPV6=n))
warning: (IMA) selects TCG_TPM which has unmet direct dependencies (HAS_IOMEM && EXPERIMENTAL)
warning: (MEDIA_TUNER) selects MEDIA_TUNER_TEA5761 which has unmet direct dependencies (MEDIA_SUPPORT && VIDEO_MEDIA && I2C && EXPERIMENTAL)
warning: (ACPI_HOTPLUG_CPU) selects ACPI_CONTAINER which has unmet direct dependencies (ACPI && EXPERIMENTAL)
root@[~src/linux] #

Now comes the most adventurous part!

Building the Kernel

# make -j8  bzImage
# make -j8 modules
# make modules_install
# make install


Building an initramfs or not?

The reason to build a kernel with an initramfs is mostly for interoperability (e.g. live-cd's) and special features like an included busybox, ssh, etc. But mostly, and that's why we are doing this here now, to have a proper kernel up and running quick'n dirty in a reasonable time without fighting hours and days until a more or less exotic hardware is perfectly run by the kernel. After having a proper basic kernel running with the help of an initramfs, I really recommend you to go a step further and build a true kernel with all features includes without an initramfs. But this could be pain in the ass and very time consuming - so we do it the funtoo way here - at least in the second example when we stick to better-initramfs instead of Red-Hat's dracut.

Option one: Initrd with dracut

To build the initrd we just execute

# dracut -f --fstab --xz /boot/initramfs-3.2.6-pf.img  3.2.6-pf

Generally, this really should be enough! If you experience booting problems like missing modules / drivers then just boot from the genkernel section and fix the initrd building. You can look into the man page to tweak the command a bit (e.g. --add-drivers "xz dm_crypt" etc...).

Ok let's go on and finish the taks, we are going to tell now grub how to boot off correctly!

root@[~src/linux] # nano /etc/boot.conf

boot {
        generate grub
        default "Funtoo Linux dracut"
        timeout 3
}

"Funtoo Linux genkernel" {
        kernel kernel-genkernel[-v]
        initrd initramfs-genkernel[-v]
        params = quiet rootfstype=ext4
        params += luks enc_root=/dev/sda3
        params += lvm root=/dev/mapper/vg-root
}

"Funtoo Linux dracut" {
        kernel vmlinuz[-v]
## this is the better-initramfs generated initrd
        initrd initramfs[-v].img
        params  = quiet rootfstype=ext4
        params += luks enc_root=/dev/sda3
        params += lvm root=/dev/mapper/vg-root
}

That's it almost!

Now write to the grub.cfg with the new handy boot-update script from funtoo:

root@[~src/linux] # boot-update -v
root@[~src/linux] # sudo boot-update -v

 boot-update 1.5.2 / Copyright 2009-2011 Funtoo Technologies

 [use option "-l" for license info, "-h" for help]

 * Generating config for grub...

 DEFAULT > Funtoo Linux - vmlinuz-3.2.6-pf
           Funtoo Linux genkernel - kernel-genkernel-x86_64-3.2.6-pf

 * Completed successfully.

Okay,... here you go..! :)

Reboot and see how it works!

Option two: using better-initramfs

Piotr's better-initramfs is another approach that is tiny, nice and shiny and seems to become more and more a favourite among funtoo'ers. The biggest plus is that, once built it is kernel version independant.

For using this you just do the following steps:

1. download sources
2. build kernel with "make bzImage"
3. download better-initramfs
4. run better-initramfs
5. adjust /etc/boot.conf

Here is how in detail:

Assuming you did install already a genkernel backup or at least you have a working bzImage + modules installed, we rush forward to step 3:

download better-initramfs

➜ # cd /usr/src/;
➜ # git clone https://github.com/slashbeast/better-initramfs.git

➜ # /src #  cd better-initramfs
➜ # better-initramfs git:(master) ls
AUTHORS    LICENSE   README.rst  bootstrap  output   sourceroot
ChangeLog  Makefile  TODO        examples   scripts

build the better-initramfs

➜ # better-initramfs git:(master) sudo bootstrap/bootstrap-all
Passwort: xxx 
# from here go and grab a coffee
➜ # sudo make prepare
➜ # sudo make image
➜ # sudo mv output/initramfs.cpio.gz /boot

adjust grub

Taking the above setup we edit the /etc/boot.conf as I installed genkernel first, and dracut after - you see this setup:

boot {
        generate grub
        default "Funtoo Linux"
        timeout 3
}

# Rootfs over lvm over luks
# /dev/sda3 - encrypted lvm's pv
# /dev/mapper/vg-root - rootfs's lv

"Funtoo Linux" {
        kernel bzImage[-v]
## this is the better-initramfs generated initrd
        initrd initramfs.cpio.gz
        params  = quiet rootfstype=ext4
        params += luks enc_root=/dev/sda3
        params += lvm root=/dev/mapper/vg-root
}

"Funtoo Linux dracut" {
        kernel vmlinuz[-v]
## this is the dracut generated initrd
        initrd initramfs[-v].img
        params  = quiet rootfstype=ext4
        params += luks enc_root=/dev/sda3
        params += lvm root=/dev/mapper/vg-root
}


"Funtoo Linux genkernel" {
        kernel kernel-genkernel[-v]
        initrd initramfs-genkernel[-v]
        params = quiet rootfstype=ext4
        params += luks enc_root=/dev/sda3
        params += lvm root=/dev/mapper/vg-root
}

Okay,... here you go..! :)

update the grub.cfg, then reboot and see how it works!

root@[~src/linux-3.2.6-pf] # boot-update -v     

 boot-update 1.5.2 / Copyright 2009-2011 Funtoo Technologies

 [use option "-l" for license info, "-h" for help]

 * Generating config for grub...

 DEFAULT > Funtoo Linux better-initramfs - vmlinuz-3.2.6-pf
           Funtoo Linux dracut - vmlinuz-3.2.6-pf
           Funtoo Linux genkernel - kernel-genkernel-x86_64-3.2.6-ck

 * Completed successfully.

root@[~src/linux-3.2.6-pf] #

root@[~src/linux-3.2.6-pf] # reboot

System going down for reboot!