Difference between pages "Rootfs over encrypted lvm" and "FLOP:Release Engineering"

From Funtoo
(Difference between pages)
Jump to: navigation, search
(Final steps)
 
 
Line 1: Line 1:
This howto describes how to setup LVM and rootfs with cryptoLUKS-encrypted drive
+
{{FLOP
 +
|Created on=2013/07/28
 +
|Summary=This is a proposal to implement a strong release engineering infrastructure for Funtoo Linux. Funtoo currently is only a rolling-release distro and does not have the option to also be non-rolling. In order to create a more stable Funtoo system, this proposal will be offering a few things that we can do to make that happen.
 +
|Author=Fearedbliss
 +
|Maintainer=Fearedbliss
 +
}}
 +
=== Introduction ===
  
= Prepare the hard drive and partitions =
+
This is a proposal to implement a strong release engineering infrastructure for Funtoo Linux. Funtoo currently is only a rolling-release distro and does not have the option to also be non rolling. In order to create a more stable and maintainable Funtoo, this proposal will be offering a few things that we can do to make that happen.
This is an example partition scheme, you may want to choose differently.
+
<code>/dev/sda1</code> used as <code>/boot</code>. <code>/dev/sda2</code> will be encrypted drive with LVM.
+
  
* <code>/dev/sda1</code> -- <code>/boot</code> partition.
+
This proposal will not change Funtoo from a rolling release distro to a non rolling one, but it will instead simply add the option to also be non rolling.
* <code>/dev/sda2</code> -- BIOS boot partition (not needed for MBR - only needed if you are using GPT) This step required for GRUB2. For more info, see: [http://www.funtoo.org/Funtoo_Linux_Installation#Prepare_Hard_Disk] for more information on GPT and MBR.
+
* <code>/dev/sda3</code> -- <code>/</code> partition, will be the drive with LUKS and LVM.
+
  
<console>
+
Funtoo will also not become a binary distro and will remain a source based one. However, binaries do provide various advantages that allow users to have faster deployments or easier disaster recovery. For this reason Funtoo will make available binary packages for the most time consuming, and most commonly used applications.
# ##i##dd if=/dev/zero of=/dev/sda3 bs=100M
+
# ##i##dd if=/dev/urandom of=/dev/sda3 bs=100M
+
</console>
+
The <code>dd</code> part is optional, and the command only needs to be run for security reasons (i.e only if you had top secret files on your drive). The command overwrites the lingering data on the device with random data. It takes around 6 hours to complete for a 200GB drive.
+
  
{{Note}} You will get a message about reaching the end of the device when the <code>dd</code> command has finished. This behavior is intended.
+
This will make Funtoo a much more stable and maintainable distro for users that want to have predictability with their system upgrades, whether they are a normal user, or an enterprise one.
  
= Encrypting the drive =
+
The following things are proposed:
<console>
+
* Semi-Rolling Releases (Funtoo Frozen)
# ##i##cryptsetup --cipher aes-xts-plain64 luksFormat /dev/sda3
+
* A Complete OS
# ##i##cryptsetup luksOpen /dev/sda3 dmcrypt_root
+
* Funtoo Binary Platform
</console>
+
  
There you'll be prompted to enter your password phrase for encrypted drive, type your paranoid password there.
+
=== Semi-Rolling Releases (Funtoo Frozen/Chinchilla) ===
  
= Create logical volumes =
+
The semi-rolling release model is a hybrid between a rolling release and a non-rolling release. This means that instead of bring new packages in all the time (rolling release), and instead of just completely freezing everything and bringing new packages/features every X months, we can have a middle ground where we can quickly and easily branch the Portage Tree git branch and then focus on stabilizing that tree. Once we stabilize it, people can use it without having to worry about major version upgrades. The user can then use this branch until another branch later in the future is created. The user can then easily upgrade to the new branch by switching their profile to the new version.
<console>
+
# ##i##pvcreate /dev/mapper/dmcrypt_root
+
# ##i##vgcreate vg /dev/mapper/dmcrypt_root
+
# ##i##lvcreate -L10G --name root vg         
+
# ##i##lvcreate -L2G --name swap vg
+
# ##i##lvcreate -L5G --name portage vg
+
# ##i##lvcreate -l 100%FREE -nhome vg
+
</console>
+
Feel free to specify your desired size by altering the numbers after the -L flag. For example, to make your portage dataset 20GB's, use the flag -L20G instead of -L5G.
+
  
= Create a filesystem on volumes =
+
==== New Funtoo Profiles for Releases ====
<console>
+
# ##i##mkfs.ext2 /dev/sda1
+
# ##i##mkswap /dev/mapper/vg-swap
+
# ##i##mkfs.ext4 /dev/mapper/vg-root
+
# ##i##mkfs.ext4 /dev/mapper/vg-portage
+
# ##i##mkfs.ext4 /dev/mapper/vg-home
+
</console>
+
  
= Basic system setup =
+
Since we want to provide users the ability to easily enter and exit into a frozen phase, new profiles will be added periodically.
<console>
+
# ##i##swapon /dev/mapper/vg-swap
+
# ##i##mkdir /mnt/funtoo
+
# ##i##mount /dev/mapper/vg-root /mnt/funtoo
+
# ##i##mkdir -p /mnt/funtoo/{boot,usr/portage,home}
+
# ##i##mount /dev/sda1 /mnt/funtoo/boot
+
# ##i##mount /dev/mapper/vg-portage /mnt/funtoo/usr/portage
+
# ##i##mount /dev/mapper/vg-home /mnt/funtoo/home
+
</console>
+
Now perform all the steps required for basic system install, please follow [http://docs.funtoo.org/wiki/Funtoo_Linux_Installation]
+
don't forget to emerge the following before your install is finished:
+
  
* '''cryptsetup'''
+
The first and main branch for development is the “current” branch. This is the same branch that everyone is using and that is the traditional rolling release branch.
* '''lvm2'''
+
* '''a bootloader (grub recommended)'''
+
* '''kernel sources (gentoo-sources recommended)'''
+
  
= Editing the fstab =
+
The second branch is the “stable” branch. This is the same stable branch that is available today , also using a rolling release approach and that uses ‘arch’ and other masks in order to provide stability.
Fire up your favorite text editor to edit <code>/etc/fstab</code>. You want to put the following in the file:
+
<console>
+
# <fs>                  <mountpoint>  <type>    <opts>                          <dump/pass>
+
/dev/sda1              /boot        ext2      noauto,noatime                  1 2
+
/dev/mapper/vg-swap    none          swap      sw                              0 0
+
/dev/mapper/vg-root    /            ext4      noatime,nodiratime,defaults    0 1
+
/dev/sr0                /mnt/cdrom    auto      noauto,ro                      0 0
+
/dev/mapper/vg-portage  /usr/portage  ext4      noatime,nodiratime              0 0
+
/dev/mapper/vg-home    /home        ext4      noatime,nodiratime              0 0
+
</console>
+
  
= Kernel options =
+
The new branches are considered “Funtoo Frozen” branches. These branches configure your system to follow the Funtoo tree selected. This funtoo tree will not introduce any version changes and will only include tree fixes (bugs or ebuild) and security updates.
{{Note}}This part is particularly important: pay close attention.
+
{{kernelop
+
|'''General setup --->'''
+
|'''[*] Initial RAM filesystem and RAM disk (initramfs/initrd) support'''
+
}}
+
  
{{kernelop
+
For example: Funtoo 14.1 is currently the January 2014 release. In order to use this release you can select this release from the profile module:
|'''Device Drivers --->''' <br> '''Generic Driver Options --->'''
+
|'''[*] Maintain a devtmpfs filesystem to mount at /dev''' <br>
+
}}
+
  
{{kernelop
+
# eselect profile set-build 5
|'''Device Drivers --->''' <br> '''[*] Multiple devices driver support --->'''
+
# eselect profile list
|'''<*>Device Mapper Support''' <br> '''<*> Crypt target support'''
+
Currently available arch profiles:
}}
+
[1] funtoo/1.0/linux-gnu/arch/x86-64bit *
 
+
[2] funtoo/1.0/linux-gnu/arch/pure64
{{kernelop
+
Currently available build profiles:
|'''Cryptographic API --->'''
+
[3] funtoo/1.0/linux-gnu/build/stable
|'''-*-AES cipher algorithms''' <br> '''<*> XTS support'''
+
[4] funtoo/1.0/linux-gnu/build/current
}}
+
[5] funtoo/1.0/linux-gnu/build/14.1 *
 +
Currently available flavor profiles:
 +
[6] funtoo/1.0/linux-gnu/flavor/minimal
 +
[7] funtoo/1.0/linux-gnu/flavor/core *
 +
[8] funtoo/1.0/linux-gnu/flavor/desktop
 +
[9] funtoo/1.0/linux-gnu/flavor/workstation
 +
[10] funtoo/1.0/linux-gnu/flavor/hardened
  
= Initramfs setup and configuration =
+
Once a new frozen release is released, you can change your profile to point to that release. Frozen releases are made to freeze the distro for the short term, and upgrading to the next frozen release is recommended. There will most likely be a “Long Term” frozen release as well for people that do not like to update every 4 months.
== Better-initramfs ==
+
'''Build your initramfs with [https://bitbucket.org/piotrkarbowski/better-initramfs better-initramfs] project.'''
+
  
{{note}}better-initramfs supports neither dynamic modules nor udev, so you should compile your kernel with built-in support for your block devices.
+
In the event that you do not update for various releases, a full system reinstall is recommended since a lot of things change over time including toolchain updates that have cascading effects.
  
<console>
+
==== Example of 4 Month Release Cycle ====
# ##i##cd /opt
+
# ##i##git clone git://github.com/slashbeast/better-initramfs.git
+
# ##i##cd better-initramfs
+
# ##i##less README.rst
+
# ##i##bootstrap/bootstrap-all
+
# ##i##make prepare
+
# ##i##make image
+
</console>
+
  
Copy resulting <code>initramfs.cpio.gz</code> to <code>/boot</code>:
+
Funtoo 14.1 (January 2014)
<console># ##i##cp output/initramfs.cpio.gz /boot</console>
+
Funtoo 14.2 (May 2014)
 +
Funtoo 14.3 (September 2014)
 +
Funtoo 15.1 (January 2015)
  
Alternatively, a pre-compiled binary initramfs is available at https://bitbucket.org/piotrkarbowski/better-initramfs/downloads
+
==== Which branch is for what person? ====
<console>
+
The “current” branch is for people who want to be on the bleeding edge all the time. You will get the latest updates, and here is where all the development happens. Your system might not be fully stable all the time, and things might fail to compile. This is the traditional Funtoo rolling release model. If you want to continue using your system the way it has always been, this is the branch for you.
# ##i##wget https://bitbucket.org/piotrkarbowski/better-initramfs/downloads/release-x86_64-v0.7.2.tar.bz2
+
# ##i##tar xf release-x86_64-v0.5.tar.bz2
+
# ##i##cd release*
+
# ##i##gzip initramfs.cpio
+
# ##i##cp initramfs.cpio.gz /boot
+
</console>
+
  
Remember, better-initramfs project is a work in progress, so you need to update from time to time. It can be done easily with <code>git</code>. Go to the better-initramfs source dir and follow:
+
The “stable” branch is for people who still want to be using the rolling release model but want to depend on the traditional method of ebuild’s hiding newer versions based on “~”.
<console>
+
# ##i##cd /opt/better-initramfs
+
# ##i##git pull
+
# ##i##less ChangeLog
+
</console>
+
{{Note}}Please read the ChangeLog carefuly and perform necessary updates to <code>/etc/boot.conf</code>. Also, please backup the working <code>/boot/initramfs.cpio.gz</code> and <code>/etc/boot.conf</code> before updating better-initramfs.
+
  
== Genkernel ==
+
The new frozen branches are for people who don’t want a lot of updates but would rather have a more stable version of the “current” tree that is audited for stability.
Funtoo's genkernel capable to create initramfs for encrypted drive. Compile and install kernel and initramfs of your favorite kernel sources:
+
<console>
+
# ##i##genkernel --kernel-config=/path/to/your/custom-kernel-config --no-mrproper --makeopts=-j5 --install --lvm --luks all </console>
+
Configure the bootloader as described above, with correct kernel and initramfs images names. An example for genkernel and grub2:
+
  
{{code|/etc/boot.conf|<pre>
+
=== A Complete OS ===
boot {
+
  generate grub
+
  default "Funtoo Linux"
+
  timeout 3
+
}
+
"Funtoo Linux" {
+
  kernel kernel-genkernel-x86_64-2.6.39
+
  initrd initramfs-genkernel-x86_64-2.6.39
+
  params += crypt_root=/dev/sda2 dolvm real_root=/dev/mapper/vg-root  rootfstype=ext4 resume=swap:/dev/mapper/vg-swap quiet
+
}</pre>}}
+
  
= Grub2 configuration =
+
An operating system is not just a stage3 tarball. The stage3 is incomplete and requires the user to compile their own kernel and bootloader before being able to use their system. We should have a stage which includes a well tested kernel, bootloader, and other utilities necessary for an user to deploy their system. This will speed up deployments and will provide predictability for kernel modules, and other applications that rely on a kernel.
An example of <code>/etc/boot.conf</code> for better-initramfs
+
{{code|/etc/boot.conf|<pre>
+
boot {
+
  generate grub
+
  default "Funtoo Linux"
+
  timeout 3
+
}
+
"Funtoo Linux" {
+
  kernel bzImage[-v]
+
  initrd /initramfs.cpio.gz
+
  params += enc_root=/dev/sda2 lvm luks root=/dev/mapper/vg-root  rootfstype=ext4 resume=swap:/dev/mapper/vg-swap quiet
+
}</pre>}}
+
  
= Lilo configuration =
+
The fundamental and primary contents of a stage3 is to have a full @system, and nothing more. A stage4 would be the next iteration which includes the stage3, a kernel, bootloader, and other utilities necessary for an user to deploy their system.
For oldschool geeks, an example for lilo bootloader. Emerge lilo with device-mapper support
+
<console>
+
# ##i##echo 'sys-boot/lilo device-mapper' >> /etc/portage/package.use/lilo
+
# ##i##emerge lilo
+
</console>
+
  
{{code|/etc/lilo.conf|<pre>append="init=/linuxrc dolvm crypt_root=/dev/sda2 real_root=/dev/mapper/vg-root"
+
=== Funtoo Binary Platform ===
boot=/dev/sda
+
compact
+
default=funtoo
+
lba32
+
prompt
+
read-only
+
timeout=50
+
image=/boot/kernel-genkernel-x86_64-2.6.39
+
initrd=/boot/initramfs-genkernel-x86_64-2.6.39
+
label=funtoo
+
</pre>}}
+
  
= Syslinux bootloader setup =
+
The Funtoo Binary Platform is intended to provide binaries for the most time consuming and most commonly used applications in the Funtoo Community. There are applications that will not be provided in the the FBP. Examples of these applications are applications that require a kernel for compilation, applications that require explicit license acceptance, or applications that cannot be distributed due to certain patents/copyright issues.
Syslinux is another advanced bootloader which you can find on all live CD's.  
+
<pre>
+
# emerge syslinux
+
# mkdir /boot/extlinux
+
# extlinux --install /boot/extlinux
+
# dd bs=440 conv=notrunc count=1 if=/usr/share/syslinux/mbr.bin of=/dev/sda
+
- or -
+
# sgdisk /dev/sda --attributes=1:set:2
+
# dd bs=440 conv=notrunc count=1 if=/usr/share/syslinux/gptmbr.bin of=/dev/sda, for GPT partition</pre>
+
{{code|/boot/extlinux/extlinux.conf|<pre>LABEL kernel1_bzImage-3.2.1
+
MENU LABEL Funtoo Linux bzImage-3.2.1
+
LINUX /bzImage-3.2.1
+
INITRD /initramfs.cpio.gz
+
APPEND rootfstype=ext4 luks enc_root=/dev/sda2 lvm root=/dev/mapper/vg-root
+
</pre>}}
+
  
= Final steps =
+
Example of some applications that will be in the FBP can be found at the link below:
Umount everything, close encrypted drive and reboot
+
<console>
+
# ##i##umount -l -v /mnt/funtoo/{dev, proc, home, usr/portage, boot}
+
# ##i##vgchange -a n
+
# ##i##cryptsetup luksClose /dev/sda2 dmcrypt_root
+
</console>
+
After reboot you will get the following:
+
<pre>>>> better-initramfs started. Kernel version 2.6.35-gentoo-r10
+
>>> Create all the symlinks to /bin/busybox.
+
>>> Initiating /dev/dir
+
>>> Getting LVM volumes up (if any)
+
Reding all physical volumes. This make take awhile...
+
No volume group found
+
No volume group found
+
>>> Opening encrypted partition and mapping to /dev/mapper/dmcrypt_root
+
Enter passphrase fore /dev/sda2:</pre>
+
Type your password
+
  
<pre>>>> Again, getting LVM volumes up (if any, after map dmcrypt).
+
Funtoo Binary Platform
  Reading all physical volumes.  This may take a while...
+
  Found volume group "vg" using metadata type lvm2
+
  4 logical volume(s) in volume group "vg" now active
+
>>> Mounting rootfs to /newroot
+
>>> Umounting /sys and /proc.
+
>>> Switching root to /newroot and executing /sbin/init.
+
INIT: version 2.88 booting
+
Loading /libexec/rc/console/keymap
+
  OpenRC 0.6.1 is starting up Funtoo Linux (x86_64)
+
...boot messages omitted for clarity
+
 
+
orion login: oleg
+
Password:
+
Last login: Thu Oct 14 20:49:21 EEST 2010 on tty1
+
oleg@orion ~ %</pre>
+
  
= Additional links =
+
You can download Funtoo 14.1 which is mostly using all the above concepts (Excluding profile selection) here:
* [[gentoo-wiki:Root filesystem over LVM2, DM-Crypt and RAID|Root filesystem over LVM2, DM-Crypt, and RAID]]
+
* [http://wiki.archlinux.org/index.php/System_Encryption_with_LUKS_for_dm-crypt System Encryption with LUKS for dm-crypt]
+
  
[[Category:HOWTO]]
+
Funtoo 14.1
 +
[[Category:Internals]]
 +
[[Category:FLOP]]
 +
{{FLOPFooter}}

Revision as of 00:32, 25 February 2014

Created on
2013/07/28
Original Author(s)
Fearedbliss
Current Maintainer(s)
Fearedbliss

Funtoo Linux Optimization Proposal: Release Engineering

This is a proposal to implement a strong release engineering infrastructure for Funtoo Linux. Funtoo currently is only a rolling-release distro and does not have the option to also be non-rolling. In order to create a more stable Funtoo system, this proposal will be offering a few things that we can do to make that happen.

Introduction

This is a proposal to implement a strong release engineering infrastructure for Funtoo Linux. Funtoo currently is only a rolling-release distro and does not have the option to also be non rolling. In order to create a more stable and maintainable Funtoo, this proposal will be offering a few things that we can do to make that happen.

This proposal will not change Funtoo from a rolling release distro to a non rolling one, but it will instead simply add the option to also be non rolling.

Funtoo will also not become a binary distro and will remain a source based one. However, binaries do provide various advantages that allow users to have faster deployments or easier disaster recovery. For this reason Funtoo will make available binary packages for the most time consuming, and most commonly used applications.

This will make Funtoo a much more stable and maintainable distro for users that want to have predictability with their system upgrades, whether they are a normal user, or an enterprise one.

The following things are proposed:

  • Semi-Rolling Releases (Funtoo Frozen)
  • A Complete OS
  • Funtoo Binary Platform

Semi-Rolling Releases (Funtoo Frozen/Chinchilla)

The semi-rolling release model is a hybrid between a rolling release and a non-rolling release. This means that instead of bring new packages in all the time (rolling release), and instead of just completely freezing everything and bringing new packages/features every X months, we can have a middle ground where we can quickly and easily branch the Portage Tree git branch and then focus on stabilizing that tree. Once we stabilize it, people can use it without having to worry about major version upgrades. The user can then use this branch until another branch later in the future is created. The user can then easily upgrade to the new branch by switching their profile to the new version.

New Funtoo Profiles for Releases

Since we want to provide users the ability to easily enter and exit into a frozen phase, new profiles will be added periodically.

The first and main branch for development is the “current” branch. This is the same branch that everyone is using and that is the traditional rolling release branch.

The second branch is the “stable” branch. This is the same stable branch that is available today , also using a rolling release approach and that uses ‘arch’ and other masks in order to provide stability.

The new branches are considered “Funtoo Frozen” branches. These branches configure your system to follow the Funtoo tree selected. This funtoo tree will not introduce any version changes and will only include tree fixes (bugs or ebuild) and security updates.

For example: Funtoo 14.1 is currently the January 2014 release. In order to use this release you can select this release from the profile module:

  1. eselect profile set-build 5
  2. eselect profile list

Currently available arch profiles: [1] funtoo/1.0/linux-gnu/arch/x86-64bit * [2] funtoo/1.0/linux-gnu/arch/pure64 Currently available build profiles: [3] funtoo/1.0/linux-gnu/build/stable [4] funtoo/1.0/linux-gnu/build/current [5] funtoo/1.0/linux-gnu/build/14.1 * Currently available flavor profiles: [6] funtoo/1.0/linux-gnu/flavor/minimal [7] funtoo/1.0/linux-gnu/flavor/core * [8] funtoo/1.0/linux-gnu/flavor/desktop [9] funtoo/1.0/linux-gnu/flavor/workstation [10] funtoo/1.0/linux-gnu/flavor/hardened

Once a new frozen release is released, you can change your profile to point to that release. Frozen releases are made to freeze the distro for the short term, and upgrading to the next frozen release is recommended. There will most likely be a “Long Term” frozen release as well for people that do not like to update every 4 months.

In the event that you do not update for various releases, a full system reinstall is recommended since a lot of things change over time including toolchain updates that have cascading effects.

Example of 4 Month Release Cycle

Funtoo 14.1 (January 2014) Funtoo 14.2 (May 2014) Funtoo 14.3 (September 2014) Funtoo 15.1 (January 2015)

Which branch is for what person?

The “current” branch is for people who want to be on the bleeding edge all the time. You will get the latest updates, and here is where all the development happens. Your system might not be fully stable all the time, and things might fail to compile. This is the traditional Funtoo rolling release model. If you want to continue using your system the way it has always been, this is the branch for you.

The “stable” branch is for people who still want to be using the rolling release model but want to depend on the traditional method of ebuild’s hiding newer versions based on “~”.

The new frozen branches are for people who don’t want a lot of updates but would rather have a more stable version of the “current” tree that is audited for stability.

A Complete OS

An operating system is not just a stage3 tarball. The stage3 is incomplete and requires the user to compile their own kernel and bootloader before being able to use their system. We should have a stage which includes a well tested kernel, bootloader, and other utilities necessary for an user to deploy their system. This will speed up deployments and will provide predictability for kernel modules, and other applications that rely on a kernel.

The fundamental and primary contents of a stage3 is to have a full @system, and nothing more. A stage4 would be the next iteration which includes the stage3, a kernel, bootloader, and other utilities necessary for an user to deploy their system.

Funtoo Binary Platform

The Funtoo Binary Platform is intended to provide binaries for the most time consuming and most commonly used applications in the Funtoo Community. There are applications that will not be provided in the the FBP. Examples of these applications are applications that require a kernel for compilation, applications that require explicit license acceptance, or applications that cannot be distributed due to certain patents/copyright issues.

Example of some applications that will be in the FBP can be found at the link below:

Funtoo Binary Platform

You can download Funtoo 14.1 which is mostly using all the above concepts (Excluding profile selection) here:

Funtoo 14.1