Difference between pages "Funtoo 1.0 Profile" and "Linux Containers"

(Difference between pages)
(What It Looks Like)
 
(Create Container Configuration Files)
 
Line 1: Line 1:
[[Category:Portage]]
+
Linux Containers, or LXC, is a Linux feature that allows Linux to run one or more isolated virtual systems (with their own network interfaces, process namespace, user namespace, and power state) using a single Linux kernel on a single server.
[[Category:Labs]]
+
 
[[Category:HOWTO]]
+
== Status ==
= What It Is =
+
 
 +
As of Linux kernel 3.1.5, LXC is usable for isolating your own private workloads from one another. It is not yet ready to isolate potentially malicious users from one another or the host system. For a more mature containers solution that is appropriate for hosting environments, see [[OpenVZ]].
 +
 
 +
LXC containers don't yet have their own system uptime, and they see everything that's in the host's <tt>dmesg</tt> output, among other things. But in general, the technology works.
  
The main idea behind the Funtoo 1.0 Profile is to do away with the current monolithic "one size fits all" approach. Instead of setting one massive profile and then overriding whatever you don't want, the Funtoo 1.0 Profile uses a new multi profile approach which allows way more flexibility and customization. Instead of having to remove what you don't want, now you'll be able to add in just the parts that you do want and leave out the rest.
+
== Control groups ==
  
= How It Works =
+
* Control groups (cgroups) in kernel since 2.6.24
 +
** Allows aggregation of tasks and their children
 +
** Subsystems (cpuset, memory, blkio,...)
 +
** accounting - to measure how much resources certain systems use
 +
** resource limiting - groups can be set to not exceed a set memory limit
 +
** prioritization - some groups may get a larger share of CPU
 +
** control - freezing/unfreezing of cgroups, checkpointing and restarting
 +
** No disk quota limitation ( -> image file, LVM, XFS, directory tree quota,...)
  
Please check [[Funtoo 1.0 Profile: Internals]](Coming Soon).
+
== Subsystems ==
  
== What It Looks Like ==
 
Here's a what a list of profiles looks like:
 
 
<console>
 
<console>
starmine portage # eselect profile list
+
cat /proc/cgroups
Currently available arch profiles:
+
#subsys_name hierarchy num_cgroups enabled
  [1]  funtoo/1.0/linux-gnu/arch/x86-32bit
+
cpuset
  [2]  funtoo/1.0/linux-gnu/arch/x86-64bit
+
cpu
Currently available build profiles:
+
cpuacct
  [3]  funtoo/1.0/linux-gnu/build/stable
+
memory
  [4]  funtoo/1.0/linux-gnu/build/current
+
devices
  [5]  funtoo/1.0/linux-gnu/build/experimental
+
freezer
Currently available flavor profiles:
+
blkio
  [6]  funtoo/1.0/linux-gnu/flavor/minimal
+
perf_event
  [7]  funtoo/1.0/linux-gnu/flavor/core
+
hugetlb
  [8]  funtoo/1.0/linux-gnu/flavor/desktop
+
  [9]  funtoo/1.0/linux-gnu/flavor/workstation
+
Currently available mix-ins profiles:
+
  [10]  funtoo/1.0/linux-gnu/mix-ins/audio
+
  [11]  funtoo/1.0/linux-gnu/mix-ins/console-extras
+
  [12]  funtoo/1.0/linux-gnu/mix-ins/dvd
+
  [13]  funtoo/1.0/linux-gnu/mix-ins/gnome
+
  [14]  funtoo/1.0/linux-gnu/mix-ins/kde
+
  [15]  funtoo/1.0/linux-gnu/mix-ins/media
+
  [16]  funtoo/1.0/linux-gnu/mix-ins/print
+
  [17]  funtoo/1.0/linux-gnu/mix-ins/python3-only
+
  [18]  funtoo/1.0/linux-gnu/mix-ins/rhel5-compat
+
  [19]  funtoo/1.0/linux-gnu/mix-ins/server-db
+
  [20]  funtoo/1.0/linux-gnu/mix-ins/server-mail
+
  [21]  funtoo/1.0/linux-gnu/mix-ins/server-web
+
  [22]  funtoo/1.0/linux-gnu/mix-ins/X
+
  [23]  funtoo/1.0/linux-gnu/mix-ins/xfce
+
 
</console>
 
</console>
As you can see, there are multiple types of profiles to choose from.
 
Let's move on to how to start using it.
 
  
== Switch to the Funtoo 1.0 Profile ==
+
#cpuset    -> limits tasks to specific CPU/CPUs
=== Define the profile sub-sets you will use ===
+
#cpu        -> CPU shares
 +
#cpuacct    -> CPU accounting
 +
#memory    -> memory and swap limitation and accounting
 +
#devices    -> device allow deny list
 +
#freezer    -> suspend/resume tasks
 +
#blkio      -> I/O priorization (weight, throttle, ...)
 +
#perf_event -> support for per-cpu per-cgroup monitoring [http://lwn.net/Articles/421574/ perf_events]
 +
#hugetlb    -> cgroup resource controller for HugeTLB pages  [http://lwn.net/Articles/499255/ hugetlb]
  
So far in Funtoo we have used the exact same profiles as Gentoo thus Funtoo/2008.0 was strictly the same thing as Gentoo/2008.0 or the barely the same 10.0. This (monolithic) profile was set though a symbolic link named '''/etc/make.profile''' pointing on a complex directory architecture located somewhere under '''/usr/portage/profiles'''. This is no longer valid with the Funtoo 1.0 profiles as they are split in several smaller bricks which are then glued together via the  '''/etc/portage/make.profile/parent''' file (You do not need to include everything, just use the "bricks" you need). Those bricks belongs to several categories:
+
== Configuring the Funtoo Host System ==
  
1. MANDATORY -- An "arch" profile which defines settings for a particular architecture. You'll want to set this to whatever arch your system is and leave it alone. '''Setting it to a different arch than your system could severely break it.'''
+
=== Install LXC kernel ===
 +
Any kernel beyond 3.1.5 will probably work. Personally I prefer the sys-kernel/gentoo-sources-3.4.9 as these have support for all the namespaces without sacrificing the xfs, FUSE or NFS support for example. These checks were introduced later starting from kernel 3.5, this could also mean that the user namespace is not working optimally.
  
2. MANDATORY -- A "build" profile which should match the tree you wish to use. '''Stable''', '''Current''' (~arch), or '''Experimental''' (use it if you are brave enough and find '''current''' too stable).
+
* User namespace (EXPERIMENTAL) depends on EXPERIMENTAL and on UIDGID_CONVERTED
 +
** config UIDGID_CONVERTED
 +
*** True if all of the selected software components are known to have uid_t and gid_t converted to kuid_t and kgid_t where appropriate and are otherwise safe to use with the user namespace.
 +
**** Networking - depends on NET_9P = n
 +
**** Filesystems - 9P_FS = n, AFS_FS = n, AUTOFS4_FS = n, CEPH_FS = n, CIFS = n, CODA_FS = n, FUSE_FS = n, GFS2_FS = n, NCP_FS = n, NFSD = n, NFS_FS = n, OCFS2_FS = n, XFS_FS = n
 +
**** Security options - Grsecurity - GRKERNSEC = n (if applicable)
  
3. MANDATORY -- A "flavor" profile (what was previously known as ''profiles'' is still known as such in Gentoo) which describes the kind of system you want.
+
** As of 3.10.xx kernel, all of the above options are safe to use with User namespaces, except for XFS_FS, therefore with kernel >=3.10.xx, you should answer XFS_FS = n, if you want User namespaces support.
* minimal - Be warned, minimal is exactly what it says, the minimal profile stuff you need for a usable system, nothing else. This is really for people who know what they're doing.
+
* core - This is the core profile. This is for stuff that affects both desktops and servers.
+
* desktop - Exactly what it says. If you're using a desktop, you should set this as your flavor.
+
* server - If you're running a server, you should set this as your flavor.
+
  
4. OPTIONAL -- One or more "mix-ins" profiles which describe optional add-ons. 'mix-ins' are the heart of the Funtoo 1.0 profiles. Unlike the monolithic profiles which sets a massive amount of use flags and options for you, we've split them into logical add-on profiles. For instance if you want support for gnome, you would add the gnome mix-in to your current profiles. That mix-in sets all the proper use flags and such for gnome. Same with others. Want dvd support? Add that one in. Using a rhel5 kernel which requires special versions of packages such as udev? There's a mix-in for that too. Run a mail server? web server? There's mix-ins for those also. Expect this category to grow in the future as new mix-ins are created.
+
==== Kernel configuration ====
 +
These options should be enable in your kernel to be able to take full advantage of LXC.
  
The contents of '''/etc/portage/make.profile/parent''' for a basic setup might look like this:
+
* General setup
 +
** CONFIG_NAMESPACES
 +
*** CONFIG_UTS_NS
 +
*** CONFIG_IPC_NS
 +
*** CONFIG_PID_NS
 +
*** CONFIG_NET_NS
 +
*** CONFIG_USER_NS
 +
** CONFIG_CGROUPS
 +
*** CONFIG_CGROUP_DEVICE
 +
*** CONFIG_CGROUP_SCHED
 +
*** CONFIG_CGROUP_CPUACCT
 +
*** CONFIG_CGROUP_MEM_RES_CTLR (in 3.6+ kernels it's called CONFIG_MEMCG)
 +
*** CONFIG_CGROUP_MEM_RES_CTLR_SWAP (in 3.6+ kernels it's called CONFIG_MEMCG_SWAP)
 +
*** CONFIG_CPUSETS (on multiprocessor hosts)
 +
* Networking support
 +
** Networking options
 +
*** CONFIG_VLAN_8021Q
 +
* Device Drivers
 +
** Character devices
 +
*** Unix98 PTY support
 +
**** CONFIG_DEVPTS_MULTIPLE_INSTANCES
 +
** Network device support
 +
*** Network core driver support
 +
**** CONFIG_VETH
 +
**** CONFIG_MACVLAN
  
 +
Once you have lxc installed, you can then check your kernel config with:
 +
<console>
 +
# ##i##CONFIG=/path/to/config /usr/sbin/lxc-checkconfig
 +
</console>
 +
 +
=== Emerge lxc ===
 +
<console>
 +
# ##i##emerge -av app-emulation/lxc
 +
</console>
 +
=== Configure Networking For Container ===
 +
 +
Typically, one uses a bridge to allow containers to connect to the network. This is how to do it under Funtoo Linux:
 +
 +
# create a bridge using the Funtoo network configuration scripts. Name the bridge something like <tt>brwan</tt> (using <tt>/etc/init.d/netif.brwan</tt>). Configure your bridge to have an IP address.
 +
# Make your physical interface, such as <tt>eth0</tt>, an interface with no IP address (use the Funtoo <tt>interface-noip</tt> template.)
 +
# Make <tt>netif.eth0</tt> a slave of <tt>netif.brwan</tt> in <tt>/etc/conf.d/netif.brwan</tt>.
 +
# Enable your new bridged network and make sure it is functioning properly on the host.
 +
 +
You will now be able to configure LXC to automatically add your container's virtual ethernet interface to the bridge when it starts, which will connect it to your network.
 +
 +
== Setting up a Funtoo Linux LXC Container ==
 +
 +
Here are the steps required to get Funtoo Linux running <i>inside</i> a container. The steps below show you how to set up a container using an existing Funtoo Linux OpenVZ template. It is now also possible to use [[Metro]] to build an lxc container tarball directly, which will save you manual configuration steps and will provide an <tt>/etc/fstab.lxc</tt> file that you can use for your host container config. See [[Metro Recipes]] for info on how to use Metro to generate an lxc container.
 +
 +
=== Create and Configure Container Filesystem ===
 +
 +
# Start with a Funtoo LXC template, and unpack it to a directory such as <tt>/lxc/funtoo0/rootfs/</tt>.
 +
# Create an empty <tt>/lxc/funtoo0/fstab</tt> file.
 +
# Ensure <tt>c1</tt> line is uncommented (enabled) and <tt>c2</tt> through <tt>c6</tt> lines are disabled in <tt>/lxc/funtoo0/etc/inittab</tt>.
 +
 +
That's almost all you need to get the container filesystem ready to start.
 +
 +
=== Create Container Configuration Files ===
 +
 +
Create the following files:
 +
 +
==== <tt>/lxc/funtoo0/config</tt> ====
 +
 +
 +
and also create symlink from
 +
==== <tt> /lxc/funtoo0/config to /etc/lxc/funtoo0.conf </tt> ====
 +
<console>
 +
ln -s /lxc/funtoo0/config /etc/lxc/funtoo0.conf
 +
</console>
 +
 +
{{fancynote|Daniel Robbins needs to update this config to be more in line with http://wiki.progress-linux.org/software/lxc/ -- this config appears to have nice, refined device node permissions and other goodies. // note by Havis to Daniel, this config is already superior.}}
 +
 +
 +
Read "man 5 lxc.conf" , to get more information about linux container configuration file.
 
<pre>
 
<pre>
gentoo:funtoo/1.0/linux-gnu/arch/x86-64bit
+
## Container
gentoo:funtoo/1.0/linux-gnu/build/current
+
lxc.utsname                            = funtoo0
gentoo:funtoo/1.0/linux-gnu/flavor/core
+
lxc.rootfs                              = /lxc/funtoo0/rootfs/
 +
lxc.arch                                = x86_64
 +
#lxc.console                            = /var/log/lxc/funtoo0.console
 +
lxc.tty                                = 6 # if you plan to use container with physical termianls (eg F1..F6)
 +
#lxc.tty                                = 0 # set to 0 if you dont plan to use the container with physical terminal, also comment out in your containers /etc/inittab  c1 to c6 respawns (e.g. c1:12345:respawn:/sbin/agetty 38400 tty1 linux)
 +
lxc.pts                                = 1024
 +
 
 +
 
 +
## Capabilities
 +
lxc.cap.drop                            = audit_control
 +
lxc.cap.drop                            = audit_write
 +
lxc.cap.drop                            = mac_admin
 +
lxc.cap.drop                            = mac_override
 +
lxc.cap.drop                            = mknod
 +
lxc.cap.drop                            = setfcap
 +
lxc.cap.drop                            = setpcap
 +
lxc.cap.drop                            = sys_admin
 +
lxc.cap.drop                            = sys_boot
 +
#lxc.cap.drop                            = sys_chroot # required by SSH
 +
lxc.cap.drop                            = sys_module
 +
#lxc.cap.drop                            = sys_nice
 +
lxc.cap.drop                            = sys_pacct
 +
lxc.cap.drop                            = sys_rawio
 +
lxc.cap.drop                            = sys_resource
 +
lxc.cap.drop                            = sys_time
 +
#lxc.cap.drop                            = sys_tty_config # required by getty
 +
 
 +
## Devices
 +
# Allow all devices
 +
#lxc.cgroup.devices.allow              = a
 +
# Deny all devices
 +
lxc.cgroup.devices.deny                = a
 +
# Allow to mknod all devices (but not using them)
 +
lxc.cgroup.devices.allow                = c *:* m
 +
lxc.cgroup.devices.allow                = b *:* m
 +
 
 +
# /dev/console
 +
lxc.cgroup.devices.allow                = c 5:1 rwm
 +
# /dev/fuse
 +
lxc.cgroup.devices.allow                = c 10:229 rwm
 +
# /dev/null
 +
lxc.cgroup.devices.allow                = c 1:3 rwm
 +
# /dev/ptmx
 +
lxc.cgroup.devices.allow                = c 5:2 rwm
 +
# /dev/pts/*
 +
lxc.cgroup.devices.allow                = c 136:* rwm
 +
# /dev/random
 +
lxc.cgroup.devices.allow                = c 1:8 rwm
 +
# /dev/rtc
 +
lxc.cgroup.devices.allow                = c 254:0 rwm
 +
# /dev/tty
 +
lxc.cgroup.devices.allow                = c 5:0 rwm
 +
# /dev/urandom
 +
lxc.cgroup.devices.allow                = c 1:9 rwm
 +
# /dev/zero
 +
lxc.cgroup.devices.allow                = c 1:5 rwm
 +
 
 +
## Limits#
 +
lxc.cgroup.cpu.shares                  = 1024
 +
lxc.cgroup.cpuset.cpus                = 0       # limits container to CPU0
 +
lxc.cgroup.memory.limit_in_bytes      = 512M
 +
lxc.cgroup.memory.memsw.limit_in_bytes = 1G
 +
lxc.cgroup.blkio.weight                = 500
 +
 
 +
## Filesystem
 +
#lxc.mount                              = /lxc/funtoo0/fstab      # container fstab should be outside it's rootfs dir (e.g. /lxc/funtoo0/fstab is ok, but /lxc/funtoo0/rootfs/etc/fstab is wrong!!!)
 +
#lxc.mount.entry is now prefered
 +
lxc.mount.entry                        = proc  /lxc/funtoo0/rootfs/proc proc nodev,noexec,nosuid 0 0
 +
lxc.mount.entry                        = sysfs /lxc/funtoo0/rootfs/sys sysfs defaults,ro 0 0
 +
lxc.mount.entry                        = tmpfs /lxc/funtoo0/rootfs/tmp tmpfs defaults,size=128m,nodev,nosuid 0 0
 +
lxc.mount.entry                        = tmpfs /lxc/funtoo0/rootfs/run tmpfs defaults,size=1g,mode=0755,nosuid 0 0
 +
##Example of having /var/tmp/portage as tmpfs in container
 +
#lxc.mount.entry                        = tmpfs /lxc/funtoo0/rootfs/var/tmp/portage tmpfs defaults,size=8g,uid=250,gid=250,mode=0775 0 0
 +
##Example of bind mount
 +
#lxc.mount.entry                        = /srv/funtoo0 /lxc/funtoo0/rootfs/srv/funtoo0 none defaults,bind 0 0
 +
 
 +
## Network
 +
lxc.network.type                        = veth
 +
lxc.network.flags                      = up
 +
lxc.network.hwaddr                      = #put your MAC address here, otherwise you will get a random one
 +
lxc.network.link                        = br0
 +
lxc.network.name                        = eth0
 +
#lxc.network.veth.pair                  = veth-example
 
</pre>
 
</pre>
  
A more rounded setup for a desktop might look like this:
+
Read "man 7 capabilities" to get more information aboout Linux capabilities.
 +
 
 +
Above, use the following command to generate a random MAC for <tt>lxc.network.hwaddr</tt>:
  
 
<pre>
 
<pre>
gentoo:funtoo/1.0/linux-gnu/arch/x86-64bit
+
# openssl rand -hex 6 | sed 's/\(..\)/\1:/g; s/.$//'
gentoo:funtoo/1.0/linux-gnu/build/current
+
gentoo:funtoo/1.0/linux-gnu/flavor/desktop
+
gentoo:funtoo/1.0/linux-gnu/mix-ins/dvd
+
gentoo:funtoo/1.0/linux-gnu/mix-ins/media
+
 
</pre>
 
</pre>
  
=== Using eselect ===
+
It is a very good idea to assign a static MAC address to your container using <tt>lxc.network.hwaddr</tt>. If you don't, LXC will auto-generate a new random MAC every time your container starts, which may confuse network equipment that expects MAC addresses to remain constant.
The preferred method of adding and removing profiles is to use [[eselect|eselect profile]]. This ensures that profiles are added correctly and in the proper order. The order is very important for things to work right.
+
For a list of options, run:
+
<console>
+
###i## eselect profile help
+
</console>
+
  
As stated by the previous command output, let's see the list of what profiles currently defined the option '''list''':
+
It might happen from case to case that you aren't able to start your LXC Container with the above generated MAC address so for all these who run into that problem here is a little script that connects your IP for the container with the MAC address. Just save the following code as <tt>/etc/lxc/hwaddr.sh</tt>, make it executable and run it like <tt>/etc/lxc/hwaddr.sh xxx.xxx.xxx.xxx</tt> where xxx.xxx.xxx.xxx represents your Container IP.
  
<console>
+
<pre>
###i## eselect profile list
+
#!/bin/sh
Currently available arch profiles:
+
IP=$*
  [1]  funtoo/1.0/linux-gnu/arch/x86-64bit *
+
HA=`printf "02:00:%x:%x:%x:%x" ${IP//./ }`
Currently available build profiles:
+
echo $HA
  [2]  funtoo/1.0/linux-gnu/build/stable
+
</pre>
  [3]  funtoo/1.0/linux-gnu/build/current *
+
  [4]  funtoo/1.0/linux-gnu/build/experimental
+
Currently available flavor profiles:
+
  [5]  funtoo/1.0/linux-gnu/flavor/minimal
+
  [6]  funtoo/1.0/linux-gnu/flavor/core
+
  [7]  funtoo/1.0/linux-gnu/flavor/desktop *
+
Currently available mix-ins profiles:
+
  [8]  funtoo/1.0/linux-gnu/mix-ins/dvd
+
  [9]  funtoo/1.0/linux-gnu/mix-ins/gnome
+
  [10]  funtoo/1.0/linux-gnu/mix-ins/kde
+
  [11]  funtoo/1.0/linux-gnu/mix-ins/media
+
  [12]  funtoo/1.0/linux-gnu/mix-ins/rhel5-compat
+
  [13]  funtoo/1.0/linux-gnu/mix-ins/server-db
+
  [14]  funtoo/1.0/linux-gnu/mix-ins/server-mail
+
  [15]  funtoo/1.0/linux-gnu/mix-ins/server-web
+
  [16]  funtoo/1.0/linux-gnu/mix-ins/workstation
+
  [17]  funtoo/1.0/linux-gnu/mix-ins/workstation-minimal
+
</console>
+
  
As in several other Funtoo utilities, a star on the right indicates an active item (your case may differ from the example above). To add, say, the mix-ins '''dvd''', '''kde''' and '''media''' you have to enter:
+
==== <tt>/lxc/funtoo0/fstab</tt> ====
 +
Note: it is now preferable to have mount entries directly in config file instead of separate fstab
  
<console>
+
<pre>
###i## eselect profile add 8
+
none /lxc/funtoo0/dev/pts devpts defaults 0 0
###i## eselect profile add 10
+
none /lxc/funtoo0/proc proc defaults 0 0
###i## eselect profile add 11
+
none /lxc/funtoo0/sys sysfs defaults 0 0
</console>
+
none /lxc/funtoo0/dev/shm tmpfs nodev,nosuid,noexec,mode=1777,rw 0 0
 +
</pre>
  
Or, in a one-shot:
+
== Initializing and Starting the Container ==
  
<console>
+
You will probably need to set the root password for the container before you can log in. You can use chroot to do this quickly:
###i## eselect profile add 8 10 11
+
</console>
+
  
Verification:
+
<pre>
 +
# chroot /lxc/funtoo
 +
(chroot) # passwd
 +
New password: XXXXXXXX
 +
Retype new password: XXXXXXXX
 +
passwd: password updated successfully
 +
# exit
 +
</pre>
  
<console>
+
Now that the root password is set, run:
###i## eselect profile list 
+
Currently available arch profiles:
+
  [1]  funtoo/1.0/linux-gnu/arch/x86-64bit *
+
Currently available build profiles:
+
  [2]  funtoo/1.0/linux-gnu/build/stable
+
  [3]  funtoo/1.0/linux-gnu/build/current *
+
  [4]  funtoo/1.0/linux-gnu/build/experimental
+
Currently available flavor profiles:
+
  [5]  funtoo/1.0/linux-gnu/flavor/minimal
+
  [6]  funtoo/1.0/linux-gnu/flavor/core
+
  [7]  funtoo/1.0/linux-gnu/flavor/desktop *
+
Currently available mix-ins profiles:
+
  [8]  funtoo/1.0/linux-gnu/mix-ins/dvd *
+
  [9]  funtoo/1.0/linux-gnu/mix-ins/gnome
+
  [10]  funtoo/1.0/linux-gnu/mix-ins/kde *
+
  [11]  funtoo/1.0/linux-gnu/mix-ins/media *
+
  [12]  funtoo/1.0/linux-gnu/mix-ins/rhel5-compat
+
  [13]  funtoo/1.0/linux-gnu/mix-ins/server-db
+
  [14]  funtoo/1.0/linux-gnu/mix-ins/server-mail
+
  [15]  funtoo/1.0/linux-gnu/mix-ins/server-web
+
  [16]  funtoo/1.0/linux-gnu/mix-ins/workstation
+
  [17]  funtoo/1.0/linux-gnu/mix-ins/workstation-minimal
+
</console>
+
  
{{Note}}You must use the numbers to reference the profiles bits you want.
+
<pre>
 +
# lxc-start -n funtoo -d
 +
</pre>
  
No magic here, what you add is put by portage in the <tt>/etc/portage/make.profile/parent</tt> file. In the present case this file contains:
+
The <tt>-d</tt> option will cause it to run in the background.
 +
 
 +
To attach to the console:
  
 
<pre>
 
<pre>
###i## cat /etc/portage/make.profile/parent
+
# lxc-console -n funtoo
gentoo:funtoo/1.0/linux-gnu/arch/x86-64bit
+
</pre>
gentoo:funtoo/1.0/linux-gnu/build/current
+
 
gentoo:funtoo/1.0/linux-gnu/flavor/desktop
+
You should now be able to log in and use the container. In addition, the container should now be accessible on the network.
gentoo:funtoo/1.0/linux-gnu/mix-ins/dvd
+
 
gentoo:funtoo/1.0/linux-gnu/mix-ins/gnome
+
To stop the container:
gentoo:funtoo/1.0/linux-gnu/mix-ins/kde
+
 
gentoo:funtoo/1.0/linux-gnu/mix-ins/media
+
<pre>
</console>
+
# lxc-stop -n funtoo
 +
</pre>
 +
 
 +
Ensure that networking is working from within the container while it is running, and you're good to go!
 +
 
 +
== Starting LXC container during host boot ==
  
=== Finish Up ===
+
# You need to create symlink in /etc/init.d/ to /etc/init.d/lxc, so that it reflects your container.
{{Note}}Removing the <tt>/etc/make.profile</tt> symlink is only necessary if the auto-upgrade script failed and you are manually switching profiles. Otherwise the <tt>/etc/make.profile</tt> symlink should already be removed.
+
# ln -s /etc/init.d/lxc /etc/init.d/lxc.funtoo0
Before updating with your new profiles, you'll need to remove the <tt>/etc/make.profile</tt> symlink if it exists. Otherwise, portage will continue using the old profile.
+
# now you can add lxc.funtoo0 to default runlevel
 +
# rc-update add lxc.funtoo0 default
 
<console>
 
<console>
###i## rm /etc/make.profile
+
# rc
 +
* Starting funtoo0 ...                  [ ok ]
 
</console>
 
</console>
  
Then update world using the new profiles.
+
== LXC Bugs/Missing Features ==
<console>
+
###i## emerge -vauDN world
+
</console
+
  
Inspect the output of the prior command carefully. It is entirely possible that the use flags of several packages have changed. Many were removed in an effort to stay minimalistic.
+
This section is devoted to documenting issues with the current implementation of LXC and its associated tools. We will be gradually expanding this section with detailed descriptions of problems, their status, and proposed solutions.
  
For instance, libreoffice/openoffice will no longer have the cups use flag enabled by default.
+
=== reboot ===
  
Adjust your system-wide and application-specific use flags as necessary then re-run the prior command and update when satisfied.
+
By default, lxc does not support rebooting a container from within. It will simply stop and the host will not know to start it.
  
== Related ==
+
=== PID namespaces ===
* [[Flavors and Mix-ins]]
+
  
[[Category:Funtoo features]]
+
Process ID namespaces are functional, but the container can still see the CPU utilization of the host via the system load (ie. in <tt>top</tt>).
 +
 
 +
=== /dev/pts newinstance ===
 +
 
 +
* Some changes may be required to the host to properly implement "newinstance" <tt>/dev/pts</tt>. See [https://bugzilla.redhat.com/show_bug.cgi?id=501718 This Red Hat bug].
 +
 
 +
=== lxc-create and lxc-destroy ===
 +
 
 +
* LXC's shell scripts are badly designed and are sure way to destruction, avoid using lxc-create and lxc-destroy.
 +
 
 +
=== network initialization and cleanup ===
 +
 
 +
* If used network.type = phys after lxc-stop the interface will be renamed to value from lxc.network.link. It supposed to be fixed in 0.7.4, happens still on 0.7.5 - http://www.mail-archive.com/lxc-users@lists.sourceforge.net/msg01760.html
 +
 
 +
* Re-starting a container can result in a failure as network resource are tied up from the already-defunct instance: [http://www.mail-archive.com/lxc-devel@lists.sourceforge.net/msg00824.html]
 +
 
 +
=== lxc-halt ===
 +
 
 +
* Missing tool to graceful shutdown container. 'lxc-halt' should be written and be posix sh-compatible, using lxc-execute to run halt in container.
 +
 
 +
=== funtoo ===
 +
 
 +
* Our udev should be updated to contain <tt>-lxc</tt> in scripts. (This has been done as of 02-Nov-2011, so should be resolved. But not fixed in our openvz templates, so need to regen them in a few days.)
 +
* Our openrc should be patched to handle the case where it cannot mount tmpfs, and gracefully handle this situation somehow. (Work-around in our docs above, which is to mount tmpfs to <tt>/libexec/rc/init.d</tt> using the container-specific <tt>fstab</tt> file (on the host.)
 +
* Emerging udev within a container can/will fail when realdev is run, if a device node cannot be created (such as /dev/console) if there are no mknod capabilities within the container. This should be fixed.
 +
 
 +
== References ==
 +
 
 +
* <tt>man 7 capabilities</tt>
 +
* <tt>man 5 lxc.conf</tt>
 +
 
 +
== Links ==
 +
 
 +
* There are a number of additional lxc features that can be enabled via patches: [http://lxc.sourceforge.net/patches/linux/3.0.0/3.0.0-lxc1/]
 +
* [https://wiki.ubuntu.com/UserNamespace Ubuntu User Namespaces page]
 +
* lxc-gentoo setup script [https://github.com/globalcitizen/lxc-gentoo on GitHub]
 +
 
 +
* '''IBM developerWorks'''
 +
** [http://www.ibm.com/developerworks/linux/library/l-lxc-containers/index.html LXC: Linux Container Tools]
 +
** [http://www.ibm.com/developerworks/linux/library/l-lxc-security/ Secure Linux Containers Cookbook]
 +
 
 +
* '''Linux Weekly News'''
 +
** [http://lwn.net/Articles/244531/ Smack for simplified access control]
 +
 
 +
[[Category:Labs]]
 +
[[Category:HOWTO]]
 +
[[Category:Virtualization]]

Revision as of 20:49, 23 September 2013

Linux Containers, or LXC, is a Linux feature that allows Linux to run one or more isolated virtual systems (with their own network interfaces, process namespace, user namespace, and power state) using a single Linux kernel on a single server.

Status

As of Linux kernel 3.1.5, LXC is usable for isolating your own private workloads from one another. It is not yet ready to isolate potentially malicious users from one another or the host system. For a more mature containers solution that is appropriate for hosting environments, see OpenVZ.

LXC containers don't yet have their own system uptime, and they see everything that's in the host's dmesg output, among other things. But in general, the technology works.

Control groups

  • Control groups (cgroups) in kernel since 2.6.24
    • Allows aggregation of tasks and their children
    • Subsystems (cpuset, memory, blkio,...)
    • accounting - to measure how much resources certain systems use
    • resource limiting - groups can be set to not exceed a set memory limit
    • prioritization - some groups may get a larger share of CPU
    • control - freezing/unfreezing of cgroups, checkpointing and restarting
    • No disk quota limitation ( -> image file, LVM, XFS, directory tree quota,...)

Subsystems

cat /proc/cgroups 
#subsys_name	hierarchy	num_cgroups	enabled
cpuset	
cpu	
cpuacct	
memory	
devices	
freezer	
blkio	
perf_event
hugetlb
  1. cpuset -> limits tasks to specific CPU/CPUs
  2. cpu -> CPU shares
  3. cpuacct -> CPU accounting
  4. memory -> memory and swap limitation and accounting
  5. devices -> device allow deny list
  6. freezer -> suspend/resume tasks
  7. blkio -> I/O priorization (weight, throttle, ...)
  8. perf_event -> support for per-cpu per-cgroup monitoring perf_events
  9. hugetlb -> cgroup resource controller for HugeTLB pages hugetlb

Configuring the Funtoo Host System

Install LXC kernel

Any kernel beyond 3.1.5 will probably work. Personally I prefer the sys-kernel/gentoo-sources-3.4.9 as these have support for all the namespaces without sacrificing the xfs, FUSE or NFS support for example. These checks were introduced later starting from kernel 3.5, this could also mean that the user namespace is not working optimally.

  • User namespace (EXPERIMENTAL) depends on EXPERIMENTAL and on UIDGID_CONVERTED
    • config UIDGID_CONVERTED
      • True if all of the selected software components are known to have uid_t and gid_t converted to kuid_t and kgid_t where appropriate and are otherwise safe to use with the user namespace.
        • Networking - depends on NET_9P = n
        • Filesystems - 9P_FS = n, AFS_FS = n, AUTOFS4_FS = n, CEPH_FS = n, CIFS = n, CODA_FS = n, FUSE_FS = n, GFS2_FS = n, NCP_FS = n, NFSD = n, NFS_FS = n, OCFS2_FS = n, XFS_FS = n
        • Security options - Grsecurity - GRKERNSEC = n (if applicable)
    • As of 3.10.xx kernel, all of the above options are safe to use with User namespaces, except for XFS_FS, therefore with kernel >=3.10.xx, you should answer XFS_FS = n, if you want User namespaces support.

Kernel configuration

These options should be enable in your kernel to be able to take full advantage of LXC.

  • General setup
    • CONFIG_NAMESPACES
      • CONFIG_UTS_NS
      • CONFIG_IPC_NS
      • CONFIG_PID_NS
      • CONFIG_NET_NS
      • CONFIG_USER_NS
    • CONFIG_CGROUPS
      • CONFIG_CGROUP_DEVICE
      • CONFIG_CGROUP_SCHED
      • CONFIG_CGROUP_CPUACCT
      • CONFIG_CGROUP_MEM_RES_CTLR (in 3.6+ kernels it's called CONFIG_MEMCG)
      • CONFIG_CGROUP_MEM_RES_CTLR_SWAP (in 3.6+ kernels it's called CONFIG_MEMCG_SWAP)
      • CONFIG_CPUSETS (on multiprocessor hosts)
  • Networking support
    • Networking options
      • CONFIG_VLAN_8021Q
  • Device Drivers
    • Character devices
      • Unix98 PTY support
        • CONFIG_DEVPTS_MULTIPLE_INSTANCES
    • Network device support
      • Network core driver support
        • CONFIG_VETH
        • CONFIG_MACVLAN

Once you have lxc installed, you can then check your kernel config with:

# CONFIG=/path/to/config /usr/sbin/lxc-checkconfig

Emerge lxc

# emerge -av app-emulation/lxc

Configure Networking For Container

Typically, one uses a bridge to allow containers to connect to the network. This is how to do it under Funtoo Linux:

  1. create a bridge using the Funtoo network configuration scripts. Name the bridge something like brwan (using /etc/init.d/netif.brwan). Configure your bridge to have an IP address.
  2. Make your physical interface, such as eth0, an interface with no IP address (use the Funtoo interface-noip template.)
  3. Make netif.eth0 a slave of netif.brwan in /etc/conf.d/netif.brwan.
  4. Enable your new bridged network and make sure it is functioning properly on the host.

You will now be able to configure LXC to automatically add your container's virtual ethernet interface to the bridge when it starts, which will connect it to your network.

Setting up a Funtoo Linux LXC Container

Here are the steps required to get Funtoo Linux running inside a container. The steps below show you how to set up a container using an existing Funtoo Linux OpenVZ template. It is now also possible to use Metro to build an lxc container tarball directly, which will save you manual configuration steps and will provide an /etc/fstab.lxc file that you can use for your host container config. See Metro Recipes for info on how to use Metro to generate an lxc container.

Create and Configure Container Filesystem

  1. Start with a Funtoo LXC template, and unpack it to a directory such as /lxc/funtoo0/rootfs/.
  2. Create an empty /lxc/funtoo0/fstab file.
  3. Ensure c1 line is uncommented (enabled) and c2 through c6 lines are disabled in /lxc/funtoo0/etc/inittab.

That's almost all you need to get the container filesystem ready to start.

Create Container Configuration Files

Create the following files:

/lxc/funtoo0/config

and also create symlink from

/lxc/funtoo0/config to /etc/lxc/funtoo0.conf

ln -s /lxc/funtoo0/config /etc/lxc/funtoo0.conf

Note

Daniel Robbins needs to update this config to be more in line with http://wiki.progress-linux.org/software/lxc/ -- this config appears to have nice, refined device node permissions and other goodies. // note by Havis to Daniel, this config is already superior.


Read "man 5 lxc.conf" , to get more information about linux container configuration file.

## Container
lxc.utsname                             = funtoo0
lxc.rootfs                              = /lxc/funtoo0/rootfs/
lxc.arch                                = x86_64
#lxc.console                            = /var/log/lxc/funtoo0.console
lxc.tty                                 = 6 # if you plan to use container with physical termianls (eg F1..F6)
#lxc.tty                                = 0 # set to 0 if you dont plan to use the container with physical terminal, also comment out in your containers /etc/inittab  c1 to c6 respawns (e.g. c1:12345:respawn:/sbin/agetty 38400 tty1 linux)
lxc.pts                                 = 1024


## Capabilities
lxc.cap.drop                            = audit_control
lxc.cap.drop                            = audit_write
lxc.cap.drop                            = mac_admin
lxc.cap.drop                            = mac_override
lxc.cap.drop                            = mknod
lxc.cap.drop                            = setfcap
lxc.cap.drop                            = setpcap
lxc.cap.drop                            = sys_admin
lxc.cap.drop                            = sys_boot
#lxc.cap.drop                            = sys_chroot # required by SSH
lxc.cap.drop                            = sys_module
#lxc.cap.drop                            = sys_nice
lxc.cap.drop                            = sys_pacct
lxc.cap.drop                            = sys_rawio
lxc.cap.drop                            = sys_resource
lxc.cap.drop                            = sys_time
#lxc.cap.drop                            = sys_tty_config # required by getty

## Devices
# Allow all devices
#lxc.cgroup.devices.allow               = a
# Deny all devices
lxc.cgroup.devices.deny                 = a
# Allow to mknod all devices (but not using them)
lxc.cgroup.devices.allow                = c *:* m
lxc.cgroup.devices.allow                = b *:* m

# /dev/console
lxc.cgroup.devices.allow                = c 5:1 rwm
# /dev/fuse
lxc.cgroup.devices.allow                = c 10:229 rwm
# /dev/null
lxc.cgroup.devices.allow                = c 1:3 rwm
# /dev/ptmx
lxc.cgroup.devices.allow                = c 5:2 rwm
# /dev/pts/*
lxc.cgroup.devices.allow                = c 136:* rwm
# /dev/random
lxc.cgroup.devices.allow                = c 1:8 rwm
# /dev/rtc
lxc.cgroup.devices.allow                = c 254:0 rwm
# /dev/tty
lxc.cgroup.devices.allow                = c 5:0 rwm
# /dev/urandom
lxc.cgroup.devices.allow                = c 1:9 rwm
# /dev/zero
lxc.cgroup.devices.allow                = c 1:5 rwm

## Limits#
lxc.cgroup.cpu.shares                  = 1024
lxc.cgroup.cpuset.cpus                 = 0        # limits container to CPU0
lxc.cgroup.memory.limit_in_bytes       = 512M
lxc.cgroup.memory.memsw.limit_in_bytes = 1G
lxc.cgroup.blkio.weight                = 500

## Filesystem
#lxc.mount                               = /lxc/funtoo0/fstab       # container fstab should be outside it's rootfs dir (e.g. /lxc/funtoo0/fstab is ok, but /lxc/funtoo0/rootfs/etc/fstab is wrong!!!)
#lxc.mount.entry is now prefered
lxc.mount.entry                         = proc  /lxc/funtoo0/rootfs/proc proc nodev,noexec,nosuid 0 0
lxc.mount.entry                         = sysfs /lxc/funtoo0/rootfs/sys sysfs defaults,ro 0 0
lxc.mount.entry                         = tmpfs /lxc/funtoo0/rootfs/tmp tmpfs defaults,size=128m,nodev,nosuid 0 0
lxc.mount.entry                         = tmpfs /lxc/funtoo0/rootfs/run tmpfs defaults,size=1g,mode=0755,nosuid 0 0
##Example of having /var/tmp/portage as tmpfs in container 
#lxc.mount.entry                         = tmpfs /lxc/funtoo0/rootfs/var/tmp/portage tmpfs defaults,size=8g,uid=250,gid=250,mode=0775 0 0
##Example of bind mount
#lxc.mount.entry                        = /srv/funtoo0 /lxc/funtoo0/rootfs/srv/funtoo0 none defaults,bind 0 0

## Network
lxc.network.type                        = veth
lxc.network.flags                       = up
lxc.network.hwaddr                      = #put your MAC address here, otherwise you will get a random one
lxc.network.link                        = br0
lxc.network.name                        = eth0
#lxc.network.veth.pair                   = veth-example

Read "man 7 capabilities" to get more information aboout Linux capabilities.

Above, use the following command to generate a random MAC for lxc.network.hwaddr:

# openssl rand -hex 6 | sed 's/\(..\)/\1:/g; s/.$//'

It is a very good idea to assign a static MAC address to your container using lxc.network.hwaddr. If you don't, LXC will auto-generate a new random MAC every time your container starts, which may confuse network equipment that expects MAC addresses to remain constant.

It might happen from case to case that you aren't able to start your LXC Container with the above generated MAC address so for all these who run into that problem here is a little script that connects your IP for the container with the MAC address. Just save the following code as /etc/lxc/hwaddr.sh, make it executable and run it like /etc/lxc/hwaddr.sh xxx.xxx.xxx.xxx where xxx.xxx.xxx.xxx represents your Container IP.

#!/bin/sh
IP=$*
HA=`printf "02:00:%x:%x:%x:%x" ${IP//./ }`
echo $HA

/lxc/funtoo0/fstab

Note: it is now preferable to have mount entries directly in config file instead of separate fstab

none /lxc/funtoo0/dev/pts devpts defaults 0 0
none /lxc/funtoo0/proc proc defaults 0 0
none /lxc/funtoo0/sys sysfs defaults 0 0
none /lxc/funtoo0/dev/shm tmpfs nodev,nosuid,noexec,mode=1777,rw 0 0

Initializing and Starting the Container

You will probably need to set the root password for the container before you can log in. You can use chroot to do this quickly:

# chroot /lxc/funtoo
(chroot) # passwd
New password: XXXXXXXX
Retype new password: XXXXXXXX
passwd: password updated successfully
# exit

Now that the root password is set, run:

# lxc-start -n funtoo -d

The -d option will cause it to run in the background.

To attach to the console:

# lxc-console -n funtoo

You should now be able to log in and use the container. In addition, the container should now be accessible on the network.

To stop the container:

# lxc-stop -n funtoo

Ensure that networking is working from within the container while it is running, and you're good to go!

Starting LXC container during host boot

  1. You need to create symlink in /etc/init.d/ to /etc/init.d/lxc, so that it reflects your container.
  2. ln -s /etc/init.d/lxc /etc/init.d/lxc.funtoo0
  3. now you can add lxc.funtoo0 to default runlevel
  4. rc-update add lxc.funtoo0 default
# rc
* Starting funtoo0 ...                  [ ok ]

LXC Bugs/Missing Features

This section is devoted to documenting issues with the current implementation of LXC and its associated tools. We will be gradually expanding this section with detailed descriptions of problems, their status, and proposed solutions.

reboot

By default, lxc does not support rebooting a container from within. It will simply stop and the host will not know to start it.

PID namespaces

Process ID namespaces are functional, but the container can still see the CPU utilization of the host via the system load (ie. in top).

/dev/pts newinstance

  • Some changes may be required to the host to properly implement "newinstance" /dev/pts. See This Red Hat bug.

lxc-create and lxc-destroy

  • LXC's shell scripts are badly designed and are sure way to destruction, avoid using lxc-create and lxc-destroy.

network initialization and cleanup

  • Re-starting a container can result in a failure as network resource are tied up from the already-defunct instance: [1]

lxc-halt

  • Missing tool to graceful shutdown container. 'lxc-halt' should be written and be posix sh-compatible, using lxc-execute to run halt in container.

funtoo

  • Our udev should be updated to contain -lxc in scripts. (This has been done as of 02-Nov-2011, so should be resolved. But not fixed in our openvz templates, so need to regen them in a few days.)
  • Our openrc should be patched to handle the case where it cannot mount tmpfs, and gracefully handle this situation somehow. (Work-around in our docs above, which is to mount tmpfs to /libexec/rc/init.d using the container-specific fstab file (on the host.)
  • Emerging udev within a container can/will fail when realdev is run, if a device node cannot be created (such as /dev/console) if there are no mknod capabilities within the container. This should be fixed.

References

  • man 7 capabilities
  • man 5 lxc.conf

Links