Difference between pages "Clang" and "Xen"

From Funtoo
(Difference between pages)
Jump to: navigation, search
m (Install LLVM and its Frontends)
 
(Basic Funtoo Xen Host Dom0 setup)
 
Line 1: Line 1:
==Introduction==
+
'''Funtoo Xen Fun'''
LLVM can be used as an alternative to GNU's compiler, GCC. The main benefit of using LLVM compilers instead of GCC is their lower memory usage, faster compile time and better diagnostics. There are some Benchmarks on the [http://clang.llvm.org/features.html#performance Clang] and [http://www.phoronix.com/scan.php?page=article&item=llvm3_gcc_open64 Phoronix] homepages.
+
We are talking about Xen on Funtoo Linux and how to setup Xen virtualization properly.
 +
Especially, we are going to show you how much fun it is to work with Xen hosts and domU's and
 +
setting up a Funtoo Xen Server without general clicky GUI's or other frontends. This is true hardcore OS Xen setup especially for NOC server systems, headless servers, etc..
  
It may happen that some programs do not compile (like glibc) because they depend on GCC-specific language extensions [http://gcc.gnu.org/onlinedocs/gcc/C-Extensions.html] (this is why the whole BSD code can be compiled with LLVM but some GNU code cannot) or segfault after successful compilation with LLVM (like xorg-server) but after following this guide, the system will still be able to compile packages with gcc. So if something goes wrong, it can be switched back to gcc for the particular package by uncommenting lines in /etc/make.conf and the bug should be reported.
+
= Funtoo Xen Server with paravirt funtoo domU =
 +
'''Assumptions'''
 +
''We build a 64bit headless XEN hypervisor rockstable and rocket fast with a funtoo headless 64bit paravirt domU.''
 +
We are '''not''' building Xen with pvgrub or hvm (which is kinda slow and overhead as long as you don't want to install Windoze).
  
LLVM's C/C++ frontends clang and clang++ version 3.0 are stable enough to be self-hosting [http://blog.llvm.org/2010/02/clang-successfully-self-hosts.html] and compile Boost [http://blog.llvm.org/2010/05/clang-builds-boost.html], Qt [http://labs.qt.nokia.com/2010/10/29/compiling-qt-with-clang/], LibreOffice [http://wiki.documentfoundation.org/Development/Building_LibreOffice_with_Clang], FreeBSD [http://wiki.freebsd.org/BuildingFreeBSDWithClang], some parts of the Linux kernel [http://lwn.net/Articles/411654/] and more.
+
== Buiding Funtoo Xen Host Dom0 ==
 +
Most of the necessary steps are covered in the Installation Tutorial.
 +
We only do outline here the steps that are necessary to enjoy an easy and successful Dom0 setup or if something differs from the normal installation tutorial.
  
Further, using LLVM 3.0 and up, there is a third way to compile with LLVM: the dragonegg package creates a gcc-plugin, that uses LLVM's optimizers but parses the code and creates binaries with gcc, which means that everything that compiles and works with gcc should work with dragonegg also. This plugin can be enabled by using a single CFLAG. Since LLVM 3.0 the old llvm-gcc package is deprecated and replaced by dragonegg, so it will disappear from portage with llvm version 2.9.
+
Please, open in a second tab the [[Installation (Tutorial)|Installation Tutorial]] and follow in both carefully the next steps!
  
==LLVM Frontends==
+
=== Basic Funtoo Xen Host Dom0 setup ===
To be able to compile some sourcecode of a specific language, LLVM needs an appropriate frontend. There are clang, llvm-gcc and dragonegg in portage.
+
  
The goal of the Clang project is to create a new C, C++, Objective C and Objective C++ front-end for the LLVM compiler.
+
I recommend you use only stable packages for the host dom0 !
  
llvm-gcc is a modified version of gcc that compiles C/ObjC programs into native objects, LLVM bitcode or LLVM assembly language, depending upon the options. As written in the previous section, dragonegg replaced llvm-gcc in version 3.0.
+
Please consider the decision carefully. I can't stress out enough, you will avoid a lot of problems taking the stable distrib as dom0.
 +
The domU guests could be either unstable or hardened, as you wish! There comes the true fun part ;-)
 +
That's why I first edit my make.conf befor building anything!
  
So after installing llvm, clang and dragonegg, you will be able to choose between gcc and llvm whenever you like or use them both at the same time.
+
Here is how I set up the system basics:
 +
Disk is <tt>/dev/sda</tt>
  
== Install LLVM and its Frontends ==
+
<pre>
Simply emerge the packages on ~arch systems. On arch systems you have to unmask some packages first. dragonegg requires gcc's ''lto'' USE-flag to be set and works with gcc 4.5 and gcc 4.6.
+
/dev/sda1 is our / partition ca 20GB ext4
 +
/dev/sda2 is our swap partition ca 4GB
 +
/dev/sda3 holds the lvm volume group vgxen
 +
</pre>
 +
 
 +
I am using volume groups over raid - which I strongly advice to everybody.
 +
 
 +
Store of xen stuff:
 +
<pre>/etc/xen/ --> xend configuration files
 +
/xen/configs/ --> my xen domU configuration files folder
 +
/xen/kernel/ --> my xen domU kernel folder
 +
/xen/disks/ --> my xen domU image files folder
 +
</pre>
 +
 
 +
Edit <tt>/etc/rc.conf</tt> and uncomment the line at the bottom for rc_sys
 +
<pre>rc_sys="xen0"</pre>
 +
 
 +
== Configure and Build Xen Dom0 Kernel ==
 
<console>
 
<console>
###i## emerge llvm clang dragonegg
+
emerge gentoo-sources
 +
cd /usr/src/linux
 +
make menuconfig
 
</console>
 
</console>
Note, that for clang++ the C++ headers search path is hardcoded to the active gcc profile.
 
If you change the active gcc profile, or update gcc to a new version, you will have to remerge clang to update the search path.
 
  
To use dragonegg, run gcc as usual, with an extra command line argument <tt>-fplugin=/usr/lib/llvm/dragonegg.so</tt>
+
These settings are current as of 3.2.1-gentoo-r2, other versions may vary:
If you change the active gcc profile, or update gcc to a new version, you will have to remerge dragonegg to update the plugin.
+
<pre>
 +
General setup  --->
 +
  <*> Kernel .config support
 +
      [*]  Enable access to .config through /proc/config.gz
 +
 
 +
Processor type and features  --->
 +
  [*] Paravirtualized guest support  --->
 +
      [*]  Xen guest support
 +
 
 +
Bus options (PCI etc.)  --->
 +
  [*]  Xen PCI Frontend 
 +
 
 +
[*] Networking support  --->
 +
  Networking options  --->
 +
      <*> 802.1d Ethernet Bridging
 +
 
 +
Device Drivers  --->
 +
  [*] Block devices (NEW)  --->
 +
      <M>  DRBD Distributed Replicated Block Device support
 +
      < >  Xen virtual block device support
 +
      <*>  Xen block-device backend driver
 +
 
 +
Device Drivers  --->
 +
  [*] Network device support  --->
 +
      < >  Xen network device frontend driver
 +
      <*>  Xen backend network device
 +
 
 +
Device Drivers  --->
 +
  Graphics support  --->
 +
      -*- Support for frame buffer devices  ---
 +
        < >  Xen virtual frame buffer support
 +
 
 +
Device Drivers  --->
 +
  Xen driver support  --->
 +
      [*] Xen memory balloon driver (NEW)
 +
      [*]  Scrub pages before returning them to system (NEW)
 +
      <*> Xen /dev/xen/evtchn device (NEW)
 +
      [*] Backend driver support (NEW)
 +
      <*> Xen filesystem (NEW)
 +
      [*]  Create compatibility mount point /proc/xen (NEW)
 +
      [*] Create xen entries under /sys/hypervisor (NEW)
 +
      <M> userspace grant access device driver (NEW)
 +
      <M> User-space grant reference allocator driver (NEW)
 +
      <M> xen platform pci device driver (NEW)
 +
 
 +
File systems  --->
 +
  < > Ext3 journalling file system support
 +
  <*> The Extended 4 (ext4) filesystem
 +
  [*]  Use ext4 for ext2/ext3 file systems (NEW)
 +
  [*]  Ext4 extended attributes (NEW)
 +
</pre>
 +
* Don't forget to add the required drivers for your networking and sata cards
 +
* If you use RAID, make sure to add the correct CONFIG_MD_RAID* entries to your config
  
After the installation, check which CPUs are supported by using the command
 
 
<console>
 
<console>
###i## llvm-as < /dev/null | llc -mcpu=help
+
make
 +
make modules_install
 
</console>
 
</console>
and then add the following lines to <code>/etc/make.conf</code> (uncommenting the lines you need) to enable compilation via LLVM, adapting the march-option according to the previous command:
 
  
in <code>/etc/portage/make.conf</code>:
+
If you experience issues with connecting to the console ensure the module "xen_gntdev" (userspace grant access device driver) is loaded before the xenconsoled process is started (you may have to restart it after loading the module).
{{File
+
== Configuring Grub ==
|/etc/portage/make.conf|<pre>
+
Work has been completed to automatically enable Xen Grub entries, so after you copy your dom0 kernel edit your /etc/boot.conf as follows:
# LLVM
+
<pre>
#CC="/usr/bin/clang"
+
"Funtoo on Xen" {
#CXX="/usr/bin/clang++"
+
  type xen
 +
  xenkernel xen.gz
 +
  xenparams loglvl=all guest_loglvl=all xsave=1 iommu=1 iommu_inclusive_mapping=1 dom0_max_vcpus=2 dom0_vcpus_pin dom0_mem=4096M
 +
  kernel kernel[-v]
 +
  params += quiet
 +
}
 +
</pre>
 +
To note a fiew things iommu is the paravirtualized instructions, if your motherboard or CPU does not support VT-d do not enable it. Xsave saves the supported CPU instruction sets, without it you're dom0 kernel may not boot. And finally dom0_vcpus_pin permanatly assigns cpu's to dom0, increasing performance.
  
# llvm-gcc for C++ code and fortran
+
== Basic Networking with the Dom0 ==
# llvm-gcc is deprecated and only used with LLVM 2.9
+
Funtoo Linux offers its own modular, template-based network configuration system. This system offers a lot of flexibility for configuring network interfaces, essentially serving as a "network interface construction kit."
#CC="/usr/bin/llvm-gcc"
+
#CXX="/usr/bin/llvm-g++"
+
#CPP="/usr/bin/llvm-cpp"
+
#F77="/usr/bin/llvm-gfortran"
+
  
# Flags for clang: Insert your arch here instead of k8 and have a look at the manpage of clang for flag descriptions.
+
There is given eth0 and eth1. We are going to set eth0 as the default interface to the outside world for now. eth1 will be part of a bridge (xenbr0) that is going to be used by various domU guests.
# Some gcc flags like -pipe and -pthread also work, though they might be ignored by clang.
+
#CFLAGS="-march=k8 -O2"
+
  
# Flags for dragonegg; just use all the gcc flags you like and append -fplugin=/path/to/dragonegg.so
+
Construct the interfaces:
#CFLAGS="-march=k8 -O2 -fplugin=/usr/lib64/llvm/dragonegg.so"
+
<console>
</pre>}}
+
cd /etc/init.d/
 +
ln -s netif.tmpl netif.xenbr0
 +
ln -s netif.tmpl netif.extbr0
 +
ln -s netif.tmpl netif.eth0
 +
ln -s netif.tmpl netif.eth1
 +
rc-update add netif.xenbr0 sysinit
 +
rc-update add netif.extbr0 sysinit
 +
</console>
  
{{Note}} Have a look at clang's manpages for additional information. If you get errors that your compiler cannot produce code, you should check your flags, e.g. don't use <tt>-O4 -flto -S</tt> or stuff like that; the examples above will work.
+
Make sure dhcpcd, eth0 and eth1 don't start at boot:
 +
<console>
 +
rc-update del dhcpcd sysinit
 +
rc-update del netif.eth0 sysinit
 +
rc-update del netif.eth1 sysinit
 +
</console>
  
== Using clang with portage ==
+
Configure the slave interfaces:
 +
<console>
 +
cd /etc/conf.d/
 +
echo 'template="interface-noip"' > netif.eth0
 +
echo 'template="interface-noip"' > netif.eth1
 +
</console>
 +
Then we are going to prepare the bridges.
 +
<console>
 +
nano netif.xenbr0
 +
</console>
 +
here we set the internal Xen bridge:
 +
<pre>
 +
template="bridge"
 +
ipaddr="10.0.1.200/24"
 +
gateway="10.0.1.1"
 +
nameservers="10.0.1.1 10.0.1.2"
 +
domain="funtoo.org"
 +
slaves="netif.eth0"
 +
</pre>
 +
Then we are setting up the external interface:
 +
<console>
 +
nano netif.extbr0
 +
</console>
 +
this is looking quiet similar, please watch out for the correct slave setting!
 +
<pre>
 +
template="bridge"
 +
ipaddr="10.0.1.201/24"
 +
gateway="10.0.1.1"
 +
nameservers="10.0.1.1 10.0.1.2"
 +
domain="funtoo.org"
 +
slaves="netif.eth1"
 +
</pre>
  
Although Gentoo package tree is not designed to be used with compiler other than GCC, clang can be enforced on most of the packages through ''CC'' and ''CXX'' variables.
+
This gives us the possibility to play around with various setups later, it's modular and easy to tweak and change.
  
Please note, however, that many of Gentoo packages still don't build with clang and a few don't work correctly after being built. That's why we suggest using ''/etc/portage/env'' file to enable the use of clang per-package.
+
Better you invoke with "rc" on the command line the openrc script and test if the network cards get initialized correctly.
  
In order to do that, first create a new environment override to use:
+
== Basic Networking with domU ==
 +
 
 +
The easiest way is to let Xen set up the networking. But if everything is up and running it is not possible to change the routings, etc.
 +
Letting Xen do the bridges will be obsolete in the near future. So this is not the recommended way anymore. As we already set up the bridges in the previous section it may be enough to comment everything network related. If not, just un-comment the last lines.
 +
 
 +
We edit the /etc/xen/xend-config.sxp
 +
<pre>
 +
#### Xen config from maiwald.tk - Xen 4.x Network in bridge mode
 +
 
 +
(logfile /var/log/xen/xend.log)
 +
(loglevel DEBUG)
 +
 
 +
(xend-relocation-server no)
 +
(xend-relocation-hosts-allow '^localhost$ ^localhost\\.localdomain$')
 +
 
 +
# The limit (in kilobytes) on the size of the console buffer
 +
(console-limit 1024)
 +
 
 +
(dom0-min-mem 384)
 +
(enable-dom0-ballooning no)
 +
 
 +
(total_available_memory 0)
 +
(dom0-cpus 0)
 +
 
 +
(vncpasswd 'geheim')
 +
 
 +
# let xen create the net
 +
# (network-script    network-bridge)
 +
# (vif-script        vif-bridge)
 +
 
 +
# we create the net - new default in Xen 4
 +
#
 +
#(network-script 'network-bridge netdev=eth0 bridge=xenbr0 vifnum=0')
 +
#(vif-script vif-bridge bridge=xenbr0)
 +
</pre>
 +
 
 +
= Building the Funtoo Xen DomU Container =
 +
 
 +
We are going to build the DomU now, preparing first from outside the domU.
 +
 
 +
=== create lvm volume or partition or image file ===
 +
 
 +
''This is a stub, please help completing this guide here!''
  
in ''<code>/etc/portage/env/clang</code>'':
 
 
<console>
 
<console>
###i## nano /etc/portage/env/clang
+
# vgcreate vgxen /dev/sda3
CC=clang
+
# lvcreate -L10G -n funtoo_root vgxen
CXX=clang++
+
# lvcreate -L1G -n funtoo_swap vgxen
 +
# vgchange -a y
 +
# mkfs.ext4 -L funtoo_root /dev/vgxen/funtoo_root
 +
# mkswap -L funtoo_swap /dev/vgxen/funtoo_swap
 +
# rc-update add lvm boot
 
</console>
 
</console>
 +
== Basic DomU System setup ==
 +
=== mount domU lvm volume or physical partition or image file===
 +
<console># mkdir /mnt/domu1
 +
# mount /dev/vgxen/funtoo_root /mnt/domu1
 +
# cd /mnt/domu1
 +
</console>
 +
 +
=== get stage3 ===
 +
from a funtoo mirror near you, I suggest you look at the funtoo homepage
 +
 +
<console># links http://www.funtoo.org/wiki/Download </console>
 +
Then choose a mirror near you ( I use Heanet in EU ) and look for the right stage3. I use XEON CPUs so I take the core2 distrib:
 +
 +
<console># wget -cv http://ftp.heanet.ie/mirrors/funtoo/funtoo-stable/x86-64bit/core2_64/stage3-latest.tar.xz </console>
 +
Unfortunately I can't find md5sums or similar which is really unpleasant.
 +
 +
=== then get latest portage tree from the snapshots firectory ===
  
Then you can enable use of clang for packages using ''[[:/etc/portage/env|/etc/portage/package.env]]'' file:
+
<console># wget -cv http://ftp.heanet.ie/mirrors/funtoo/funtoo-stable/snapshots/portage-current.tar.xz </console>
in ''<code>/etc/portage/package.env</code>'':
+
=== extract stage3 ===
 
<console>
 
<console>
###i## nano /etc/portage/package.env
+
# tar xpf stage3-current.tar.xz
app-foo/bar clang
+
app-bar/baz clang
+
 
</console>
 
</console>
  
If you want to use clang by default you can and need to specify some core packages. Here is small list of core packages that are currently failing on clang, but not that could be outdated:
+
=== extract portage ===
 +
<console># cd usr
 +
# tar xf ../portage-current.tar.xz </console>
  
You need to add ''<code>/etc/portage/env/gcc</code>'':
+
== Preparing the chroot environment ==
 +
 
 +
=== Editing the make.conf ===
 +
copy /etc/make.conf from dom0 and adjust it
 
<console>
 
<console>
###i## nano /etc/portage/env/gcc
+
# cp /etc/portage/make.conf /mnt/domu1/etc/
CC=gcc
+
CXX=g++
+
 
</console>
 
</console>
  
in addition, it is recommend to add compiler flags there:
+
make sure to adjust MAKEOPTS to your assigned CPUs (rule of thumb cpu cores +1 - yes, even in XEN)
 
<console>
 
<console>
###i## nano /etc/portage/env/gcc
+
# nano -w /mnt/domu1/etc/portage/make.conf
CFLAGS="-O2 -march=native -mtune=native -pipe"
+
CXXFLAGS="-O2 -march=native -mtune=native -pipe"
+
LDFLAGS="-Wl,--as-needed"
+
#You can disable gold link here
+
#EXTRA_ECONF="--enable-gold=default"
+
 
</console>
 
</console>
 +
out there the MAKEOPTS variable in:
 +
<pre>MAKEOPTS="-j2"</pre>
 +
 +
=== copy /etc/resolv.conf ===
 +
<console># cp -L /etc/resolv.conf /mnt/domu1/etc/ </console>
 +
 +
=== mount proc and dev ===
 +
<console># mount -t proc none /mnt/domu1/proc
 +
# mount --rbind /dev /mnt/domu1/dev </console>
 +
 +
= Building Funtoo Xen Guest(s) DomU =
 +
 +
== Final DomU System setup ==
 +
=== chroot ===
 +
<console># chroot /mnt/domu1 /bin/bash
 +
# env-update
 +
# source /etc/profile
 +
# export PS1="(domU-chroot) $PS1" </console>
 +
 +
=== sync portage ===
 +
<console>
 +
# emerge --sync
 +
</console>
 +
 +
=== set locales ===
 +
<console># nano -w /etc/locale.gen
 +
# locale-gen
 +
</console>
 +
 +
=== set your timezone ===
 +
(choose your timezone in /usr/share/zoneinfo)
 +
<console># cp -L /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime</console>
 +
 +
=== edit /etc/fstab (see also gentoo handbook as reference) ===
 +
we assume that we name our root partition xvda1 and the swap partition xvda2 in our domU-xen-config (we will do that later)
 +
<console>
 +
# nano -w /etc/fstab</console>
 +
<pre>/dev/xvda1      /              ext4    noatime 0 1
 +
/dev/xvda2      none          swap    sw      0 0
 +
shm            /dev/shm      tmpfs  nodev,nosuid,noexec    0 0
 +
</pre>
 +
 +
=== Now comes the most important stuff ===
 +
please just copy this into your terminal:
 +
 +
<pre>
 +
echo '
 +
                        Larry loves Funtoo
 +
                      _________________________
 +
                      < Have you mooed today? >
 +
                      -------------------------
 +
                        \  ^__^
 +
                        \  (oo)\_______
 +
                            (__)\      )\/\
 +
                                ||----w |
 +
                                ||    ||
 +
.::::::::::::::: WELCOME TO ^^^^^^^^^^^^^^^^^^^:::::::::::::..
 +
...............................................................
 +
:########:'##::::'##:'##::: ##:'########::'#######:::'#######::.
 +
:##.....:: ##:::: ##: ###:: ##:... ##..::'##.... ##:'##.... ##::
 +
:##::::::: ##:::: ##: ####: ##:::: ##:::: ##:::: ##: ##:::: ##::
 +
:######::: ##:::: ##: ## ## ##:::: ##:::: ##:::: ##: ##:::: ##::
 +
:##...:::: ##:::: ##: ##. ####:::: ##:::: ##:::: ##: ##:::: ##::
 +
:##::::::: ##:::: ##: ##:. ###:::: ##:::: ##:::: ##: ##:::: ##::
 +
:##:::::::. #######:: ##::. ##:::: ##::::. #######::. #######::′
 +
.::::::::::.......:::..::::..:::::..::::::.......::::.......::´
 +
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 +
'> /etc/motd
 +
</pre>
 +
We are using the echo instead of „emerge --moo „ as larry still moo's in gentoo'ish
 +
 +
So that's it - almost.
 +
 +
==== adding networking to the domU: ====
  
And in ''<code>/etc/portage/package.env</code>'':
 
 
<console>
 
<console>
###i## nano /etc/portage/package.env
+
(domU-chroot) # cd /etc/init.d/
#---------------CORE PACKAGES TO BUILD WITH GCC:
+
(domU-chroot) # ln -sf netif.tmpl netif.eth0
sys-apps/which gcc
+
(domU-chroot) # rc-update add netif.eth0
sys-fs/reiserfsprogs gcc
+
* service netif.eth0 added to runlevel sysinit
sys-libs/ncurses gcc
+
sys-libs/zlib gcc
+
sys-apps/busybox gcc
+
sys-fs/e2fsprogs gcc
+
sys-devel/binutils gcc
+
sys-libs/glibc gcc
+
sys-devel/dragonegg gcc
+
dev-libs/openssl gcc
+
sys-boot/grub gcc
+
#---------------USER PACKAGES TO BUILD WITH GCC:
+
sys-apps/pacman gcc
+
www-client/firefox gcc
+
x11-libs/cairo gcc
+
media-libs/mesa gcc
+
 
</console>
 
</console>
  
If you have {{Package|app-portage/flaggie}} installed, you can modify ''<code>/etc/portage/package.env</code>'' by running the following:
+
==== Now we are ready to do the last setups ====
 
<console>
 
<console>
###i## flaggie app-foo/bar app-bar/baz +clang
+
(domU-chroot) # emerge eix
 
</console>
 
</console>
  
== Enabling link-time optimizations ==
+
After that I am feeling better now, I do the rest..
 +
<console>
 +
(domU-chroot) # eix-update
 +
Reading Portage settings ..
 +
Building database (/var/cache/eix) ..
 +
[0] "gentoo" /usr/portage/ (cache: metadata-md5-or-flat)
 +
    Reading category 154|154 (100%) Finished           
 +
Applying masks ..
 +
Calculating hash tables ..
 +
Writing database file /var/cache/eix ..
 +
Database contains 15729 packages in 154 categories.
  
The ''link-time optimization'' feature defers optimizing the resulting executables to linking phase. This can result in better optimization of packages but is unsupported in Gentoo, and many packages simply fail to build.
+
(domU-chroot) # exit
 +
exit
 +
</console>
  
When using LTO, clang compiles units into LLVM byte-code rather than machine code. In order to support linking such object files, the [[gold]] linker must be installed and set as the default linker, as it does support plugins.
+
From here you have to decide how you want to run your domU: with unpriviledged users and sudo ? or with a root account enabled? as a webserver or firewall?
  
Similarly, ''ar'' needs plugin support as well. Sadly, binutils ar doesn't support passing '--plugin'' option before the actual command. Thus, we need to create a wrapper for it:
+
Me, personally, I always do install openssh server and just place my ssh keys in there. From there the steps differ.
  
in ''<code>/usr/local/bin/clang-ar</code>'':
 
 
<console>
 
<console>
###i## nano /usr/local/bin/clang-ar
+
(dom0-xen) # cp /root/.ssh/authorized_keys /mnt/domu1/root/.ssh/
#!/bin/sh
+
</console>
firstarg=${1}
+
Also, don't forget here to enable PubKeyAuth in your sshd_config in your domU and set PermitRootLogin to yes!
shift
+
  
exec /usr/bin/ar "${firstarg}" --plugin /usr/lib/llvm/LLVMgold.so "${@}"
+
Double checking! Does your domU kernel uses modules or not? If you haven't built a monolitic kernel you should copy the modules from the dom0 to the domU now:
 +
<console>
 +
(dom0-xen) # mkdir /mnt/domu1/lib/modules
 +
(dom0-xen) # rsync -aP /lib/modules/2.6.38-xen-maiwald.tk-dom0 /mnt/domu1/lib/modules/
 
</console>
 
</console>
  
If that's done, you can create a new environment override profile for LTO-enabled clang:
+
Ok, that's it from here..
 +
 
 +
Don't forget to clean up the mounts!
  
in ''<code>/etc/portage/env/clang-lt</code>'':
 
 
<console>
 
<console>
CC='clang'
+
(dom0-xen) # cd
CXX='clang++'
+
(dom0-xen) # umount -l /mnt/domu1/proc
CFLAGS="${CFLAGS} -O4"
+
(dom0-xen) # umount -l /mnt/domu1/dev
CXXFLAGS="${CXXFLAGS} -O4"
+
(dom0-xen) # umount -l /mnt/domu1
LDFLAGS="${LDFLAGS} -O4 -Wl,-plugin,/usr/lib/llvm/LLVMgold.so"
+
AR='/usr/local/bin/clang-ar'
+
RANLIB=':'
+
NM='nm --plugin /usr/lib64/llvm/LLVMgold.so'
+
 
</console>
 
</console>
  
Note that the link-time optimizations were indirectly enabled here via ''-O4''. If you don't want to enable other optimizations enforced by ''-O3'', please use ''-flto'' instead. You need to also pass optimization flags when linking because that's where clang needs them.
+
That's it! ;-)
  
You may also need to adjust the libdir path to plugin. Newer (live) versions of clang add `-plugin` when linking automatically, so `-Wl,-plugin`… is no longer necessary.
+
=== Booting the Xen DomU Guest ===
  
== Using clang with distcc ==
+
Ok, let's try the first boot of the newly created Xen DomU in Funtoo!
  
In order to use clang on distcc client, additional symlinks have to be created in ''<code>/usr/lib*/distcc/bin</code>'':
 
 
<console>
 
<console>
###i## ln -s /usr/bin/distcc /usr/lib/distcc/bin/clang
+
(dom0-xen) # cd /xen
###i## ln -s /usr/bin/distcc /usr/lib/distcc/bin/clang++
+
(dom0-xen) # xm create -c configs/funtoo.cfg
 
</console>
 
</console>
 +
Huuuuiiiii......
 +
<pre>
 +
Using config file "./configs/funtoo.cfg".
 +
Started domain funtoo (id=4)
 +
[    0.000000] Linux version 2.6.38-xen-maiwald.tk-domU (root@xen) (gcc version 4.4.5 (Gentoo 4.4.5 p1.0, pie-0.4.5) ) #4 SMP Wed Feb 8 17:30:33 CET 2012
 +
[    0.000000] Command line: root=/dev/xvda1 ro ip=217.x.x.211:127.0.255.255:217.x.x.1:255.255.255.0:domU:eth0:off xencons=tty console=xvc0 raid=noautodetect
 +
[    0.000000] Xen-provided physical RAM map:
 +
[    0.000000]  Xen: 0000000000000000 - 0000000040800000 (usable)
 +
[    0.000000] NX (Execute Disable) protection: active
 +
[    0.000000] last_pfn = 0x40800 max_arch_pfn = 0x80000000
 +
[    0.000000] init_memory_mapping: 0000000000000000-0000000040800000
 +
[    0.000000] Zone PFN ranges:
 +
[    0.000000]  DMA      0x00000000 -> 0x00001000
 +
[    0.000000]  DMA32    0x00001000 -> 0x00100000
 +
[    0.000000]  Normal  empty
 +
[    0.000000] Movable zone start PFN for each node
 +
[    0.000000] early_node_map[2] active PFN ranges
 +
[    0.000000]    0: 0x00000000 -> 0x00040000
 +
[    0.000000]    0: 0x00040800 -> 0x00040800
 +
[    0.000000] setup_percpu: NR_CPUS:16 nr_cpumask_bits:16 nr_cpu_ids:1 nr_node_ids:1
 +
[    0.000000] PERCPU: Embedded 18 pages/cpu @ffff88003efc0000 s42304 r8192 d23232 u73728
 +
[    0.000000] Swapping MFNs for PFN 6d6 and 3efc7 (MFN 15deb0 and 1223bf)
 +
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 256109
 +
[    0.000000] Kernel command line: root=/dev/xvda1 ro ip=217.171.190.211:127.0.255.255:217.171.190.1:255.255.255.0:alyx1:eth0:off xencons=tty console=xvc0 raid=noautodetect
 +
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
 +
[    0.000000] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
 +
[    0.000000] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
 +
[    0.000000] Software IO TLB disabled
 +
[    0.000000] Memory: 1022732k/1056768k available (3657k kernel code, 8192k absent, 25844k reserved, 1261k data, 264k init)
 +
[    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
 +
[    0.000000] Hierarchical RCU implementation.
 +
[    0.000000] NR_IRQS:96
 +
[    0.000000] Xen reported: 2992.570 MHz processor.
 +
[    0.000000] Console: colour dummy device 80x25
 +
[    0.000000] console [tty-1] enabled
 +
[    0.150003] Calibrating delay using timer specific routine.. 6018.63 BogoMIPS (lpj=30093193)
 +
[    0.150008] pid_max: default: 32768 minimum: 301
 +
[    0.150034] Mount-cache hash table entries: 256
 +
[    0.150173] SMP alternatives: switching to UP code
 +
[    0.170232] Freeing SMP alternatives: 20k freed
 +
[    0.170342] Brought up 1 CPUs
 +
[    0.170377] devtmpfs: initialized
 +
[    0.170601] xor: automatically using best checksumming function: generic_sse
 +
[    0.220004]    generic_sse:  7325.200 MB/sec
 +
[    0.220008] xor: using function: generic_sse (7325.200 MB/sec)
 +
[    0.220091] NET: Registered protocol family 16
 +
[    0.220186] Brought up 1 CPUs
 +
[    0.220217] bio: create slab <bio-0> at 0
 +
[    0.390014] raid6: int64x1  2353 MB/s
 +
[    0.560003] raid6: int64x2  2964 MB/s
 +
[    0.730026] raid6: int64x4  2357 MB/s
 +
[    0.900012] raid6: int64x8  2116 MB/s
 +
[    1.070007] raid6: sse2x1    5349 MB/s
 +
[    1.240009] raid6: sse2x2    5404 MB/s
 +
[    1.410005] raid6: sse2x4    8597 MB/s
 +
[    1.410008] raid6: using algorithm sse2x4 (8597 MB/s)
 +
[    1.410022] suspend: event channel 6
 +
[    1.410022] xen_mem: Initialising balloon driver.
 +
[    1.410096] Switching to clocksource xen
 +
[    1.410125] FS-Cache: Loaded
 +
[    1.410152] CacheFiles: Loaded
 +
[    1.410268] NET: Registered protocol family 2
 +
[    1.410288] IP route cache hash table entries: 32768 (order: 6, 262144 bytes)
 +
[    1.410391] TCP established hash table entries: 131072 (order: 9, 2097152 bytes)
 +
[    1.410951] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
 +
[    1.411180] TCP: Hash tables configured (established 131072 bind 65536)
 +
[    1.411183] TCP reno registered
 +
[    1.411186] UDP hash table entries: 512 (order: 2, 16384 bytes)
 +
[    1.411192] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
 +
[    1.411229] NET: Registered protocol family 1
 +
[    1.411290] platform rtc_cmos: registered platform RTC device (no PNP device found)
 +
[    1.411401] Intel AES-NI instructions are not detected.
 +
[    1.411437] audit: initializing netlink socket (disabled)
 +
[    1.411444] type=2000 audit(1330014455.606:1): initialized
 +
[    1.412612] fuse init (API version 7.16)
 +
[    1.412674] msgmni has been set to 2048
 +
[    1.412990] NET: Registered protocol family 38
 +
[    1.413018] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
 +
[    1.413024] io scheduler noop registered (default)
 +
[    1.413026] io scheduler deadline registered
 +
[    1.413049] io scheduler cfq registered
 +
[    1.413079] Non-volatile memory driver v1.3
 +
[    1.413088] Hangcheck: starting hangcheck timer 0.9.1 (tick is 180 seconds, margin is 60 seconds).
 +
[    1.413090] Hangcheck: Using getrawmonotonic().
 +
[    1.419520] Switched to NOHz mode on CPU #0
 +
[    1.423394] brd: module loaded
 +
[    1.423665] loop: module loaded
 +
[    1.423771] nbd: registered device at major 43
 +
[    1.426180] Xen virtual console successfully installed as tty1
 +
[    1.426216] Event-channel device installed.
 +
[    1.441658] netfront: Initialising virtual ethernet driver.
 +
[    1.444972] xen-vbd: registered block device major 202
 +
[    1.444988] blkfront: xvda1: barriers enabled
 +
[    1.450287] Setting capacity to 20971520
 +
[    1.450294] xvda1: detected capacity change from 0 to 10737418240
 +
[    1.450677] blkfront: xvda2: barriers enabled
 +
[    1.451661] Setting capacity to 2097152
 +
[    1.451665] xvda2: detected capacity change from 0 to 1073741824
 +
[    1.452020] bonding: Ethernet Channel Bonding Driver: v3.7.0 (June 2, 2010)
 +
[    1.452023] bonding: Warning: either miimon or arp_interval and arp_ip_target module parameters must be specified, otherwise bonding will not detect link failures! see bonding.txt for details.
 +
[    1.453016] i8042: No controller found
 +
[    1.453066] mousedev: PS/2 mouse device common for all mice
 +
[    1.453113] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
 +
[    1.453145] rtc_cmos: probe of rtc_cmos failed with error -38
 +
[    1.453155] md: linear personality registered for level -1
 +
[    1.453158] md: raid0 personality registered for level 0
 +
[    1.453161] md: raid1 personality registered for level 1
 +
[    1.453163] md: raid6 personality registered for level 6
 +
[    1.453166] md: raid5 personality registered for level 5
 +
[    1.453168] md: raid4 personality registered for level 4
 +
[    1.453224] device-mapper: uevent: version 1.0.3
 +
[    1.453273] device-mapper: ioctl: 4.19.1-ioctl (2011-01-07) initialised: dm-devel@redhat.com
 +
[    1.453340] device-mapper: multipath: version 1.2.0 loaded
 +
[    1.453343] device-mapper: multipath round-robin: version 1.0.0 loaded
 +
[    1.453345] device-mapper: multipath queue-length: version 0.1.0 loaded
 +
[    1.453347] device-mapper: multipath service-time: version 0.2.0 loaded
 +
[    1.453396] Netfilter messages via NETLINK v0.30.
 +
[    1.453410] nf_conntrack version 0.5.0 (8192 buckets, 32768 max)
 +
[    1.453478] ctnetlink v0.93: registering with nfnetlink.
 +
[    1.453486] IPv4 over IPv4 tunneling driver
 +
[    1.453548] TCP westwood registered
 +
[    1.453550] TCP highspeed registered
 +
[    1.453552] TCP htcp registered
 +
[    1.453553] TCP vegas registered
 +
[    1.453555] Initializing XFRM netlink socket
 +
[    1.453630] NET: Registered protocol family 10
 +
[    1.453803] IPv6 over IPv4 tunneling driver
 +
[    1.453863] NET: Registered protocol family 17
 +
[    1.453868] NET: Registered protocol family 15
 +
[    1.453870] Registering the dns_resolver key type
 +
[    1.550094] /usr/src/linux-2.6.38-xen/drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
 +
[    3.070104] IP-Config: Complete:
 +
[    3.070109]      device=eth0, addr=217.171.190.211, mask=255.255.255.0, gw=217.171.190.1,
 +
[    3.070116]      host=alyx1, domain=, nis-domain=(none),
 +
[    3.070119]      bootserver=127.0.255.255, rootserver=127.0.255.255, rootpath=
 +
[    3.070212] md: Skipping autodetection of RAID arrays. (raid=autodetect will force)
 +
[    3.107309] EXT4-fs (xvda1): mounted filesystem with ordered data mode. Opts: (null)
 +
[    3.107321] VFS: Mounted root (ext2 filesystem) readonly on device 202:1.
 +
[    3.140059] devtmpfs: mounted
 +
[    3.140239] Freeing unused kernel memory: 264k freed
 +
INIT: version 2.88 booting
 +
 +
  OpenRC 0.8.3 is starting up Funtoo Linux (x86_64)
 +
 +
* Mounting /proc ...
 +
[ ok ]
 +
* WARNING: rc_sys not defined in rc.conf. Falling back to automatic detection
 +
* Caching service dependencies ...
 +
[ ok ]
 +
* Mounting /sys ...
 +
[ ok ]
 +
* udev: /dev already mounted, skipping...
 +
* Mounting /dev/pts ...
 +
[ ok ]
 +
* Mounting /dev/shm ...
 +
[ ok ]
 +
* Bringing up network interface lo ...
 +
RTNETLINK answers: File exists
 +
[ ok ]
 +
* Bringing up network interface lo ...
 +
RTNETLINK answers: File exists
 +
RTNETLINK answers: File exists
 +
[ ok ]
 +
* Starting udevd daemon ...
 +
* Populating /dev with existing devices through uevents ...
 +
[ ok ]
 +
* Autoloaded 0 module(s)
 +
* Checking local filesystems  ...
 +
funtoo_root: Superblock last write time is in the future.
 +
        (by less than a day, probably due to the hardware clock being incorrectly set).  FIXED.
 +
funtoo_root: clean, 173796/655360 files, 436917/2621440 blocks
 +
[ ok ]
 +
* Remounting root filesystem read/write ...
 +
[ ok ]
 +
* Updating /etc/mtab ...
 +
[ ok ]
 +
* Mounting local filesystems ...
 +
[ ok ]
 +
* Configuring kernel parameters ...
 +
[ ok ]
 +
* Creating user login records ...
 +
[ ok ]
 +
* Cleaning /var/run ...
 +
[ ok ]
 +
* Wiping /tmp directory ...
 +
[ ok ]
 +
* Setting hostname to localhost ...
 +
[ ok ]
 +
* Activating swap devices ...
 +
[ ok ]
 +
* udev: storing persistent rules ...
 +
[ ok ]
 +
* Initializing random number generator ...
 +
[ ok ]
 +
INIT: Entering runlevel: 3
 +
* Mounting network filesystems ...
 +
[ ok ]
 +
* Generating dsa host key ...
 +
Generating public/private dsa key pair.
 +
Your identification has been saved in /etc/ssh/ssh_host_dsa_key.
 +
Your public key has been saved in /etc/ssh/ssh_host_dsa_key.pub.
 +
The key fingerprint is:
 +
25:e0:a8:05:xxxxxxxxxxxx:1c:1f:ba root@localhost
 +
The key's randomart image is:
 +
+--[ DSA 1024]----+
 +
|  ooo.B.o        |
 +
| o o *.B o .    |
 +
|  . + + = =      |
 +
|  o  + *      |
 +
|  .  E S        |
 +
|                |
 +
|                |
 +
|                |
 +
|                |
 +
+-----------------+
 +
[ ok ]
 +
* Generating rsa host key ...
 +
Generating public/private rsa key pair.
 +
Your identification has been saved in /etc/ssh/ssh_host_rsa_key.
 +
Your public key has been saved in /etc/ssh/ssh_host_rsa_key.pub.
 +
The key fingerprint is:
 +
22:e3:46:28:67:xxxxxxxxxxxxxxxxxxxxx:e5:c3 root@localhost
 +
The key's randomart image is:
 +
+--[ RSA 2048]----+
 +
|.    o. ..      |
 +
|oo  o ..o        |
 +
|=oo  o  E      |
 +
|.*oo.    .      |
 +
|o *.+ . S        |
 +
| + o o .        |
 +
|    o            |
 +
|  .            |
 +
|                |
 +
+-----------------+
 +
[ ok ]
 +
* Starting sshd ...
 +
[ ok ]
 +
* Starting local
 +
[ ok ]
 +
 +
 +
                        Larry loves Funtoo
 +
                      _________________________
 +
                      < Have you mooed today? >
 +
                      -------------------------
 +
                          ^__^
 +
                          (oo)_______
 +
                            (__)      )/
 +
                                ||----w |
 +
                                ||    ||
 +
.::::::::::::::::::::: WELCOME TO ::::::::::::::::::::::::::..
 +
...............................................................
 +
:########:'##::::'##:'##::: ##:'########::'#######:::'#######::.
 +
:##.....:: ##:::: ##: ###:: ##:... ##..::'##.... ##:'##.... ##::
 +
:##::::::: ##:::: ##: ####: ##:::: ##:::: ##:::: ##: ##:::: ##::
 +
:######::: ##:::: ##: ## ## ##:::: ##:::: ##:::: ##: ##:::: ##::
 +
:##...:::: ##:::: ##: ##. ####:::: ##:::: ##:::: ##: ##:::: ##::
 +
:##::::::: ##:::: ##: ##:. ###:::: ##:::: ##:::: ##: ##:::: ##::
 +
:##:::::::. #######:: ##::. ##:::: ##::::. #######::. #######::′
 +
.::::::::::.......:::..::::..:::::..::::::.......::::.......::´
 +
This is localhost.unknown_domain (Linux x86_64 2.6.38-xen-maiwald.tk-domU) 17:27:40
 +
 +
localhost login:
 +
</pre>
 +
 +
=== Finalizing the setup ===
 +
Now we test if we can reach the DomU from our Desktop:
 +
<console>
 +
(2034)-~% ssh -lroot 217.x.x.211 
 +
The authenticity of host '217.x.x.211 (217.x.x.211)' can't be established.
 +
RSA key fingerprint is 22:e3:xxxxxxxx:b0:3c:xxxxx:d6:e5:c3.
 +
Are you sure you want to continue connecting (yes/no)? yes
 +
Warning: Permanently added '217.x.x.211' (RSA) to the list of known hosts.
 +
Enter passphrase for key '/home/mm/.ssh/id_rsa':
 +
localhost ~ # uname -a
 +
Linux localhost 2.6.38-xen-maiwald.tk-domU #4 SMP Wed Feb 8 17:30:33 CET 2012 x86_64 Intel(R) Xeon(R) CPU E3110 @ 3.00GHz GenuineIntel GNU/Linux
 +
localhost ~ #
 +
</console>
 +
All seems good for now.
 +
 +
Now switch back to the Funto [[Installation (Tutorial)|Installation Tutorial]] and go on with setting up your new domU guest like a normal funtoo linux system!
 +
 +
'''Please consider to help supporting this Wiki with editing this page and keeping it current!'''
  
{{GLW|src=http://wiki.gentoo.org/wiki/Clang}}
+
Funtoo is a perfect Xen Host and I can really recommend it to everybody as an alternative to .deb/.rpm Systems.
  
[[Category:HOWTO]]
+
Have fun!
 +
[[Category:Virtualization]]
 +
[[Category:Featured]]

Revision as of 02:16, 31 January 2014

Funtoo Xen Fun We are talking about Xen on Funtoo Linux and how to setup Xen virtualization properly. Especially, we are going to show you how much fun it is to work with Xen hosts and domU's and setting up a Funtoo Xen Server without general clicky GUI's or other frontends. This is true hardcore OS Xen setup especially for NOC server systems, headless servers, etc..

Funtoo Xen Server with paravirt funtoo domU

Assumptions We build a 64bit headless XEN hypervisor rockstable and rocket fast with a funtoo headless 64bit paravirt domU. We are not building Xen with pvgrub or hvm (which is kinda slow and overhead as long as you don't want to install Windoze).

Buiding Funtoo Xen Host Dom0

Most of the necessary steps are covered in the Installation Tutorial. We only do outline here the steps that are necessary to enjoy an easy and successful Dom0 setup or if something differs from the normal installation tutorial.

Please, open in a second tab the Installation Tutorial and follow in both carefully the next steps!

Basic Funtoo Xen Host Dom0 setup

I recommend you use only stable packages for the host dom0 !

Please consider the decision carefully. I can't stress out enough, you will avoid a lot of problems taking the stable distrib as dom0. The domU guests could be either unstable or hardened, as you wish! There comes the true fun part ;-) That's why I first edit my make.conf befor building anything!

Here is how I set up the system basics: Disk is /dev/sda

/dev/sda1 is our / partition ca 20GB ext4 
/dev/sda2 is our swap partition ca 4GB
/dev/sda3 holds the lvm volume group vgxen

I am using volume groups over raid - which I strongly advice to everybody.

Store of xen stuff:

/etc/xen/ --> xend configuration files
/xen/configs/ --> my xen domU configuration files folder 
/xen/kernel/ --> my xen domU kernel folder 
/xen/disks/ --> my xen domU image files folder

Edit /etc/rc.conf and uncomment the line at the bottom for rc_sys

rc_sys="xen0"

Configure and Build Xen Dom0 Kernel

emerge gentoo-sources
cd /usr/src/linux
make menuconfig

These settings are current as of 3.2.1-gentoo-r2, other versions may vary:

General setup  --->
   <*> Kernel .config support
      [*]   Enable access to .config through /proc/config.gz

Processor type and features  ---> 
   [*] Paravirtualized guest support  ---> 
      [*]   Xen guest support

Bus options (PCI etc.)  --->
   [*]   Xen PCI Frontend   

[*] Networking support  --->
   Networking options  --->
      <*> 802.1d Ethernet Bridging

Device Drivers  ---> 
   [*] Block devices (NEW)  --->
      <M>   DRBD Distributed Replicated Block Device support
      < >   Xen virtual block device support
      <*>   Xen block-device backend driver

Device Drivers  ---> 
   [*] Network device support  ---> 
      < >   Xen network device frontend driver 
      <*>   Xen backend network device

Device Drivers  --->
   Graphics support  --->
      -*- Support for frame buffer devices  ---
         < >   Xen virtual frame buffer support

Device Drivers  ---> 
   Xen driver support  ---> 
      [*] Xen memory balloon driver (NEW) 
      [*]   Scrub pages before returning them to system (NEW) 
      <*> Xen /dev/xen/evtchn device (NEW) 
      [*] Backend driver support (NEW) 
      <*> Xen filesystem (NEW) 
      [*]   Create compatibility mount point /proc/xen (NEW) 
      [*] Create xen entries under /sys/hypervisor (NEW) 
      <M> userspace grant access device driver (NEW) 
      <M> User-space grant reference allocator driver (NEW) 
      <M> xen platform pci device driver (NEW)

File systems  --->
   < > Ext3 journalling file system support
   <*> The Extended 4 (ext4) filesystem
   [*]   Use ext4 for ext2/ext3 file systems (NEW)
   [*]   Ext4 extended attributes (NEW)
  • Don't forget to add the required drivers for your networking and sata cards
  • If you use RAID, make sure to add the correct CONFIG_MD_RAID* entries to your config
make
make modules_install

If you experience issues with connecting to the console ensure the module "xen_gntdev" (userspace grant access device driver) is loaded before the xenconsoled process is started (you may have to restart it after loading the module).

Configuring Grub

Work has been completed to automatically enable Xen Grub entries, so after you copy your dom0 kernel edit your /etc/boot.conf as follows:

"Funtoo on Xen" {
  type xen
  xenkernel xen.gz
  xenparams loglvl=all guest_loglvl=all xsave=1 iommu=1 iommu_inclusive_mapping=1 dom0_max_vcpus=2 dom0_vcpus_pin dom0_mem=4096M 
  kernel kernel[-v]
  params += quiet
}

To note a fiew things iommu is the paravirtualized instructions, if your motherboard or CPU does not support VT-d do not enable it. Xsave saves the supported CPU instruction sets, without it you're dom0 kernel may not boot. And finally dom0_vcpus_pin permanatly assigns cpu's to dom0, increasing performance.

Basic Networking with the Dom0

Funtoo Linux offers its own modular, template-based network configuration system. This system offers a lot of flexibility for configuring network interfaces, essentially serving as a "network interface construction kit."

There is given eth0 and eth1. We are going to set eth0 as the default interface to the outside world for now. eth1 will be part of a bridge (xenbr0) that is going to be used by various domU guests.

Construct the interfaces:

cd /etc/init.d/
ln -s netif.tmpl netif.xenbr0
ln -s netif.tmpl netif.extbr0
ln -s netif.tmpl netif.eth0
ln -s netif.tmpl netif.eth1
rc-update add netif.xenbr0 sysinit
rc-update add netif.extbr0 sysinit

Make sure dhcpcd, eth0 and eth1 don't start at boot:

rc-update del dhcpcd sysinit
rc-update del netif.eth0 sysinit
rc-update del netif.eth1 sysinit

Configure the slave interfaces:

cd /etc/conf.d/
echo 'template="interface-noip"' > netif.eth0
echo 'template="interface-noip"' > netif.eth1

Then we are going to prepare the bridges.

nano netif.xenbr0

here we set the internal Xen bridge:

template="bridge"
ipaddr="10.0.1.200/24"
gateway="10.0.1.1"
nameservers="10.0.1.1 10.0.1.2"
domain="funtoo.org"
slaves="netif.eth0"

Then we are setting up the external interface:

nano netif.extbr0

this is looking quiet similar, please watch out for the correct slave setting!

template="bridge"
ipaddr="10.0.1.201/24"
gateway="10.0.1.1"
nameservers="10.0.1.1 10.0.1.2"
domain="funtoo.org"
slaves="netif.eth1"

This gives us the possibility to play around with various setups later, it's modular and easy to tweak and change.

Better you invoke with "rc" on the command line the openrc script and test if the network cards get initialized correctly.

Basic Networking with domU

The easiest way is to let Xen set up the networking. But if everything is up and running it is not possible to change the routings, etc. Letting Xen do the bridges will be obsolete in the near future. So this is not the recommended way anymore. As we already set up the bridges in the previous section it may be enough to comment everything network related. If not, just un-comment the last lines.

We edit the /etc/xen/xend-config.sxp

#### Xen config from maiwald.tk - Xen 4.x Network in bridge mode

(logfile /var/log/xen/xend.log)
(loglevel DEBUG)

(xend-relocation-server no)
(xend-relocation-hosts-allow '^localhost$ ^localhost\\.localdomain$')

# The limit (in kilobytes) on the size of the console buffer
(console-limit 1024)

(dom0-min-mem 384)
(enable-dom0-ballooning no)

(total_available_memory 0)
(dom0-cpus 0)

(vncpasswd 'geheim')

# let xen create the net
# (network-script    network-bridge)
# (vif-script        vif-bridge)

# we create the net - new default in Xen 4
#
#(network-script 'network-bridge netdev=eth0 bridge=xenbr0 vifnum=0')
#(vif-script vif-bridge bridge=xenbr0)

Building the Funtoo Xen DomU Container

We are going to build the DomU now, preparing first from outside the domU.

create lvm volume or partition or image file

This is a stub, please help completing this guide here!

# vgcreate vgxen /dev/sda3
# lvcreate -L10G -n funtoo_root vgxen
# lvcreate -L1G -n funtoo_swap vgxen
# vgchange -a y
# mkfs.ext4 -L funtoo_root /dev/vgxen/funtoo_root
# mkswap -L funtoo_swap /dev/vgxen/funtoo_swap
# rc-update add lvm boot

Basic DomU System setup

mount domU lvm volume or physical partition or image file

# mkdir /mnt/domu1
# mount /dev/vgxen/funtoo_root /mnt/domu1
# cd /mnt/domu1

get stage3

from a funtoo mirror near you, I suggest you look at the funtoo homepage

# links http://www.funtoo.org/wiki/Download 

Then choose a mirror near you ( I use Heanet in EU ) and look for the right stage3. I use XEON CPUs so I take the core2 distrib:

# wget -cv http://ftp.heanet.ie/mirrors/funtoo/funtoo-stable/x86-64bit/core2_64/stage3-latest.tar.xz 

Unfortunately I can't find md5sums or similar which is really unpleasant.

then get latest portage tree from the snapshots firectory

# wget -cv http://ftp.heanet.ie/mirrors/funtoo/funtoo-stable/snapshots/portage-current.tar.xz 

extract stage3

# tar xpf stage3-current.tar.xz

extract portage

# cd usr
# tar xf ../portage-current.tar.xz 

Preparing the chroot environment

Editing the make.conf

copy /etc/make.conf from dom0 and adjust it

# cp /etc/portage/make.conf /mnt/domu1/etc/

make sure to adjust MAKEOPTS to your assigned CPUs (rule of thumb cpu cores +1 - yes, even in XEN)

# nano -w /mnt/domu1/etc/portage/make.conf

out there the MAKEOPTS variable in:

MAKEOPTS="-j2"

copy /etc/resolv.conf

# cp -L /etc/resolv.conf /mnt/domu1/etc/ 

mount proc and dev

# mount -t proc none /mnt/domu1/proc
# mount --rbind /dev /mnt/domu1/dev 

Building Funtoo Xen Guest(s) DomU

Final DomU System setup

chroot

# chroot /mnt/domu1 /bin/bash
# env-update
# source /etc/profile
# export PS1="(domU-chroot) $PS1" 

sync portage

 
# emerge --sync

set locales

# nano -w /etc/locale.gen
# locale-gen

set your timezone

(choose your timezone in /usr/share/zoneinfo)

# cp -L /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime

edit /etc/fstab (see also gentoo handbook as reference)

we assume that we name our root partition xvda1 and the swap partition xvda2 in our domU-xen-config (we will do that later)

# nano -w /etc/fstab
/dev/xvda1      /              ext4    noatime 0 1
/dev/xvda2      none           swap    sw      0 0
shm             /dev/shm       tmpfs   nodev,nosuid,noexec     0 0

Now comes the most important stuff

please just copy this into your terminal:

echo '
                         Larry loves Funtoo
                      _________________________
                      < Have you mooed today? >
                      -------------------------
                        \   ^__^
                         \  (oo)\_______
                            (__)\       )\/\
                                 ||----w |
                                 ||     ||
.::::::::::::::: WELCOME TO ^^^^^^^^^^^^^^^^^^^:::::::::::::..
...............................................................
:########:'##::::'##:'##::: ##:'########::'#######:::'#######::.
:##.....:: ##:::: ##: ###:: ##:... ##..::'##.... ##:'##.... ##::
:##::::::: ##:::: ##: ####: ##:::: ##:::: ##:::: ##: ##:::: ##::
:######::: ##:::: ##: ## ## ##:::: ##:::: ##:::: ##: ##:::: ##::
:##...:::: ##:::: ##: ##. ####:::: ##:::: ##:::: ##: ##:::: ##::
:##::::::: ##:::: ##: ##:. ###:::: ##:::: ##:::: ##: ##:::: ##::
:##:::::::. #######:: ##::. ##:::: ##::::. #######::. #######::′
.::::::::::.......:::..::::..:::::..::::::.......::::.......::´ 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
'> /etc/motd

We are using the echo instead of „emerge --moo „ as larry still moo's in gentoo'ish

So that's it - almost.

adding networking to the domU:

(domU-chroot) # cd /etc/init.d/
(domU-chroot) # ln -sf netif.tmpl netif.eth0
(domU-chroot) # rc-update add netif.eth0
 * service netif.eth0 added to runlevel sysinit

Now we are ready to do the last setups

(domU-chroot) # emerge eix

After that I am feeling better now, I do the rest..

(domU-chroot) # eix-update 
Reading Portage settings ..
Building database (/var/cache/eix) ..
[0] "gentoo" /usr/portage/ (cache: metadata-md5-or-flat)
     Reading category 154|154 (100%) Finished             
Applying masks ..
Calculating hash tables ..
Writing database file /var/cache/eix ..
Database contains 15729 packages in 154 categories.

(domU-chroot) # exit 
exit

From here you have to decide how you want to run your domU: with unpriviledged users and sudo ? or with a root account enabled? as a webserver or firewall?

Me, personally, I always do install openssh server and just place my ssh keys in there. From there the steps differ.

(dom0-xen) # cp /root/.ssh/authorized_keys /mnt/domu1/root/.ssh/

Also, don't forget here to enable PubKeyAuth in your sshd_config in your domU and set PermitRootLogin to yes!

Double checking! Does your domU kernel uses modules or not? If you haven't built a monolitic kernel you should copy the modules from the dom0 to the domU now:

(dom0-xen) # mkdir /mnt/domu1/lib/modules
(dom0-xen) # rsync -aP /lib/modules/2.6.38-xen-maiwald.tk-dom0 /mnt/domu1/lib/modules/

Ok, that's it from here..

Don't forget to clean up the mounts!

(dom0-xen) # cd
(dom0-xen) # umount -l /mnt/domu1/proc
(dom0-xen) # umount -l /mnt/domu1/dev
(dom0-xen) # umount -l /mnt/domu1

That's it! ;-)

Booting the Xen DomU Guest

Ok, let's try the first boot of the newly created Xen DomU in Funtoo!

(dom0-xen) # cd /xen
(dom0-xen) # xm create -c configs/funtoo.cfg

Huuuuiiiii......

Using config file "./configs/funtoo.cfg".
Started domain funtoo (id=4)
[    0.000000] Linux version 2.6.38-xen-maiwald.tk-domU (root@xen) (gcc version 4.4.5 (Gentoo 4.4.5 p1.0, pie-0.4.5) ) #4 SMP Wed Feb 8 17:30:33 CET 2012
[    0.000000] Command line: root=/dev/xvda1 ro ip=217.x.x.211:127.0.255.255:217.x.x.1:255.255.255.0:domU:eth0:off xencons=tty console=xvc0 raid=noautodetect
[    0.000000] Xen-provided physical RAM map:
[    0.000000]  Xen: 0000000000000000 - 0000000040800000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] last_pfn = 0x40800 max_arch_pfn = 0x80000000
[    0.000000] init_memory_mapping: 0000000000000000-0000000040800000
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000000 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   empty
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[2] active PFN ranges
[    0.000000]     0: 0x00000000 -> 0x00040000
[    0.000000]     0: 0x00040800 -> 0x00040800
[    0.000000] setup_percpu: NR_CPUS:16 nr_cpumask_bits:16 nr_cpu_ids:1 nr_node_ids:1
[    0.000000] PERCPU: Embedded 18 pages/cpu @ffff88003efc0000 s42304 r8192 d23232 u73728
[    0.000000] Swapping MFNs for PFN 6d6 and 3efc7 (MFN 15deb0 and 1223bf)
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 256109
[    0.000000] Kernel command line: root=/dev/xvda1 ro ip=217.171.190.211:127.0.255.255:217.171.190.1:255.255.255.0:alyx1:eth0:off xencons=tty console=xvc0 raid=noautodetect
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.000000] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.000000] Software IO TLB disabled
[    0.000000] Memory: 1022732k/1056768k available (3657k kernel code, 8192k absent, 25844k reserved, 1261k data, 264k init)
[    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS:96
[    0.000000] Xen reported: 2992.570 MHz processor.
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty-1] enabled
[    0.150003] Calibrating delay using timer specific routine.. 6018.63 BogoMIPS (lpj=30093193)
[    0.150008] pid_max: default: 32768 minimum: 301
[    0.150034] Mount-cache hash table entries: 256
[    0.150173] SMP alternatives: switching to UP code
[    0.170232] Freeing SMP alternatives: 20k freed
[    0.170342] Brought up 1 CPUs
[    0.170377] devtmpfs: initialized
[    0.170601] xor: automatically using best checksumming function: generic_sse
[    0.220004]    generic_sse:  7325.200 MB/sec
[    0.220008] xor: using function: generic_sse (7325.200 MB/sec)
[    0.220091] NET: Registered protocol family 16
[    0.220186] Brought up 1 CPUs
[    0.220217] bio: create slab <bio-0> at 0
[    0.390014] raid6: int64x1   2353 MB/s
[    0.560003] raid6: int64x2   2964 MB/s
[    0.730026] raid6: int64x4   2357 MB/s
[    0.900012] raid6: int64x8   2116 MB/s
[    1.070007] raid6: sse2x1    5349 MB/s
[    1.240009] raid6: sse2x2    5404 MB/s
[    1.410005] raid6: sse2x4    8597 MB/s
[    1.410008] raid6: using algorithm sse2x4 (8597 MB/s)
[    1.410022] suspend: event channel 6
[    1.410022] xen_mem: Initialising balloon driver.
[    1.410096] Switching to clocksource xen
[    1.410125] FS-Cache: Loaded
[    1.410152] CacheFiles: Loaded
[    1.410268] NET: Registered protocol family 2
[    1.410288] IP route cache hash table entries: 32768 (order: 6, 262144 bytes)
[    1.410391] TCP established hash table entries: 131072 (order: 9, 2097152 bytes)
[    1.410951] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    1.411180] TCP: Hash tables configured (established 131072 bind 65536)
[    1.411183] TCP reno registered
[    1.411186] UDP hash table entries: 512 (order: 2, 16384 bytes)
[    1.411192] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
[    1.411229] NET: Registered protocol family 1
[    1.411290] platform rtc_cmos: registered platform RTC device (no PNP device found)
[    1.411401] Intel AES-NI instructions are not detected.
[    1.411437] audit: initializing netlink socket (disabled)
[    1.411444] type=2000 audit(1330014455.606:1): initialized
[    1.412612] fuse init (API version 7.16)
[    1.412674] msgmni has been set to 2048
[    1.412990] NET: Registered protocol family 38
[    1.413018] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[    1.413024] io scheduler noop registered (default)
[    1.413026] io scheduler deadline registered
[    1.413049] io scheduler cfq registered
[    1.413079] Non-volatile memory driver v1.3
[    1.413088] Hangcheck: starting hangcheck timer 0.9.1 (tick is 180 seconds, margin is 60 seconds).
[    1.413090] Hangcheck: Using getrawmonotonic().
[    1.419520] Switched to NOHz mode on CPU #0
[    1.423394] brd: module loaded
[    1.423665] loop: module loaded
[    1.423771] nbd: registered device at major 43
[    1.426180] Xen virtual console successfully installed as tty1
[    1.426216] Event-channel device installed.
[    1.441658] netfront: Initialising virtual ethernet driver.
[    1.444972] xen-vbd: registered block device major 202
[    1.444988] blkfront: xvda1: barriers enabled
[    1.450287] Setting capacity to 20971520
[    1.450294] xvda1: detected capacity change from 0 to 10737418240
[    1.450677] blkfront: xvda2: barriers enabled
[    1.451661] Setting capacity to 2097152
[    1.451665] xvda2: detected capacity change from 0 to 1073741824
[    1.452020] bonding: Ethernet Channel Bonding Driver: v3.7.0 (June 2, 2010)
[    1.452023] bonding: Warning: either miimon or arp_interval and arp_ip_target module parameters must be specified, otherwise bonding will not detect link failures! see bonding.txt for details.
[    1.453016] i8042: No controller found
[    1.453066] mousedev: PS/2 mouse device common for all mice
[    1.453113] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[    1.453145] rtc_cmos: probe of rtc_cmos failed with error -38
[    1.453155] md: linear personality registered for level -1
[    1.453158] md: raid0 personality registered for level 0
[    1.453161] md: raid1 personality registered for level 1
[    1.453163] md: raid6 personality registered for level 6
[    1.453166] md: raid5 personality registered for level 5
[    1.453168] md: raid4 personality registered for level 4
[    1.453224] device-mapper: uevent: version 1.0.3
[    1.453273] device-mapper: ioctl: 4.19.1-ioctl (2011-01-07) initialised: dm-devel@redhat.com
[    1.453340] device-mapper: multipath: version 1.2.0 loaded
[    1.453343] device-mapper: multipath round-robin: version 1.0.0 loaded
[    1.453345] device-mapper: multipath queue-length: version 0.1.0 loaded
[    1.453347] device-mapper: multipath service-time: version 0.2.0 loaded
[    1.453396] Netfilter messages via NETLINK v0.30.
[    1.453410] nf_conntrack version 0.5.0 (8192 buckets, 32768 max)
[    1.453478] ctnetlink v0.93: registering with nfnetlink.
[    1.453486] IPv4 over IPv4 tunneling driver
[    1.453548] TCP westwood registered
[    1.453550] TCP highspeed registered
[    1.453552] TCP htcp registered
[    1.453553] TCP vegas registered
[    1.453555] Initializing XFRM netlink socket
[    1.453630] NET: Registered protocol family 10
[    1.453803] IPv6 over IPv4 tunneling driver
[    1.453863] NET: Registered protocol family 17
[    1.453868] NET: Registered protocol family 15
[    1.453870] Registering the dns_resolver key type
[    1.550094] /usr/src/linux-2.6.38-xen/drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
[    3.070104] IP-Config: Complete:
[    3.070109]      device=eth0, addr=217.171.190.211, mask=255.255.255.0, gw=217.171.190.1,
[    3.070116]      host=alyx1, domain=, nis-domain=(none),
[    3.070119]      bootserver=127.0.255.255, rootserver=127.0.255.255, rootpath=
[    3.070212] md: Skipping autodetection of RAID arrays. (raid=autodetect will force)
[    3.107309] EXT4-fs (xvda1): mounted filesystem with ordered data mode. Opts: (null)
[    3.107321] VFS: Mounted root (ext2 filesystem) readonly on device 202:1.
[    3.140059] devtmpfs: mounted
[    3.140239] Freeing unused kernel memory: 264k freed
INIT: version 2.88 booting

   OpenRC 0.8.3 is starting up Funtoo Linux (x86_64)

 * Mounting /proc ...
 [ ok ]
 * WARNING: rc_sys not defined in rc.conf. Falling back to automatic detection
 * Caching service dependencies ...
 [ ok ]
 * Mounting /sys ...
 [ ok ]
 * udev: /dev already mounted, skipping...
 * Mounting /dev/pts ...
 [ ok ]
 * Mounting /dev/shm ...
 [ ok ]
 * Bringing up network interface lo ...
RTNETLINK answers: File exists
 [ ok ]
 * Bringing up network interface lo ...
RTNETLINK answers: File exists
RTNETLINK answers: File exists
 [ ok ]
 * Starting udevd daemon ...
 * Populating /dev with existing devices through uevents ...
 [ ok ]
 * Autoloaded 0 module(s)
 * Checking local filesystems  ...
funtoo_root: Superblock last write time is in the future.
        (by less than a day, probably due to the hardware clock being incorrectly set).  FIXED.
funtoo_root: clean, 173796/655360 files, 436917/2621440 blocks
 [ ok ]
 * Remounting root filesystem read/write ...
 [ ok ]
 * Updating /etc/mtab ...
 [ ok ]
 * Mounting local filesystems ...
 [ ok ]
 * Configuring kernel parameters ...
 [ ok ]
 * Creating user login records ...
 [ ok ]
 * Cleaning /var/run ...
 [ ok ]
 * Wiping /tmp directory ...
 [ ok ]
 * Setting hostname to localhost ...
 [ ok ]
 * Activating swap devices ...
 [ ok ]
 * udev: storing persistent rules ...
 [ ok ]
 * Initializing random number generator ...
 [ ok ]
INIT: Entering runlevel: 3
 * Mounting network filesystems ...
 [ ok ]
 * Generating dsa host key ...
Generating public/private dsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_dsa_key.
Your public key has been saved in /etc/ssh/ssh_host_dsa_key.pub.
The key fingerprint is:
25:e0:a8:05:xxxxxxxxxxxx:1c:1f:ba root@localhost
The key's randomart image is:
+--[ DSA 1024]----+
|  ooo.B.o        |
| o o *.B o .     |
|  . + + = =      |
|   o   + *       |
|  .   E S        |
|                 |
|                 |
|                 |
|                 |
+-----------------+
 [ ok ]
 * Generating rsa host key ...
Generating public/private rsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_rsa_key.
Your public key has been saved in /etc/ssh/ssh_host_rsa_key.pub.
The key fingerprint is:
22:e3:46:28:67:xxxxxxxxxxxxxxxxxxxxx:e5:c3 root@localhost
The key's randomart image is:
+--[ RSA 2048]----+
|.    o. ..       |
|oo  o ..o        |
|=oo  o   E       |
|.*oo.     .      |
|o *.+ . S        |
| + o o .         |
|    o            |
|   .             |
|                 |
+-----------------+
 [ ok ]
 * Starting sshd ...
 [ ok ]
 * Starting local
 [ ok ]


                         Larry loves Funtoo
                      _________________________
                      < Have you mooed today? >
                      -------------------------
                           ^__^
                           (oo)_______
                            (__)       )/
                                 ||----w |
                                 ||     ||
 .::::::::::::::::::::: WELCOME TO ::::::::::::::::::::::::::..
 ...............................................................
 :########:'##::::'##:'##::: ##:'########::'#######:::'#######::.
 :##.....:: ##:::: ##: ###:: ##:... ##..::'##.... ##:'##.... ##::
 :##::::::: ##:::: ##: ####: ##:::: ##:::: ##:::: ##: ##:::: ##::
 :######::: ##:::: ##: ## ## ##:::: ##:::: ##:::: ##: ##:::: ##::
 :##...:::: ##:::: ##: ##. ####:::: ##:::: ##:::: ##: ##:::: ##::
 :##::::::: ##:::: ##: ##:. ###:::: ##:::: ##:::: ##: ##:::: ##::
 :##:::::::. #######:: ##::. ##:::: ##::::. #######::. #######::′
.::::::::::.......:::..::::..:::::..::::::.......::::.......::´
This is localhost.unknown_domain (Linux x86_64 2.6.38-xen-maiwald.tk-domU) 17:27:40

localhost login: 

Finalizing the setup

Now we test if we can reach the DomU from our Desktop:

(2034)-~% ssh -lroot 217.x.x.211  
The authenticity of host '217.x.x.211 (217.x.x.211)' can't be established.
RSA key fingerprint is 22:e3:xxxxxxxx:b0:3c:xxxxx:d6:e5:c3.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '217.x.x.211' (RSA) to the list of known hosts.
Enter passphrase for key '/home/mm/.ssh/id_rsa': 
localhost ~ # uname -a
Linux localhost 2.6.38-xen-maiwald.tk-domU #4 SMP Wed Feb 8 17:30:33 CET 2012 x86_64 Intel(R) Xeon(R) CPU E3110 @ 3.00GHz GenuineIntel GNU/Linux
localhost ~ # 

All seems good for now.

Now switch back to the Funto Installation Tutorial and go on with setting up your new domU guest like a normal funtoo linux system!

Please consider to help supporting this Wiki with editing this page and keeping it current!

Funtoo is a perfect Xen Host and I can really recommend it to everybody as an alternative to .deb/.rpm Systems.

Have fun!