Difference between pages "KVM" and "Maximum Swappage"

(Difference between pages)
(Changed qemu-kvm to qemu (http://forums.funtoo.org/viewtopic.php?id=1657))
 
(Created page with "{{Article |Subtitle=Getting the most out of swap |Summary=Learn how to improve the swap performance on your Linux server by several orders of magnitude. Author Daniel Robbins...")
 
Line 1: Line 1:
 +
{{Article
 +
|Subtitle=Getting the most out of swap
 +
|Summary=Learn how to improve the swap performance on your Linux server by several orders of magnitude. Author Daniel Robbins takes you through this quick tip on getting the most from your server.
 +
|Author=Drobbins
 +
}}
 +
When you set up a brand new Linux server, do you create a single 128 MB swap partition? If so, did you know that you are severely limiting swap performance? Would you like to increase swap performance by several orders of magnitude, and to create swap partitions larger than 1 GB? It's possible, requiring no kernel patches or special hardware, just pure geek know-how!
  
== Introduction ==
+
Some of you may not really care about swap. After all, Linux systems are typically very memory efficient, and swap is often barely touched. While often true on desktop systems, servers are another story. Because servers may handle unexpected stresses, such as runaway processes, denial of service attacks, or even the Slashdot effect, they need to have adequate high-speed swap so that they do not grind to a halt and possibly crash when all physical memory (and then some) is exhausted.
  
KVM is a hardware-accelerated full-machine hypervisor and virtualization solution included as part of kernel 2.6.20 and later. It allows you to create and start hardware-accelerated virtual machines under Linux using the QEMU tools.
+
Still not convinced that this is a big deal? I'll show you how easy it is to bring down a server by launching a massive amount of new processes.
  
== Kernel Setup ==
+
{{Warning|Please, if you try this, do it only on a non-production server that you actually administer!}}
  
To enable KVM, the following kernel config parameters should be enabled (this is based on a 3.x kernel):
+
Let's say you have two customized grep commands in /usr/bin, called bobgrep and jimgrep. Now, let's assume that bobgrep is simply a shell script that calls the ELF executable jimgrep, as follows:
  
Under <tt>Processor type and features</tt>, enable <tt>Paravirtualized Guest Support</tt>. Under the <tt>Paravirtualized Guest Support</tt> menu, enable any options related to KVM, such as <tt>KVM paravirtualized clock</tt> and in particular <tt>KVM Guest Support</tt>.
+
{{file|lang=bash|desc=The bobgrep script|name=bobgrep|body=
 
+
Under the <tt>Virtualization</tt> category from the main kernel config menu, enable <tt>Kernel-based Virtual Machine (KVM) support</tt>, and enable at least one type of KVM, either for Intel or AMD processors. It is also recommended to enable <tt>Host kernel acceleration for virtio net</tt>.
+
 
+
You can use modules or build these parts directly into the kernel. Build your new kernel and modules, and reboot.
+
 
+
== User-space tools ==
+
 
+
KVM is essentially a kernel-accelerated version of QEMU. To enable KVM support in the user-space tools, add the following lines to <tt>/etc/make.conf</tt>:
+
 
+
<pre>
+
QEMU_SOFTMMU_TARGETS="i386 x86_64"
+
QEMU_USER_TARGETS="i386 x86_64"
+
</pre>
+
 
+
Once the <tt>make.conf</tt> variables above are set, emerge qemu:
+
 
+
<console>
+
# ##i## emerge qemu
+
</console>
+
 
+
==Initial Setup==
+
 
+
Prior to using KVM, modprobe the appropriate accelerated driver for Intel or AMD:
+
 
+
<console>
+
# ##i##modprobe kvm_intel
+
</console>
+
 
+
== Starting your first KVM virtual machine ==
+
 
+
To start your first KVM virtual machine, first download SysRescueCD and save it to systemrescuecd.iso. Then use the following commands, which will create a 10GB qcow disk image to use for the first disk, and then the next command will start your virtual machine, booting from the CD:
+
 
+
<console>
+
# ##i##qemu-img create -f qcow2 vdisk.qcow2 10
+
# ##i##qemu-system-x86_64 vdisk.qcow2 -m 1024 -cdrom systemrescuecd.iso  -vnc 127.0.0.1:1 -cpu host -net nic -net user
+
VNC server running on `127.0.0.1:5900'
+
</console>
+
 
+
Now you should be able to use a VNC client to connect to 127.0.0.1:5901 (VNC session 1) and access your virtual machine.
+
 
+
== Networking Options ==
+
 
+
Above, networking will be enabled but will be on its own private LAN, and ping will not work. If you have a local bridge that you use for networking, the following steps will allow you use your existing bridge to provide higher-performance and full-featured network access to your virtual machine.
+
 
+
First, create <tt>/etc/qemu-ifup</tt> and add the following to it. Replace <tt>brlan</tt> with the name of your bridge:
+
 
+
<syntaxhighlight lang="bash">
+
 
#!/bin/bash
 
#!/bin/bash
ifconfig $1 0.0.0.0 promisc up
+
jimgrep -r $*
brctl addif brlan $1
+
}}
sleep 2
+
</syntaxhighlight>
+
 
+
Make it executable:
+
 
+
<console>
+
# ##i##chmod +x /etc/qemu-ifup
+
</console>
+
  
Start the virtual machine as follows:
+
Everything looks good so far, but what happens if jimgrep gets accidentally replaced with a symbolic link to bobgrep? Well, in that case, calling bobgrep or jimgrep will cause an infinite loop, causing hundreds of bash processes to be spawned in mere seconds. This actually happened to me once, and believe me, it hurt!
  
<console>
+
If a server doesn't have adequate swap, a situation like this can cause the machine to lock up in much less than a minute. How do we fix the problem? One way is to increase the swap size beyond 128 MB. With Linux 2.0, there was a swap limit of 128MB on x86 systems. With 2.2, this limit was raised to 2GB. Currently, with Linux 3.x kernels, the limit is about 17.1TB, so you are unlikely to find yourself unable to allocate enough swap.
# ##i##qemu-system-x86_64 vdisk.qcow2 -m 1024 -cdrom systemrescuecd-x86-2.8.0.iso -cpu host -vnc 127.0.0.1:1 -net nic -net tap,id=foo
+
</console>
+
  
== Tweaking KVM ==
+
While it's nice to be able to increase swap partition size to beyond 128 MB, how about increasing performance? Ideally, it would be nice if we could set up swap partitions in a RAID 0 stripe, so that reads and writes are equally distributed between all partitions. If these partitions are on separate drives and/or controllers, this will multiply swap file performance, allowing your servers to handle temporary memory usage "spikes" without getting dramatically bogged down.
  
=== VNC Output ===
+
Amazingly, all modern Linux kernels, by default (with no special kernel options or patches) allow you to parallelize swap, just like a RAID 0 stripe. By using the pri option in /etc/fstab to set multiple swap partitions to the same priority, we tell Linux to use them in parallel:
  
If you wanted to have VNC listen on a different IP address or port, you can use the format <tt>-vnc IP:vncnum</tt> which will cause VNC to listen on the IP specified, and the TCP port 5900+vncnum.
+
{{file|name=/etc/fstab|desc=Set multiple swap partitions to the same priority|body=
 +
/dev/sda2      none    swap    sw,pri=3        0      0
 +
/dev/sdb2      none    swap    sw,pri=3        0      0
 +
/dev/sdc2      none    swap    sw,pri=3        0      0
 +
/dev/sdd2      none    swap    sw,pri=1        0      0
 +
}}
  
=== CPU Settings ===
+
In the above example, Linux will use swap partitions sda2, sdb2, and sdc2 in parallel. Since these partitions are on different drives, and possibly even different SCSI controllers, read and write throughput will nearly triple. The fourth swap partition, sdd2, will be used only after the first three partitions have been exhausted.
  
By default, the KVM guest will have one CPU with one core. To change this, use <tt>-cpu host</tt> (to export all of the host's CPU features) and <tt>-smp cores=X,threads=Y</tt>, where X is the number of cores, and Y is the number of threads on each core. You can emulate more CPUs and cores than you actually have.
+
The pri option is really easy to use. The priority must be a number between 0 and 32767, with 32767 being the highest priority. The swap partitions will be used from highest priority to lowest priority, meaning that a partition with a priority of x will only be used only if all partitions with a priority greater than x are already full. If several partitions have the same priority, Linux will automatically parallelize access between them. This allows you to not only parallelize swap, but also prioritize access so that the partitions on the fastest drives (or regions of the drives) are used first. So, you can set up an emergency swap partition on an old, slower drive that will be used only if all high-speed swap is exhausted first.
  
[[Category:Virtualization]]
+
Now it's time to put some of this swapping knowledge into action. To loosely quote Mr. Miyagi of Karate Kid fame: "Swap on, swap off, geek-san!"
[[Category:OpenStack]]
+
{{ArticleFooter}}
[[Category:Featured]]
+

Latest revision as of 00:08, January 2, 2015

Getting the most out of swap

Learn how to improve the swap performance on your Linux server by several orders of magnitude. Author Daniel Robbins takes you through this quick tip on getting the most from your server.

Support Funtoo and help us grow! Donate $15 per month and get a free SSD-based Funtoo Virtual Container.

When you set up a brand new Linux server, do you create a single 128 MB swap partition? If so, did you know that you are severely limiting swap performance? Would you like to increase swap performance by several orders of magnitude, and to create swap partitions larger than 1 GB? It's possible, requiring no kernel patches or special hardware, just pure geek know-how!

Some of you may not really care about swap. After all, Linux systems are typically very memory efficient, and swap is often barely touched. While often true on desktop systems, servers are another story. Because servers may handle unexpected stresses, such as runaway processes, denial of service attacks, or even the Slashdot effect, they need to have adequate high-speed swap so that they do not grind to a halt and possibly crash when all physical memory (and then some) is exhausted.

Still not convinced that this is a big deal? I'll show you how easy it is to bring down a server by launching a massive amount of new processes.

Warning

Please, if you try this, do it only on a non-production server that you actually administer!

Let's say you have two customized grep commands in /usr/bin, called bobgrep and jimgrep. Now, let's assume that bobgrep is simply a shell script that calls the ELF executable jimgrep, as follows:

bobgrep (bash source code) - The bobgrep script
#!/bin/bash
jimgrep -r $*

Everything looks good so far, but what happens if jimgrep gets accidentally replaced with a symbolic link to bobgrep? Well, in that case, calling bobgrep or jimgrep will cause an infinite loop, causing hundreds of bash processes to be spawned in mere seconds. This actually happened to me once, and believe me, it hurt!

If a server doesn't have adequate swap, a situation like this can cause the machine to lock up in much less than a minute. How do we fix the problem? One way is to increase the swap size beyond 128 MB. With Linux 2.0, there was a swap limit of 128MB on x86 systems. With 2.2, this limit was raised to 2GB. Currently, with Linux 3.x kernels, the limit is about 17.1TB, so you are unlikely to find yourself unable to allocate enough swap.

While it's nice to be able to increase swap partition size to beyond 128 MB, how about increasing performance? Ideally, it would be nice if we could set up swap partitions in a RAID 0 stripe, so that reads and writes are equally distributed between all partitions. If these partitions are on separate drives and/or controllers, this will multiply swap file performance, allowing your servers to handle temporary memory usage "spikes" without getting dramatically bogged down.

Amazingly, all modern Linux kernels, by default (with no special kernel options or patches) allow you to parallelize swap, just like a RAID 0 stripe. By using the pri option in /etc/fstab to set multiple swap partitions to the same priority, we tell Linux to use them in parallel:

/etc/fstab - Set multiple swap partitions to the same priority
/dev/sda2       none    swap    sw,pri=3        0       0
/dev/sdb2       none    swap    sw,pri=3        0       0
/dev/sdc2       none    swap    sw,pri=3        0       0
/dev/sdd2       none    swap    sw,pri=1        0       0

In the above example, Linux will use swap partitions sda2, sdb2, and sdc2 in parallel. Since these partitions are on different drives, and possibly even different SCSI controllers, read and write throughput will nearly triple. The fourth swap partition, sdd2, will be used only after the first three partitions have been exhausted.

The pri option is really easy to use. The priority must be a number between 0 and 32767, with 32767 being the highest priority. The swap partitions will be used from highest priority to lowest priority, meaning that a partition with a priority of x will only be used only if all partitions with a priority greater than x are already full. If several partitions have the same priority, Linux will automatically parallelize access between them. This allows you to not only parallelize swap, but also prioritize access so that the partitions on the fastest drives (or regions of the drives) are used first. So, you can set up an emergency swap partition on an old, slower drive that will be used only if all high-speed swap is exhausted first.

Now it's time to put some of this swapping knowledge into action. To loosely quote Mr. Miyagi of Karate Kid fame: "Swap on, swap off, geek-san!"


Support Funtoo and help us grow! Donate $15 per month and get a free SSD-based Funtoo Virtual Container.

Learn how to improve the swap performance on your Linux server by several orders of magnitude. Author Daniel Robbins takes you through this quick tip on getting the most from your server.
About the Author

Daniel Robbins is best known as the creator of Gentoo Linux and author of many IBM developerWorks articles about Linux. Daniel currently serves as Benevolent Dictator for Life (BDFL) of Funtoo Linux. Funtoo Linux is a Gentoo-based distribution and continuation of Daniel's original Gentoo vision.

Got Funtoo?

Have you installed Funtoo Linux yet? Discover the power of a from-source meta-distribution optimized for your hardware! See our installation instructions and browse our CPU-optimized builds.

Funtoo News

Drobbins

IP Space Migration Continues

All Funtoo user containers in the 8.28 IP space will be moving into our new IP space (172.97) over the next few days. If you have DNS set up -- be sure to watch your container and update to the new IP! container.host.funtoo.org DNS will be updated after the move.
2015-08-27 by Drobbins
Drobbins

Funtoo Hosting IP Move

Funtoo user containers with IPs in the 72.18.x.x range will be gradually migrating to new IP addresses this week. If you have DNS entries for your containers, please be aware that your DNS will need to be updated.
2015-08-11 by Drobbins
Drobbins

New ARM Stages

New ARM Stages, built with a new toolchain, are now hitting mirrors. Existing ARM users should re-install using these stages (dated Aug 3, 2015 or later,) rather than upgrade using emerge.
2015-08-06 by Drobbins
More...

More Articles

Browse all our Linux-related articles, below:

A

B

F

G

K

L

M

O

P

S

T

W

X