Learning Linux LVM, Part 2

From Funtoo
Jump to: navigation, search

The cvs.gentoo.org upgrade

In this article, Daniel shares his experiences converting cvs.gentoo.org's /home filesystem to an LVM logical volume. After the transition, we get to see the benefits of LVM when cvs.gentoo.org's /home partition is dynamically resized in real-time, without rebooting, unmounting /home, or even dropping to runlevel 1. All processes continue to work without any interruption. Daniel's step-by-step details of the conversion will help anyone interested in peforming a similar transition on their own machine.

Support Funtoo and help us grow! Donate $15 per month and get a free SSD-based Funtoo Virtual Container.


In my first LVM article, I explained the concepts behind LVM. Now it's time to put LVM into action. In this article, I'm going to set up LVM on the official Gentoo Linux cvs server, cvs.gentoo.org. Although cvs.gentoo.org has only one hard drive, LVM's flexibility still provides an incredible improvement over the standard static partitioning approach. I'll show you all the steps of the LVM conversion process, so that if you're interested you can perform a similar conversion on one of your machines.


Because implementing LVM is a major change to the system (involving the creation of new partitions and other potentially hazardous actions) it's a really good idea to perform a full system backup before beginning this process. If you're not going to perform a backup, I hope you're using a test box with no important data on it. I should mention that I didn't experience any problems while converting to LVM, but it's best to be prepared in case something goes wrong.

That said, let's continue. Cvs.gentoo.org had a nice new IBM 45 GB hard drive sitting in it; however, when I installed Gentoo Linux on cvs, I only partitioned about 10 gigabytes of the drive, keeping the remaining 35 GB for future filesystems. Such are the little tricks you need to employ when not using LVM -- leaving part of the drive unpartitioned is a primitive but effective way to allow for future expansion. However, with LVM there is a better approach.

The space problem

In the past few weeks, I had been noticing that my root ReiserFS partition had been slowly filling up, as you can see from this df output:

# df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hda3              9765200   6989312   2775888  72% /
tmpfs                   269052         0    269052   0% /dev/shm

Now, a 72% full root partition isn't exactly a crisis, but it isn't a wonderful situation either. ReiserFS, like many other filesystems, starts slowing down as it gets more and more full, and it was just a matter of time before my root filesystem would fill up completely and filesystem performance would take a hit.

I decided to fix this problem by using LVM to create a new logical volume out of the 35 GB of currently unpartitioned space at the end of my hard drive. Then, I'd create a filesystem on this volume and move a good chunk of the contents of /dev/hda3 to it.

If you're thinking of making a similar transition on one of your machines, the first thing you need to do is find a suitable piece of your root filesystem to move to a logical volume. For me, the choice was easy -- my /home tree was taking up around 5.7 GB. By moving /home to its own LVM logical volume, my root filesystem would then be at about 20% capacity. Since most new data is being added to /home, my root filesystem would likely stay at around 20% capacity as well -- a very healthy situation.

The beginnings of a solution

To begin the conversion, I first had to partition the unused space at the end of my hard drive. Using cfdisk, I created a 35 GB partition (/dev/hda5) and set the partition type of the partition to 8E (the official LVM partition type). After this change, I rebooted to force a reread of my partition table. After the reboot, my partition table looked like this:

# sfdisk -l
Disk /dev/hda: 89355 cylinders, 16 heads, 63 sectors/track
Units = cylinders of 516096 bytes, blocks of 1024 bytes, counting from 0
   Device Boot Start     End   #cyls   #blocks   Id  System
/dev/hda1   *      0+    247     248-   124960+  83  Linux
/dev/hda2        248     743     496    249984   82  Linux swap
/dev/hda3        744   20119   19376   9765504   83  Linux
/dev/hda4      20120   89354   69235  34894440    5  Extended
/dev/hda5      20120+  89354   69235- 34894408+  8e  Linux LVM

Now that I had an empty 35 GB partition, I was ready to initialize it for LVM. Here's the procedure -- first, I would initialize the 35 gigabytes as a physical volume; then, I would create a volume group using this physical volume, and finally, I would allocate some of the extents on the volume group, creating a logical volume that would contain my new filesystem and house all the files currently in /home.

To begin the process, I used the pvcreate command to initialize /dev/hda5 as a physical volume:

# pvcreate /dev/hda5
pvcreate -- physical volume "/dev/hda5" successfully created

pvcreate set up a special "accounting" area on /dev/hda5, called the VGDA (volume group descriptor area). LVM uses this area to keep track of how the physical extents are allocated, among other things.

My next step was to create a volume group and add /dev/hda5 to this group. The volume group would act as a pool of extents (chunks of storage blocks). Once the volume group was created, I could create as many logical volumes as I wanted. I decided that my volume group would be called "main":

# vgcreate main /dev/hda5
vgcreate -- INFO: using default physical extent size 4 MB
vgcreate -- doing automatic backup of volume group "main"
vgcreate -- volume group "main" successfully created and activated

The vgcreate command did a couple of things. In addition to creating the "main" volume group, it also set up /dev/hda5 to use 4 MB extents, the default extent size. This means that any logical volumes I create from this volume group can be expanded and shrunk in 4 MB increments.

It is possible to specify a larger extent size at vgcreate time. Extents can range anywhere from 8 KB up to terabytes in size, and must always be a multiple of two. For example, if I wanted to create a volume group with 32 megabyte extents, I'd type:

# vgcreate -s 32M main /dev/hda5

Use an extent size that makes sense for the amount of storage you have available. If you have petabytes of storage, it would make sense to have extents in the gigabyte range. The size you choose for extents in your volume group does not affect filesystem performance, but it does have an impact on the performance of the LVM tools themselves. The fewer extents they have to manage behind the scenes, the faster the tools operate.

Once your volume group is created, you can view its information by typing vgdisplay:

# vgdisplay
--- Volume group ---
VG Name               main
VG Access             read/write
VG Status             available/resizable
VG #                  0
MAX LV                256
Cur LV                0
Open LV               0
MAX LV Size           255.99 GB
Max PV                256
Cur PV                1
Act PV                1
VG Size               33.28 GB
PE Size               4 MB
Total PE              8519
Alloc PE / Size       0 / 0
Free  PE / Size       8519 / 33.28 GB
VG UUID               2qC2H2-iA8s-qW6F-cwXx-JVIh-I6VC-VVCGmn

Now that I had my volume group, I was ready to create a logical volume. I decided to initially make it 8 gigabytes in size and call it "lv_home":

# lvcreate -L8G -nlv_home main
lvcreate -- doing automatic backup of "main"
lvcreate -- logical volume "/dev/main/lv_home" successfully created

Then, I created a filesystem on the volume:

# mkreiserfs /dev/main/lv_home

  <----------- MKREISERFSv2 ----------->

   Block size 4096 bytes
   Block count 2097152
   Used blocks 8275
           Journal - 8192 blocks (18-8209), journal header is in block 8210
           Bitmaps: 17, 32768, 65536, 98304, 131072, 163840,
           196608, 229376, 262144, 294912, 327680, 360448,
           393216, 425984, 458752, 491520, 524288, 557056,
           589824, 622592, 655360, 688128, 720896, 753664,
           786432, 819200, 851968, 884736, 917504, 950272,
           983040, 1015808, 1048576, 1081344, 1114112,
           1146880, 1179648, 1212416, 1245184, 1277952,
           1310720, 1343488, 1376256, 1409024, 1441792,
           1474560, 1507328, 1540096, 1572864, 1605632,
           1638400, 1671168, 1703936, 1736704, 1769472,
           1802240, 1835008, 1867776, 1900544, 1933312,
           1966080, 1998848, 2031616, 2064384
   Root block 8211
Hash function "r5"
ATTENTION: ALL DATA WILL BE LOST ON '/dev/main/lv_home'! (y/n)y
journal size 8192 (from 18)
Initializing journal - 0%....20%....40%....60%....80%....100%

Now that the filesystem was created, I could mount it at /mnt/newhome:

# mkdir /mnt/newhome
# mount /dev/main/lv_home /mnt/newhome
# df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hda3              9765200   6989840   2775360  72% /
tmpfs                   291388         0    291388   0% /dev/shm
/dev/main/lv_home      8388348     32840   8355508   1% /mnt/newhome

As you can see above, I was almost ready to copy over all my data in /home. Before I began, I dropped to runlevel 1 to ensure that no users or processes would be accessing or modifying files in /home as they were being copied over:

# init 1

Then, I began copying the files:

# cp -avx /home/* /mnt/newhome

The copy completed in about ten minutes. Then, I backed up my original /home to /home.old, just in case something was wrong with my copy. I created a new mount point, and remounted the new home at /home:

# cd /
# mv home home.old
# mkdir home
# umount /mnt/newhome
# mount /dev/main/lv_home /home

Then, it was time to set up the server so that my new /home partition would be available every time the machine started up. First, I modified my /etc/fstab so that it included a new /home entry:

#fs                 mountpoint       type         opts          dump/pass
/dev/hda3           /                reiserfs     defaults      1 1
/dev/main/lv_home   /home            reiserfs     defaults      2 2
/dev/hda2           none             swap         sw            0 0
/dev/hda1           /boot            reiserfs     noauto        0 0
/dev/cdrom          /mnt/cdrom       iso9660      noauto,ro     0 0

Once I had completed these steps, I rebooted the machine, and to my delight everything worked perfectly. After a day or so of absolutely no problems, I deleted /home.old to free up some space on my root filesystem. Yay! The transition to LVM was a success.

The beauty of LVM

While the transition to LVM is a bit of an ordeal, once the transition is complete, managing filesystems becomes tremendously easier. As an example, I decided to resize my new /home logical volume, adding about 2 gigabytes worth of space to the end of the filesystem. First, I added additional capacity to my "lv_home" logical volume, and then I used the resize_reiserfs utility to expand the filesystem so that it would use this additional capacity. Here are the two commands that did all this:

# lvextend -L+2G /dev/main/lv_home
# resize_reiserfs -f /dev/main/lv_home

In about a second, I had enlarged my /home filesystem by 2 GB; amazingly, I didn't need to reboot, drop to runlevel 1, or even unmount /home to perform the resize. Everything continued to work as it had before. Isn't that great? Here's the current state of my filesystems:

# df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hda3              9765200   1413340   8351860  15% /
/dev/main/lv_home     10485436   5609836   4875600  54% /home

You can see how LVM really can make an administrator's work a whole lot easier. In the future, I hope to move additional parts of my root filesystem over to LVM, and eventually even convert my root filesystem over to an LVM logical volume. The resources below will help you learn even more about LVM.


Browse all our available articles below. Use the search field to search for topics and keywords in real-time.

Article Subtitle
Article Subtitle
Awk by Example, Part 1 An intro to the great language with the strange name
Awk by Example, Part 2 Records, loops, and arrays
Awk by Example, Part 3 String functions and ... checkbooks?
Bash by Example, Part 1 Fundamental programming in the Bourne again shell (bash)
Bash by Example, Part 2 More bash programming fundamentals
Bash by Example, Part 3 Exploring the ebuild system
Funtoo Filesystem Guide, Part 1 Journaling and ReiserFS
Funtoo Filesystem Guide, Part 2 Using ReiserFS and Linux
Funtoo Filesystem Guide, Part 3 Tmpfs and Bind Mounts
Funtoo Filesystem Guide, Part 4 Introducing Ext3
Funtoo Filesystem Guide, Part 5 Ext3 in Action
GUID Booting Guide
Learning Linux LVM, Part 1 Storage management magic with Logical Volume Management
Learning Linux LVM, Part 2 The cvs.gentoo.org upgrade
Linux Fundamentals, Part 1
Linux Fundamentals, Part 2
Linux Fundamentals, Part 3
Linux Fundamentals, Part 4
Making the Distribution, Part 1
Making the Distribution, Part 2
Making the Distribution, Part 3
Maximum Swappage Getting the most out of swap
On screen annotation Write on top of apps on your screen
OpenSSH Key Management, Part 1 Understanding RSA/DSA Authentication
OpenSSH Key Management, Part 2 Introducing ssh-agent and keychain
OpenSSH Key Management, Part 3 Agent Forwarding
Partition Planning Tips Keeping things organized on disk
Partitioning in Action, Part 1 Moving /home
Partitioning in Action, Part 2 Consolidating data
POSIX Threads Explained, Part 1 A simple and nimble tool for memory sharing
POSIX Threads Explained, Part 2
POSIX Threads Explained, Part 3 Improve efficiency with condition variables
Sed by Example, Part 1
Sed by Example, Part 2
Sed by Example, Part 3
The Gentoo.org Redesign, Part 1 A site reborn
The Gentoo.org Redesign, Part 2 The Documentation System
The Gentoo.org Redesign, Part 3 The New Main Pages
The Gentoo.org Redesign, Part 4 The Final Touch of XML
Traffic Control
Windows 10 Virtualization with KVM
X Window System