Difference between pages "LVM Fun" and "Creating System Rescue CD ZFS Modules"

From Funtoo
(Difference between pages)
Jump to: navigation, search
(Fixed a typing error)
 
 
Line 1: Line 1:
= Introduction =
+
This HOWTO will show you how to create your own zfs srm or download the prebuilt zfs srm module from my website.
  
LVM (Logical Volume Management) offers a great flexibility in managing your storage and significantly reduces server downtimes by allowing on-line disk space management: The great idea beneath LVM is to '''make the data and its storage loosely coupled''' through several layers of abstraction. You (the system administrator) have the hand of each of those layers making the entire space management process extremely simple and flexible through  various set of coherent commands.
+
== Compiling a compatible kernel ==
  
Several other well-known binary Linux distributions makes an aggressive use of LVM and several Unixes including HP-UX, AIX and Solaris offers since a while a similar functionality modulo the commands to be used. LVM is not mandatory but its usage can bring you additional flexibility and make your everyday life much more simpler.
+
The first thing you need to do is decide for which version of System Rescue CD you will be building for. Each System Rescue CD version has a different set of kernels. Specifically each version has a Stable kernel and an Alternate Kernel. The stable kernels get minor version increments every release (Sometimes major ones but usually doesn't happen too quickly), the Alternate kernels tend to move up much faster. For example System Rescue CD had the following stable/alternate kernels:
  
= Concepts =
+
{| class="wikitable"
 +
|-
 +
! Version !! Stable !! Alternate
 +
|-
 +
| 3.7.0 || 3.4.47 || 3.9.4
 +
|-
 +
| 3.5.0 || 3.4.37 || 3.8.4
 +
|}
  
As usual, having a good idea of the concepts lying beneath is mandatory. LVM is not very complicated, but it is easy to become confused, especially because it is a multi-layered system; however LVM designers had the good idea of keeping the command names consistent between all LVM command sets, making your life easier.  
+
===Download kernel and patches===
 +
Once you decide which version you want to use, you can go to http://kernel.sysresccd.org/ and pick the version you want. After you do this, download all the stuff in that directory and put them in their own folder (a work directory). The files that are named std-sources are the patches for the regular stable kernel, the patches named alt-sources are the patches for the alternate kernel. Once you have the patches, go to kernel.org and also download the initial release of the kernel sources you want. For example, if you want to build an srm for 3.4.2 which uses the 3.4.35 kernel, just download the 3.4 (3.4.0) kernel. You just need to download the initial release because one of the patches you downloaded earlier is the patch to update the 3.4.0 to 3.4.35.
  
LVM consists of, mainly, three things:
+
Once you download the kernel you want, let's extract and patch it.
  
* '''Physical volumes (or ''PV'')''': nothing more than a physical storage space. A physical volume can by anything like a partition on a local hard disk, a partition located on a remote SAN disk, a USB key or whatever else that could offer a storage space (so yes, technically it could be possible to use an optical storage device accessed in packet writing mode). The storage space on a physical volumes is divided (and managed) in small units called '''Physical Extents''' (or ''PE''). Just to give an analogy if you are a bit familiar with RAID, PE are a bit like RAID stripes.
+
<console>
* '''Volume Groups (or ''VG'')''': a group of at least one PV. VG are '''named''' entities and will appear in the system via the device mapper as '''/dev/''volume-group-name'''''.
+
# ##i##tar xf linux-3.4.tar.bz2
* '''Logical Volumes (or ''LV'')''': a '''named''' division of a volume group in which a filesystem is created and that can be mounted in the VFS. Just for the record, just as for the PE in PV, a LV is managed as chucks known as Logical Extents (or ''LE''). Most of the time those LE are hidden to the system administrator due to a 1:1 mapping between them and the PE lying be just beneath but a cool fact to know about LEs is that they can be spread over PV just like RAID stripes in a RAID-0 volume. However, researches done on the Web tends to demonstrate system administrators prefer to build RAID volumes with mdadm than use LVM over them for performance reasons.
+
# ##i##mv linux-3.4 linux-3.4.35-std342-amd64
 +
# ##i##bzcat std-sources-3.4-01-stable-3.4.35.patch.bz2 | patch -p1 -d linux-3.4.35-std342-amd64
 +
# ##i##bzcat std-sources-3.4-02-fc16.patch.bz2 | patch -p1 -d linux-3.4.35-std342-amd64
 +
# ##i##bzcat std-sources-3.4-03-aufs.patch.bz2 | patch -p1 -d linux-3.4.35-std342-amd64
 +
</console>
  
In short words:  LVM logical volumes (LV) are containers that can hold a single filesystem and which are created inside a volume group (VG) itself composed by an aggregation of at least one physical volumes (PV) themselves stored on various media (usb key, harddisk partition and so on). The data is stored in chunks spread over the various PV.  
+
===Retrieve kernel configuration===
 +
Once that is complete, you will need to get the kernel config specifically for the kernel and architecture you want. The easiest way I found to do this was to download the System Rescue CD you want, and then boot it inside a VM. Once you are inside the VM, go to the /root directory, and it will be a file named kernel-<version>.conf. Copy this file over SSH and put it in a safe location. Also copy it over into the linux kernel directory and rename it to .config.
  
{{fancynote|Retain what PV, VG and LV means as we will use those abbreviations in the rest of this article.}}
+
===Build the kernel===
 +
Once you copy the kernel config over, do a make menuconfig to begin. You will only need to change two settings inside the config, add the proper kernel name, and remove the old initramfs source files directory. This directory doesn't exist locally. This is a remnant of upstream's build process.
  
= Your first tour of LVM =
+
Follow the below tree structure, and make it equal to what is below:
  
== Physical volumes creation ==
+
{{kernelop
 +
|title=
 +
|desc=
 +
(-std342-amd64) Local version - append to kernel release
 +
      [*] Initial RAM filesystem and RAM disk (initramfs/initrd) support
 +
      ( ) Initramfs source file(s)
 +
}}
  
{{fancynote|We give the same size to all volumes for the sake of the demonstration. This is not mandatory and be possible to have mixed sizes PV inside a same VG. }}
+
The Local version above will end up making the kernel modules directory be 3.4.35-std342-amd64. The reason the kernel modules directory has to be equal to this is because our directory will be merged into the System Rescue CD later on, so the names need to be exactly the same so that the System Rescue CD system can find our kernel modules.
  
To start with, just create three raw disk images:
+
If you want to build an srm (which you do), you also need to change the following inside the kernel config so that squashfs-tools can work later on:
  
<pre>
+
{{kernelop
# dd if=/dev/zero of=/tmp/hdd1.img bs=2G count=1
+
|title=
# dd if=/dev/zero of=/tmp/hdd2.img bs=2G count=1
+
|desc=
# dd if=/dev/zero of=/tmp/hdd3.img bs=2G count=1
+
Miscellaneous Filesystems ->
</pre>
+
      <*>   SquashFS 4.0 - Squashed file system support
 +
      [*]    Squashfs XATTR support
 +
      [*]    Include support for ZLIB compressed file systems
 +
      [*]    Include support for LZO compressed file systems
 +
      [*]    Include support for XZ compressed file systems
 +
      [ ]    Use 4K device block size?
 +
      [ ]    Additional option for memory-constrained systems
 +
}}
  
and associate them to a loopback device:
+
Now save and exit, and continue to build your kernel.
  
<pre>
+
<console>
# losetup -f
+
# ##i##make bzImage modules
/dev/loop0
+
# ##i##make modules_install
# losetup /dev/loop0 /tmp/hdd1.img
+
</console>
# losetup /dev/loop1 /tmp/hdd2.img
+
# losetup /dev/loop2 /tmp/hdd3.img
+
  
</pre>
+
{{fancynote|If you have multiple cores, do a make -j<# of processors+1> to build faster. Example: '''make -j9''' for an 8 core machine.}}
  
Okay nothing really exciting there, but wait the fun is coming! First check that '''sys-fs/lvm2''' is present on your system and emerge it if not. At this point, we must tell you a secret: although several articles and authors uses the taxonomy "LVM" it denotes "LVM version 2" or "LVM 2" nowadays. You must know that LVM had, in the old good times (RHEL 3.x and earlier), a previous revision known as "LVM version 1". LVM 1 is now considered as an extincted specie and is not compatible with LVM 2, although LVM 2 tools maintain a backward compatibility. 
+
=== Using the portage tree ===
 +
Inside the portage tree, there are ebuilds already prepared that will download the kernel, patch it, and perform any required substitutions. Afterwards you literally just go into your /usr/src/<kernel> directory, and run make. To see the available versions, run the following:
  
The very frst step in LVM is to create the physical devices or ''PV''. "Wait create ''what''?! Aren't the loopback devices present on the system?" Yes they are present but they are empty, we must initialize them some metadata to make them usable by LVM. This is simply done by:
+
<console>
 +
Versions available for standard kernel
 +
# ##i##equery y std-sources
  
<pre>
+
[I]3.4.37-r1 | o ~ o o o o o o o o o o o | o 3.4.37 | gentoo
# pvcreate /dev/loop0
+
  Physical volume "/dev/loop0" successfully created
+
# pvcreate /dev/loop1
+
  Physical volume "/dev/loop1" successfully created
+
# pvcreate /dev/loop2
+
  Physical volume "/dev/loop2" successfully created
+
</pre>
+
  
It is absolutely normal that nothing in particular is printed at the output of each command but we assure you: you have three LVM PVs. You can check them by issuing:
+
Versions available for alternate kernel
 +
# ##i##equery y alt-sources
  
<pre>
+
[I]3.8.4 | o ~ o o o o o o o o o o o | o 3.8.4 | gentoo
# pvs
+
</console>
  PV        VG  Fmt  Attr PSize PFree
+
  /dev/loop0      lvm2 a-  2.00g 2.00g
+
  /dev/loop1      lvm2 a-  2.00g 2.00g
+
  /dev/loop2      lvm2 a-  2.00g 2.00g
+
</pre>
+
  
 +
I haven't written the ebuilds in a way where it's easy to tell (at first glance) what System Rescue CD version you need. However, you can check the ebuild and you will see the version of the CD inside there. You can also check the chart above.
  
Some good information there:
+
If you wanted to install the standard kernel sources, you just emerge it like any other application:
* PV: indicates the physical path the PV lies on
+
* VG indicates the VG the PV belongs to. At this time, we didn't created any VG yet and the column remains empty.
+
* Fmt: indicates the format of the PV (here it says we have a LVM version 2 PV)
+
* Attrs: indicates some status information, the 'a' here just says that the PV is accessible.
+
* PSize and PFree: indicates the PV size and the amount of remaining space for this PV. Here we have three empty PV so it bascially says "2 gigabytes large, 2 out of gigabytes free"
+
  
It is now time to introduce you to another command: '''pvdisplay'''. Just run it without any arguments:
+
<console>
 +
# ##i##emerge std-sources
 +
</console>
  
<pre>
+
Once the kernel and the kernel module are installed, you need to make the SRM.
pvdisplay
+
  "/dev/loop0" is a new physical volume of "2.00 GiB"
+
  --- NEW Physical volume ---
+
  PV Name              /dev/loop0
+
  VG Name             
+
  PV Size              2.00 GiB
+
  Allocatable          NO
+
  PE Size              0 
+
  Total PE              0
+
  Free PE              0
+
  Allocated PE          0
+
  PV UUID              b9i1Hi-llka-egCF-2vU2-f7tp-wBqh-qV4qEk
+
 
+
  "/dev/loop1" is a new physical volume of "2.00 GiB"
+
  --- NEW Physical volume ---
+
  PV Name              /dev/loop1
+
  VG Name             
+
  PV Size              2.00 GiB
+
  Allocatable          NO
+
  PE Size              0 
+
  Total PE              0
+
  Free PE              0
+
  Allocated PE          0
+
  PV UUID              i3mdBO-9WIc-EO2y-NqRr-z5Oa-ItLS-jbjq0E
+
 
+
  "/dev/loop2" is a new physical volume of "2.00 GiB"
+
  --- NEW Physical volume ---
+
  PV Name              /dev/loop2
+
  VG Name             
+
  PV Size              2.00 GiB
+
  Allocatable          NO
+
  PE Size              0 
+
  Total PE              0
+
  Free PE              0
+
  Allocated PE          0
+
  PV UUID              dEwVuO-a5vQ-ipcH-Rvlt-5zWt-iAB2-2F0XBf
+
</pre>
+
  
The third three lines of each PV shows:
+
== Creating the SRM ==
* what is the storage device beneath a PV
+
It's time to gather the required files and pack them together in order for it to become an SRM. An SRM is nothing more than a directory that has been compressed with squashfs. You can emerge the "Bliss Initramfs Creator" which has a feature that will automagically build the srm for you.
* the VG it is tied to  
+
* the size of this PV.  
+
''Allocatable'' indicates whether the PV is used to store data. As the PV is not a member of a VG, it cannot not be used (yet) hence the "NO" shown. Another set of information is the lines starting with ''PE''. ''PE'' stands for ''' ''Physical Extents'' ''' (data stripe) and is the finest granularity LVM can manipulate. The size of a PE is "0" here because we have a blank PV however it typically holds 32 MB of data. Following ''PE Size'' are ''Total PE'' which show the the total '''number''' of PE available on this PV and ''Free PE'' the number of PE remaining available for use. ''Allocated PE'' just show the difference between ''Total PE'' and ''Free PE''.
+
  
The latest line (''PV UUID'') is a unique identifier used internally by LVM to name the PV. You have to know that it exists because it is sometimes useful when having to recover from corruption or do weird things with PV however most of the time you don't have to worry about its existence.
 
 
{{fancynote|It is possible to force how LVM handles the alignments on the physical storage. This is useful when dealing with 4K sectors drives that lies on their physical sectors size. Refer to the manual page. }}
 
  
== Volume group creation ==
+
You also need to have spl, zfs, and zfs-kmod installed on your system before you try to make the srm. Emerging '''"bliss-initramfs"''' should automatically pull those packages. If it doesn't, make sure you install them.
  
We have the blank PV at this time but to make them a bit more usable for storage we must tell to LVM how they are grouped to form a VG (storage pool) where LV will be created. A nice aspect of VGs resides in the fact that they are not "written in the stone" once created: you can still add, remove or exchange PV (in the case the device the PV is stored on fails for example) inside a VG at a later time. To create our first volume group named ''vgtest'':
+
<console>
 +
# ##i##echo "sys-kernel/bliss-initramfs srm" >> /etc/portage/package.use
 +
# ##i##emerge bliss-initramfs
 +
</console>
  
<pre>
 
# vgcreate vgtest /dev/loop0 /dev/loop1 /dev/loop2
 
  Volume group "vgtest" successfully created
 
</pre>
 
  
Just like we did before with PV, we can get a list of what are the VG known by the system. This is done through the command '''vgs''':
+
You should now have the following directory layout inside the '''/opt/bliss-initramfs''' folder:
  
<pre>
+
<console>
# vgs
+
octopus bliss-initramfs # ls -l
  VG    #PV #LV #SN Attr  VSize VFree
+
total 117
  vgtest  3   0  0 wz--n- 5.99g 5.99g
+
-rw-r--r-- 1 root root 6718 May 23 18:05 CHANGES
</pre>
+
-rw-r--r-- 1 root root  176 May 23 18:05 CREDITS
 +
-rw-r--r-- 1 root root  431 May 23 18:05 HELP
 +
-rw-r--r-- 1 root root 1397 May 23 18:05 LICENSE
 +
-rw-r--r-- 1 root root 1852 May 23 18:05 README
 +
-rw-r--r-- 1 root root 3194 May 23 18:05 USAGE
 +
-rwxr-xr-x 1 root root 2891 May 23 18:05 createInit
 +
drwxr-xr-x 3 root root    4 May 23 18:11 files
 +
drwxr-xr-x 3 root root    6 May 23 18:11 hooks
 +
drwxr-xr-x 2 root root    5 May 23 22:01 resources
 +
</console>
  
'''vgs''' show you a tabluar view of information:
 
* '''VG:''' the name of the VG
 
* '''#PV:''' the number of PV composing the VG
 
* '''#LV:''' the number of logical volumes (LV) located inside the VG
 
* '''Attrs:''' a status field. w, z and n here means that VG is:
 
** '''w:''' '''w'''ritable
 
** '''z:''' resi'''z'''able
 
** '''n:''' using the allocation policy '''''n'''ormal'' (tweaking allocation policies is beyond the scope of this article, we will use the default value ''normal'' in the rest of this article)
 
* VSize and VFree gives statistics on how full a VG is versus its size
 
  
Note the dashes in ''Attrs'', they mean that the attribute is not active:
+
Then run the '''createInit''' script and follow the instructions for the kernel you want to make the srm for (In this case it's '''3.4.37-std350-amd64'''):
* First dash (3rd position) indicates if the VG would have been exported (a 'x' would have been showed at this position in that case).
+
* Second dash (4th position) indicates if the VG would have been partial (a 'p' would have been showed at this position in that case).
+
* Third dash (rightmost position) indicates if the VG is a clustered (a 'c' would have been showed at this position in that case)
+
  
Exporting a VG and clustered VG are a bit more advanced aspects of LVM and won't be covered here especially the clustered VGs which are used in the case of a shared storage space used in a cluster of machines. Talking about clustered VGs management in particular would require and entire article in itself. '''For now the only detail you have to worry about those dashes in ''Attrs'' is to see a dash at the 4th position of ''Attrs'' instead of a ''p'''''. Seeing ''p'' there would be a bad news: the VG would have missing parts (PV) making it not usable.
+
<console>
 +
-----------------------------------
 +
| Bliss Initramfs Creator - v1.8.1
 +
| Author: Jonathan Vasquez <jvasquez1011@gmail.com>
 +
| Distributed under the Simplified BSD License
 +
-----------------------------------
  
{{fancynote|In the exact same manner you can see a detailed information about physical volumes with '''pvdisplay''', you can see detailed information of a volume group with '''vgdisplay'''. We will demonstrate that latter command in the paragraphs to follow.}}
+
>>> Which initramfs would you like to generate:
 +
>> 1. ZFS
 +
>> 2. Encrypted ZFS (LUKS + ZFS)
 +
>> 3. More Options
 +
>> 4. Exit Program
  
Before leaving the volume group aspect, do you remember the '''pvs''' command shown in the previous paragraphs? Try it gain:
+
>>> Current choice [1]: ##i##3 ↵
  
<pre>
+
>>> More Options:
# pvs
+
  PV        VG    Fmt  Attr PSize PFree
+
  /dev/loop0 vgtest lvm2 a-  2.00g 2.00g
+
  /dev/loop1 vgtest lvm2 a-  2.00g 2.00g
+
  /dev/loop2 vgtest lvm2 a-  2.00g 2.00g
+
</pre>
+
  
Now it shows the VG our PVs belong to :-)
+
>> 1. ZFS - System Rescue Module
 +
>> 2. Back
 +
>> 3. Exit Program
  
== Logical volumes creation ==
+
>>> Current choice [1]: ##i##↵
  
Now the final steps: we will create the storage areas (logical volumes or ''LV'') inside the VG where we will then create filesystems on. Just like a VG has a name, a LV has also a name which is unique in the VG.
+
>>> Creating a ZFS System Rescue Module!
  
{{fancynote|Two LV can be given the same name as long as they are located on a different VG.}}
+
>>> Do you want to use the current kernel: 3.8.13-ALL? [Y/n]: ##i##n ↵
  
To divide our VG like below:
+
>>> Please enter the kernel name: ##i##3.4.37-std350-amd64 ↵
  
* lvdata1: 2 GB
+
>>> Detected 64 bit platform
* lvdata2: 1 GB
+
>>> Checking to see if modules directory exists for 3.4.37-std350-amd64...
* lvdata3 : 10% of the VG size
+
>>> Creating SRMs for 3.4.37-std350-amd64...
* lvdata4 : All of remaining free space in the VG
+
>>> Creating directory structure for initramfs...
 +
>>> Checking preliminary binaries...
 +
>>> Checking binaries...
 +
>>> Using ZFS
 +
>>> Checking modules...
 +
>>> Copying binaries...
 +
>>> Copying modules...
 +
>>> Copying documentation...
 +
>>> Copying udev rules...
 +
>>> Compressing kernel modules...
 +
>>> Getting dependencies...
 +
>>> Copying dependencies...
 +
>>> Configuring files...
 +
>>> Creating and Packing SRMs...
 +
>>> Complete :)
 +
</console>
  
We use the following commands (notice the capital 'L' and the small 'l' to declare absolute or relative sizes):
 
  
<pre>
+
Now you should have two new files:
# lvcreate -n lvdata1 -L 2GB vgtest
+
  Logical volume "lvdata1" created
+
#  lvcreate -n lvdata2 -L 1GB vgtest
+
  Logical volume "lvdata2" created
+
# lvcreate -n lvdata3 -l 10%VG vgtest
+
  Logical volume "lvdata2" created
+
</pre>
+
  
What is going on so far? Let's check with the pvs/vgs counterpart known as '''lvs''':
+
<console>
 +
-rw-r--r-- 1 root root 2068480 May 24 01:05 zfs-core-3.4.37-std350-amd64.srm
 +
-rw-r--r-- 1 root root  483328 May 24 01:05 zfs-kmod-3.4.37-std350-amd64.srm
 +
</console>
  
<pre>
 
# lvs
 
  LV      VG    Attr  LSize  Origin Snap%  Move Log Copy%  Convert
 
  lvdata1 vgtest -wi-a-  2.00g                                     
 
  lvdata2 vgtest -wi-a-  1.00g                                     
 
  lvdata3 vgtest -wi-a- 612.00m
 
#
 
</pre>
 
  
Notice the size of ''lvdata3'', it is roughly 600MB (10% of 6GB). How much free space remains in the VG? Time to see what '''vgs''' and '''vgdisplay''' returns:
+
Now all you need to do is put those two files in the root of your USB directory.
  
<pre>
+
If you are making srms for both the standard and alternate kernels, you will end up with two zfs-core files and two zfs-kmod files (1 set for each kernel). You don't need to put the zfs-core that it makes for one of the kernels. The zfs-core srm only has zfs program binaries, man pages, udev rules, and a few other things. The zfs-kmod is the srm that has the kernel modules (and only the kernel modules). So you can easily just put two zfs-kmods (one for each kernel version) and just re-use the same zfs-core for both of them.
# vgs
+
  VG    #PV #LV #SN Attr  VSize VFree
+
  vgtest  3  3  0 wz--n- 5.99g 2.39g
+
# vgdisplay
+
  --- Volume group ---
+
  VG Name              vgtest
+
  System ID           
+
  Format                lvm2
+
  Metadata Areas        3
+
  Metadata Sequence No  4
+
  VG Access            read/write
+
  VG Status            resizable
+
  MAX LV                0
+
  Cur LV                3
+
  Open LV              0
+
  Max PV                0
+
  Cur PV                3
+
  Act PV                3
+
  VG Size              5.99 GiB
+
  PE Size              4.00 MiB
+
  Total PE              1533
+
  Alloc PE / Size      921 / 3.60 GiB
+
  Free  PE / Size      612 / 2.39 GiB
+
  VG UUID              baM3vr-G0kh-PXHy-Z6Dj-bMQQ-KK6R-ewMac2
+
</pre>
+
  
Basically it say we have 1533 PE (chunks) available for a total size of 5.99 GiB. On those 1533, 921 are used (for a size of 3.60 GiB) and 612 remains free (for a size of 2.39 GiB). So we expect to see lvdata4 having an approximative size of 2.4 GiB. Before creating it, have a look at some statistics at the PV level:
+
== Using the prebuilt srm ==
  
<pre>
+
=== Download the SRM ===
# pvs
+
If you didn't build your own srm and want to use the prebuilt one, just emerge '''"zfs-srm"''' from the tree:
  PV        VG    Fmt  Attr PSize PFree 
+
  /dev/loop0 vgtest lvm2 a-  2.00g      0
+
  /dev/loop1 vgtest lvm2 a-  2.00g 404.00m
+
  /dev/loop2 vgtest lvm2 a-   2.00g  2.00g
+
  
# pvdisplay
+
<console>
  --- Physical volume ---
+
# ##i##emerge zfs-srm
  PV Name              /dev/loop0
+
</console>
  VG Name              vgtest
+
  PV Size              2.00 GiB / not usable 4.00 MiB
+
  Allocatable          yes (but full)
+
  PE Size              4.00 MiB
+
  Total PE              511
+
  Free PE              0
+
  Allocated PE          511
+
  PV UUID              b9i1Hi-llka-egCF-2vU2-f7tp-wBqh-qV4qEk
+
 
+
  --- Physical volume ---
+
  PV Name              /dev/loop1
+
  VG Name              vgtest
+
  PV Size              2.00 GiB / not usable 4.00 MiB
+
  Allocatable          yes
+
  PE Size              4.00 MiB
+
  Total PE              511
+
  Free PE              101
+
  Allocated PE          410
+
  PV UUID              i3mdBO-9WIc-EO2y-NqRr-z5Oa-ItLS-jbjq0E
+
 
+
  --- Physical volume ---
+
  PV Name              /dev/loop2
+
  VG Name              vgtest
+
  PV Size              2.00 GiB / not usable 4.00 MiB
+
  Allocatable          yes
+
  PE Size              4.00 MiB
+
  Total PE              511
+
  Free PE              511
+
  Allocated PE          0
+
  PV UUID              dEwVuO-a5vQ-ipcH-Rvlt-5zWt-iAB2-2F0XBf
+
</pre>
+
  
Quite interesting! Did you notice? The first PV is full, the second is more or less full and the third is empty. This is due to the allocation policy used for the VG: it fills its first PV then its second PV and then its third PV (this, by the way, gives you a chance to recover from a dead physical storage if by luck none of your PE was present on it).
+
You can check available versions just as you did above for the kernels:
  
It is now time to create our last LV, again notice the small 'l' to specify a relative size:
+
<console>
 +
Available versions of the zfs-srm
 +
# ##i##equery y zfs-srm
  
<pre>
+
[I]3.5.0 | o ~ o o o o o o o o o o o | o 3.5.0 | gentoo
# lvcreate -n lvdata4 -l 100%FREE vgtest
+
</console>
  Logical volume "lvdata4" created
+
# lvs
+
  LV      VG    Attr  LSize  Origin Snap%  Move Log Copy%  Convert
+
  lvdata1 vgtest -wi-a-  2.00g                                     
+
  lvdata2 vgtest -wi-a-  1.00g                                     
+
  lvdata3 vgtest -wi-a- 612.00m                                     
+
  lvdata4 vgtest -wi-a-  2.39g
+
</pre>
+
  
Now the $100 question: if '''pvdisplay''' and '''vgdisplay''' commands exist, does command named '''lvdisplay''' exist as well? Yes absolutely! Indeed the command sets are coherent between abstraction levels (PV/VG/LV) and they are named in the exact same manner modulo their first 2 letters:
+
(These versions match the version of the System Rescue CD).
  
* PV: pvs/pvdisplay/pvchange....
+
Installing the zfs-srm will automatically pull the '''"bliss-isomaker"''' package which is just a script that assists you with rebuilding the System Rescue CD ISO so that the ISO includes the ZFS srms. The script lets you make a bootable ISO or a bootable USB.
* VG: vgs/vgdisplay/vgchange....
+
* LG: lvs/lvdisplay/lvchange....
+
  
Back to our '''lvdisplay''' command, here is how it shows up:
+
Once it's installed, switch to the /opt/bliss-isomaker folder
  
<pre>
+
<console>
# lvdisplay
+
# ##i##cd /opt/bliss-isomaker
  --- Logical volume ---
+
</console>
  LV Name                /dev/vgtest/lvdata1
+
  VG Name                vgtest
+
  LV UUID                fT22is-cmSL-uhwM-zwCd-jeIe-DWO7-Hkj4k3
+
  LV Write Access        read/write
+
  LV Status              available
+
  # open                0
+
  LV Size                2.00 GiB
+
  Current LE            512
+
  Segments              2
+
  Allocation            inherit
+
  Read ahead sectors    auto
+
  - currently set to    256
+
  Block device          253:0
+
 
+
  --- Logical volume ---
+
  LV Name                /dev/vgtest/lvdata2
+
  VG Name                vgtest
+
  LV UUID                yd07wA-hj77-rOth-vxW8-rwo9-AX7q-lcyb3p
+
  LV Write Access        read/write
+
  LV Status              available
+
  # open                0
+
  LV Size                1.00 GiB
+
  Current LE            256
+
  Segments              1
+
  Allocation            inherit
+
  Read ahead sectors    auto
+
  - currently set to    256
+
  Block device          253:1
+
 
+
  --- Logical volume ---
+
  LV Name                /dev/vgtest/lvdata3
+
  VG Name                vgtest
+
  LV UUID                ocMCL2-nkcQ-Fwdx-pss4-qeSm-NtqU-J7vAXG
+
  LV Write Access        read/write
+
  LV Status              available
+
  # open                0
+
  LV Size                612.00 MiB
+
  Current LE            153
+
  Segments              1
+
  Allocation            inherit
+
  Read ahead sectors    auto
+
  - currently set to    256
+
  Block device          253:2
+
 
+
  --- Logical volume ---
+
  LV Name                /dev/vgtest/lvdata4
+
  VG Name                vgtest
+
  LV UUID                iQ2rV7-8Em8-85ts-anan-PePb-gk18-A31bP6
+
  LV Write Access        read/write
+
  LV Status              available
+
  # open                0
+
  LV Size                2.39 GiB
+
  Current LE            612
+
  Segments              2
+
  Allocation            inherit
+
  Read ahead sectors    auto
+
  - currently set to    256
+
  Block device          253:3
+
</pre>
+
  
Nothing extremely useful to comment for an overview beyond showing at the exception of two things:
+
You should now see a directory layout that looks similar to this:
# '''LVs are accessed via the device mapper''' (see the lines starting by ''LV Name'' and notice how the name is composed). So '''lvdata1''' will be accessed via ''/dev/vgtest/lvdata1'', ''lvdata2'' will be accessed via ''/dev/vgtest/lvdata2'' and so on.
+
# just like PV are managed in sets of data chunks (the so famous Physical Extents or PEs), LVs are managed in a set of data chunks known as Logical Extents or LEs. Most of the time you don't have to worry about the existence of LEs because they fits withing a single PE although it is possible to make them smaller hence having several LE within a single PE. Demonstration: if you consider the first LV, '''lvdisplay''' says it has a size of 2 GiB and holds 512 logical extents. Dividing 2GiB by 512 gives 4 MiB as the size of a LE which is the exact same size used for PEs as seen when demonstrating the '''pvdisplay''' command some paragraphs above. So in our case we have a 1:1 match between a LE and the underlying PE.
+
  
Oh another great point to underline: you can display the PV in relation with a LV :-) Just give a special option to '''lvdisplay''':
+
<console>
 +
octopus bliss-isomaker # ls -l
 +
total 100
 +
drwxr-xr-x 2 root root    5 May 24 01:41 3.5.0
 +
-rw-r--r-- 1 root root 1397 May 24 01:31 LICENSE
 +
-rw-r--r-- 1 root root  312 May 24 01:31 README
 +
-rw-r--r-- 1 root root  576 May 24 01:31 USAGE
 +
-rwxr-xr-x 1 root root 3228 May 24 01:31 create
 +
drwxr-xr-x 2 root root    3 May 24 01:41 iso
 +
drwxr-xr-x 2 root root    4 May 24 01:41 resources
 +
drwxr-xr-x 2 root root    3 May 24 01:41 srm
 +
</console>
  
<pre>
+
The layout is as follows:
# lvdisplay -m
+
* 3.5.0 - This folder contains the System Rescue CD 3.5.0 specific srms for both the standard and alternate kernel that were installed by emerge.
  --- Logical volume ---
+
* create - A script to automatically recreate the system rescue cd iso or usb with the zfs stuff included
  LV Name                /dev/vgtest/lvdata1
+
* iso - directory to place your system rescue cd iso in
  VG Name                vgtest
+
* srm - directory that has the srms
  (...)
+
* resources - the files in this folder contain function calls that the 'create' script uses. You don't need to worry about these.
  Current LE            512
+
  Segments              2
+
  (...)
+
  --- Segments ---
+
  Logical extent 0 to 510:
+
    Type                linear
+
    Physical volume    /dev/loop0
+
    Physical extents    0 to 510
+
 
+
  Logical extent 511 to 511:
+
    Type                linear
+
    Physical volume    /dev/loop1
+
    Physical extents    0 to 0
+
 
+
 
+
  --- Logical volume ---
+
  LV Name                /dev/vgtest/lvdata2
+
  VG Name                vgtest
+
  (...)
+
  Current LE            256
+
  Segments              1
+
  (...)
+
 
+
  --- Segments ---
+
  Logical extent 0 to 255:
+
    Type                linear
+
    Physical volume    /dev/loop1
+
    Physical extents    1 to 256
+
 
+
 
+
  --- Logical volume ---
+
  LV Name                /dev/vgtest/lvdata3
+
  VG Name                vgtest
+
  (...)
+
  Current LE            153
+
  Segments              1
+
  (...)
+
 
+
  --- Segments ---
+
  Logical extent 0 to 152:
+
    Type                linear
+
    Physical volume    /dev/loop1
+
    Physical extents    257 to 409
+
 
+
 
+
  --- Logical volume ---
+
  LV Name                /dev/vgtest/lvdata4
+
  VG Name                vgtest
+
  (...)
+
  Current LE            612
+
  Segments              2
+
  (...)
+
 
+
  --- Segments ---
+
  Logical extent 0 to 510:
+
    Type                linear
+
    Physical volume    /dev/loop2
+
    Physical extents    0 to 510
+
 
+
  Logical extent 511 to 611:
+
    Type                linear
+
    Physical volume    /dev/loop1
+
    Physical extents    410 to 510
+
</pre>
+
  
To go one step further let's analyze a bit how the PE are used: the first LV has 512 LEs (remember: one LE fits within one PE here so 1 LE = 1 PE). Amongst those 512 LEs, 511 of them (0 to 510) are stored on /dev/loop0 and the 512th LE is on /dev/loop1. Huh? Something seems to be wrong here, '''pvdisplay''' said that /dev/loop0 was holding 512 PV so why an extent has been placed on the second storage device? Indeed its not a misbehaviour and absolutely normal: LVM uses some metadata internally with regards the PV, VG and LV thus making some of storage space unavailable for the payload. This explains why 1 PE has been "eaten" to store that metadata. Also notice the linear allocation process: ''/dev/loop0'' has been used, then when being full ''/dev/loop1'' has also been used then the turn of /''dev/loop2'' came.
+
=== Installing the SRM ===
 +
There are a few ways to do this, you can either use one of the scripts, or you can do it manually. Before anything, make sure to copy the SRMs from the <Version> folder to the srm folder if you are using a prebuilt one:
  
Now everything is in place, if you want just check again with '''vgs/pvs/vgdisplay/pvdisplay''' and will notice that the VG is now 100% full and all of the underlying PV are also 100% full.
+
<console>
 +
# ##i##cp 3.5.0/* srm
 +
</console>
  
== Filesystems creation  and mounting  ==  
+
==== Generating a new iso ====
 +
If you just want to remake a new iso so that you can burn onto a cd or use in a virtual machine, just copy your iso over to the iso directory, and run the 'create' script. The new iso will be located in the iso directory as well with a -zfs ending.
  
Now we have our LVs it could be fun if we could do something useful with them. In the case you missed it, LVs are accessed via the device mapper which uses a combination of the VG and LV names thus:
+
<console>
* lvdata1 is accessible via /dev/vgtest/lvdata1
+
Running the script with 1 (or iso) chooses to make an iso. You can also pass the version as well.
* lvdata2 is accessible via /dev/vgtest/lvdata2
+
# ##i##./create 1
* and so on!
+
  
Just like any traditional storage device, the newly created LVs are seen as block devices as well just as if they were a kind of harddisk (don't worry about the "dm-..", it is just an internal block device automatically allocated by the device mapper for you):
+
or
<pre>
+
# ls -l /dev/vgtest
+
total 0
+
lrwxrwxrwx 1 root root 7 Dec 27 12:54 lvdata1 -> ../dm-0
+
lrwxrwxrwx 1 root root 7 Dec 27 12:54 lvdata2 -> ../dm-1
+
lrwxrwxrwx 1 root root 7 Dec 27 12:54 lvdata3 -> ../dm-2
+
lrwxrwxrwx 1 root root 7 Dec 27 12:54 lvdata4 -> ../dm-3
+
  
# ls -l /dev/dm-[0-3]
+
# ##i##./create iso 3.5.0
brw-rw---- 1 root disk 253, 0 Dec 27 12:54 /dev/dm-0
+
</console>
brw-rw---- 1 root disk 253, 1 Dec 27 12:54 /dev/dm-1
+
brw-rw---- 1 root disk 253, 2 Dec 27 12:54 /dev/dm-2
+
brw-rw---- 1 root disk 253, 3 Dec 27 12:54 /dev/dm-3
+
</pre>
+
  
So if LVs are block device a filesystem can be created on them just like if they were a real harddisk or hardisk partitions? Absolutely! Now let's create ext4 filesystems on our LVs:
+
==== Creating a fresh usb ====
 +
If you want to have zfs on a usb with system rescue cd, put the iso in your iso dir, and then run the usb script. It will ask you what usb you want to format (This will delete everything), and then install system rescue cd onto it. Once that is done it will copy the zfs stuff over.
  
<pre>
 
# mkfs.ext4 /dev/vgtest/lvdata1
 
  
mke2fs 1.42 (29-Nov-2011)
+
<console>
Discarding device blocks: done                           
+
Running the script with 2 (or usb) chooses to make an usb. You can also pass the version as well.
Filesystem label=
+
# ##i##./create 2
OS type: Linux
+
Block size=4096 (log=2)
+
Fragment size=4096 (log=2)
+
Stride=0 blocks, Stripe width=0 blocks
+
131072 inodes, 524288 blocks
+
26214 blocks (5.00%) reserved for the super user
+
First data block=0
+
Maximum filesystem blocks=536870912
+
16 block groups
+
32768 blocks per group, 32768 fragments per group
+
8192 inodes per group
+
Superblock backups stored on blocks:
+
        32768, 98304, 163840, 229376, 294912
+
  
Allocating group tables: done                           
+
or
Writing inode tables: done                           
+
Creating journal (16384 blocks): done
+
Writing superblocks and filesystem accounting information: done
+
  
# mkfs.ext4 /dev/vgtest/lvdata1
+
# ##i##./create usb 3.5.0
(...)
+
</console>
# mkfs.ext4 /dev/vgtest/lvdata2
+
(...)
+
# mkfs.ext4 /dev/vgtest/lvdata3
+
(..)
+
</pre>
+
  
Once the creation ended we must create the mount points and mount the newly created filesystems on them:
+
==== Manual installation onto a usb ====
 +
Assuming that your installing to a flash drive and that the flash drive is mounted in /mnt
  
<pre>
+
1. Mount your usb drive
# mkdir /mnt/data-01
+
# mkdir /mnt/data-02
+
# mkdir /mnt/data-03
+
# mkdir /mnt/data-04
+
# mount /dev/vgtest/lvdata1 /mnt/data01
+
# mount /dev/vgtest/lvdata2 /mnt/data02
+
# mount /dev/vgtest/lvdata3 /mnt/data03
+
# mount /dev/vgtest/lvdata4 /mnt/data04
+
</pre>
+
  
Finally we can check that everything is in order:
+
<console>
 +
# ##i##mount /dev/sdX# /mnt/usbstick
 +
</console>
  
<pre>
+
'''Where X# is the letter and number of your device. Immediately after you plug your usb in, type `dmesg | tail` in the console and you should see it.'''
# df -h
+
Filesystem                    Size  Used Avail Use% Mounted on
+
(...)
+
/dev/mapper/vgtest-lvdata1    2.0G  96M  1.9G  5% /mnt/data01
+
/dev/mapper/vgtest-lvdata2  1022M  47M  924M  5% /mnt/data02
+
/dev/mapper/vgtest-lvdata3    611M  25M  556M  5% /mnt/data03
+
/dev/mapper/vgtest-lvdata4    2.4G  100M  2.2G  5% /mnt/data04
+
</pre>
+
  
Did you notice the device has changed? Indeed everything is in order, mount just uses another set of symlinks which point to the exact same block devices:
 
  
<pre>
+
2. Copy the zfs .srms to /mnt/usbstick
# ls -l /dev/mapper/vgtest-lvdata[1-4]
+
lrwxrwxrwx 1 root root 7 Dec 28 20:12 /dev/mapper/vgtest-lvdata1 -> ../dm-0
+
lrwxrwxrwx 1 root root 7 Dec 28 20:13 /dev/mapper/vgtest-lvdata2 -> ../dm-1
+
lrwxrwxrwx 1 root root 7 Dec 28 20:13 /dev/mapper/vgtest-lvdata3 -> ../dm-2
+
lrwxrwxrwx 1 root root 7 Dec 28 20:13 /dev/mapper/vgtest-lvdata4 -> ../dm-3
+
</pre>
+
  
== Renaming a volume group and its logical volumes ==
+
<console>
 +
# ##i##cp zfs-core-3.4.37-std350-amd64.srm zfs-kmod-3.4.37-std350-amd64.srm zfs-kmod-3.8.4-alt350-amd64.srm /mnt/usbstick
 +
</console>
  
So far we have four LVs named lvdata1 to lvdata4 mounted on /mnt/data01 to /mnt/data04. It would be more adequate to :
+
3. List your /mnt/usbstick directory and you should see something similar to the following in your /mnt/usbstick
# make the number in our LV names being like "01" instead of "1"
+
# rename our volume groupe to "vgdata" instead of "vgtest"
+
  
To show how dynamic is the LVM world, we will rename our VG and LV on the fly using two commands: '''vgrename''' for acting at the VG level and its counterpart '''lvrename''' to act at the LV level. Starting by the VG or the LVs makes strictly no difference, you can start either way and get the same result. In our example we have chosen to start with the VG:
+
<console>
 +
octopus 3.5.0 # ls -l /mnt/usbstick
 +
total 305028
 +
drwxr-xr-x 3 root root      4096 May 23 20:51 boot
 +
drwxr-xr-x 2 root root      4096 May 23 20:51 bootdisk
 +
drwxr-xr-x 2 root root      4096 May 23 20:51 bootprog
 +
drwxr-xr-x 3 root root      4096 May 23 20:51 efi
 +
drwxr-xr-x 2 root root      4096 May 23 20:51 ntpasswd
 +
-rwxr-xr-x 1 root root      2349 May 23 20:51 readme.txt
 +
drwxr-xr-x 3 root root      4096 May 23 20:52 syslinux
 +
-rwxr-xr-x 1 root root 309252096 May 23 20:51 sysrcd.dat
 +
-rwxr-xr-x 1 root root        45 May 23 20:51 sysrcd.md5
 +
drwxr-xr-x 2 root root      4096 May 23 20:51 usb_inst
 +
-rwxr-xr-x 1 root root    15889 May 23 20:51 usb_inst.sh
 +
-rwxr-xr-x 1 root root      877 May 23 20:51 usbstick.htm
 +
-rwxr-xr-x 1 root root        6 May 23 20:51 version
 +
-rwxr-xr-x 1 root root  2068480 May 23 20:51 zfs-core-3.4.37-std350-amd64.srm
 +
-rwxr-xr-x 1 root root    483328 May 23 20:51 zfs-kmod-3.4.37-std350-amd64.srm
 +
-rwxr-xr-x 1 root root    483328 May 23 20:51 zfs-kmod-3.8.4-alt350-amd64.srm
 +
</console>
  
<pre>
+
Now un-mount your flash drive and boot it into the machine that you want to use ZFS on.
# vgrename vgtest vgdata
+
  Volume group "vgtest" successfully renamed to "vgdata"
+
# lvrename vgdata/lvdata1 vgdata/lvdata01
+
  Renamed "lvdata1" to "lvdata01" in volume group "vgdata"
+
# lvrename vgdata/lvdata2 vgdata/lvdata02
+
  Renamed "lvdata2" to "lvdata02" in volume group "vgdata"
+
# lvrename vgdata/lvdata3 vgdata/lvdata03
+
  Renamed "lvdata3" to "lvdata03" in volume group "vgdata"
+
# lvrename vgdata/lvdata4 vgdata/lvdata04
+
  Renamed "lvdata4" to "lvdata04" in volume group "vgdata"
+
</pre>
+
  
What happened? Simple:
+
<console>
 +
# ##i##umount /mnt/usbstick
 +
</console>
  
<pre>
+
== Booting into the correct kernel ==
# vgs
+
'''If you are using the standard srm:'''
  VG    #PV #LV #SN Attr  VSize VFree
+
<console>
  vgdata  3  4  0 wz--n- 5.99g    0
+
C) Standard 64bit kernel (rescue64) with more choice... >
# lvs
+
1. SystemRescueCd with default options
  LV      VG    Attr  LSize  Origin Snap%  Move Log Copy%  Convert
+
</console>
  lvdata01 vgdata -wi-ao  2.00g                                     
+
  lvdata02 vgdata -wi-ao  1.00g                                     
+
  lvdata03 vgdata -wi-ao 612.00m                                     
+
  lvdata04 vgdata -wi-ao  2.39g
+
</pre>
+
  
Sounds good, our VG and LVs have been renamed! What a command like ''mount'' will say?
+
'''If you are using the alternative srm:'''
 +
<console>
 +
E) Alternative 64bit kernel (altker64) with more choice... >
 +
1. SystemRescueCd with default options
 +
</console>
  
<pre>
+
2. Run 'depmod' so that the new modules can be seen by the kernel.
# mount
+
(...)
+
/dev/mapper/vgtest-lvdata1 on /mnt/data01 type ext4 (rw)
+
/dev/mapper/vgtest-lvdata2 on /mnt/data02 type ext4 (rw)
+
/dev/mapper/vgtest-lvdata3 on /mnt/data03 type ext4 (rw)
+
/dev/mapper/vgtest-lvdata4 on /mnt/data04 type ext4 (rw)
+
</pre>
+
  
Ooops... It is not exactly a bug, mount still shows the symlinks used at the time the LVs were mounted in the VFS and has not updated its information. However once again everything is correct because the underlying  block devices (/dev/dm-0 to /dev/dm-3) did not changed at all. To see the right information the LVs must be unmounted and mounted again:
+
<console>
 +
# ##i##depmod
 +
</console>
  
<pre>
+
{{fancywarning|You must run '''depmod'''. If you don't, then you will get failed to load the ZFS stack error!}}
# umount /mnt/data01
+
(...)
+
# umount /mnt/data04
+
# mount /dev/vgdata/lvdata01 /mnt/data01
+
(...)
+
# mount /dev/vgdata/lvdata04 /mnt/data04
+
# mount
+
/dev/mapper/vgdata-lvdata01 on /mnt/data01 type ext4 (rw)
+
/dev/mapper/vgdata-lvdata02 on /mnt/data02 type ext4 (rw)
+
/dev/mapper/vgdata-lvdata03 on /mnt/data03 type ext4 (rw)
+
/dev/mapper/vgdata-lvdata04 on /mnt/data04 type ext4 (rw)
+
</pre>
+
  
{{fancynote|Using /dev/''volumegroup''/''logicalvolume'' or /dev/''volumegroup''-''logicalvolume'' makes no difference at all, those are two sets of symlinks pointing on the '''exact''' same block device. }}
+
3. Use ZFS as usual. If you type '''zpool status''' and then type '''dmesg | tail''',
  
= Expanding and shrinking the storage space  =
+
you should see something that says:
  
Did you notice in the previous section we have never talked on topic like "create this partition at the beginning" or "allocate 10 sectors more". In LVM you do not have to worry about that kind of problematics: your only concern is more "Do I have the space to allocate a new LV or how can I extend an existing LV?". '''LVM takes cares of the low levels aspects for you, just focus on what you want to do with your storage space.'''
+
<console>
 +
ZFS: Loaded module v0.6.1-1, ZFS pool version 5000, ZFS filesystem version 5
 +
</console>
  
The most common problem with computers is the shortage of space on a volume, most of the time production servers can run months or years without requiring a reboot for various reasons (kernel upgrade, hardware failure...) however they regularly requires to extend their storage space because we do generate more and more data as the time goes. With "traditional" approach like fiddling directly with hard drives partitions, storage space manipulation can easily become a headache mainly because it requires coherent copy to be made and thus application downtimes. Don't expect the situation to be more enjoyable with a SAN storage rather a directly attached storage device... Basically the problems remains the same.
+
If you see the above, then the modules loaded successfully!
  
== Expanding a storage space ==
+
Enjoy System Rescue CD with ZFS :)
  
The most common task for a system administrator is to expand the available storage space. In the LVM world this implies:
 
* Creating a new PV
 
* Adding the PV to the VG (thus extending the VG capacity)
 
* Extending the existing LVs or create new ones
 
* Extending the structures of the filesystems located on a LV in the case a LV is extended (Not all of the filesystems around support that capability).
 
  
=== Bringing a new PV in the VG ===
+
== Using the premade iso ==
  
In the exact same manner we have created our first PV let's create our additional storage device, associate it to a loopback device and then create a PV on it:
+
<big>If you don't want to do any of the above stuff but just want the ISO with the ZFS SRMs already included, simply [http://ftp.osuosl.org/pub/funtoo/distfiles/sysresccd/ Download the ISO].</big>
  
<pre>
+
[[Category:HOWTO]]
# dd if=/dev/zero of=/tmp/hdd4.img bs=2G count=1
+
# losetup /dev/loop3 /tmp/hdd4.img
+
# pvcreate /dev/loop3
+
</pre>
+
 
+
A '''pvs''' should report the new PV with 2 GB of free space:
+
 
+
<pre>
+
# pvs
+
  PV        VG    Fmt  Attr PSize PFree
+
  /dev/loop0 vgdata lvm2 a-  2.00g    0
+
  /dev/loop1 vgdata lvm2 a-  2.00g    0
+
  /dev/loop2 vgdata lvm2 a-  2.00g    0
+
  /dev/loop3        lvm2 a-  2.00g 2.00g
+
</pre>
+
 
+
Excellent! The next step consist of adding this newly created PV inside our VG ''vgdata'', this is where the '''vgextend''' command comes at our rescue:
+
 
+
<pre>
+
# vgextend vgdata /dev/loop3
+
  Volume group "vgdata" successfully extended
+
# vgs
+
  VG    #PV #LV #SN Attr  VSize VFree
+
  vgdata  4  4  0 wz--n- 7.98g 2.00g
+
</pre>
+
 
+
Great, ''vgdata'' is now 8 GB large instead of 6 GB and have 2 GB of free space to allocate to either new LVs either existing LVs.
+
 
+
=== Extending the LV and its filesystem ===
+
 
+
Bringing new LV would demonstrate nothing more nevertheless extending our existing LVs is much more interesting. How can we use our 2GB extra free space? We can, for example, split it in two allocating a 50% to our first (''lvdata01'') and third (''lvdata03'') LV adding 1GB of space to both. The best of the story is that operation is very simple and is realized with a command named '''lvextend''':
+
 
+
<pre>
+
# lvextend vgdata/lvdata01 -l +50%FREE
+
  Extending logical volume lvdata01 to 3.00 GiB
+
  Logical volume lvdata01 successfully resized
+
# lvextend vgdata/lvdata03 -l +50%FREE
+
  Extending logical volume lvdata03 to 1.10 GiB
+
  Logical volume lvdata03 successfully resized
+
</pre>
+
 
+
Ouaps!! We did a mistake there: lvdata01 has the expected size (2GB + 1GB for a grand total of 3 GB) but lvdata03 only grown of 512 MB (for a grand total size of 1.1 GB). Our mistake was obvious: once the first gigabyte (50% of 2GB) of extra space has been given to lvdata01, only one gigabyte remained free on the VG thus when we said "allocate 50% of the remaining gigabyte to ''lvdata03''" LVM added only 512 MB leaving the other half of this gigabyte unused. The '''vgs''' command can confirm this:
+
 
+
<pre>
+
# vgs
+
  VG    #PV #LV #SN Attr  VSize VFree 
+
  vgdata  4  4  0 wz--n- 7.98g 512.00m
+
</pre>
+
 
+
Nevermind about that voluntary mistake we will keep that extra space for a later paragraph :-) What happened to the storage space visible from the operating system?
+
 
+
<pre>
+
# df -h | grep lvdata01
+
/dev/mapper/vgdata-lvdata01  2.0G  96M  1.9G  5% /mnt/data01
+
</pre>
+
 
+
Obviously resizing a LV does not "automagically" resize the filesystem structures to take into account the new LV size making that step part of our duty. Happily for us, ext3 can be resized and better it can be grown when mounted in the VFS. This is known as ''online resizing'' and a few others filesystems supports that capability, among them we can quote ext2 (ext3 without a journal), ext4 (patches integrated very recently as of Nov/Dec 2011), XFS, ResiserFS and BTRFS. To our knowledge, only BTRFS support both online resizing '''and''' online shrinking as of Decembrer 2011, all of the others require a filesystem to be unmounted first before being shrunk.
+
 
+
{{fancynote|Consider using the option -r when invoking lvextend, it asks the command to perform a filesystem resize.}}
+
 
+
Now let's extend (grow) the ext3 filesystem located on lvdata01. As said above, ext3 support online resizing hence we do not need to kick it out of the VFS first:
+
 
+
<pre>
+
# resize2fs /dev/vgdata/lvdata01
+
resize2fs 1.42 (29-Nov-2011)
+
Filesystem at /dev/vgdata/lvdata01 is mounted on /mnt/data01; on-line resizing required
+
old_desc_blocks = 1, new_desc_blocks = 1
+
Performing an on-line resize of /dev/vgdata/lvdata01 to 785408 (4k) blocks.
+
The filesystem on /dev/vgdata/lvdata01 is now 785408 blocks long.
+
 
+
# df -h | grep lvdata01
+
/dev/mapper/vgdata-lvdata01  3.0G  96M  2.8G  4% /mnt/data01
+
</pre>
+
 
+
''Et voila!'' Our  LV has now plenty of new space usable :-) '''We do not bother about ''how'' the storage is organized by LVM amongst the underlying storage devices and it is not our problem after all. We only worry about having our storage requirements being satisfied without any further details. From our point of view everything is seen just as if we were manipulating a single storage device subdivided in several partitions of a dynamic size and always organized in a set of contiguous blocks.'''
+
 
+
Now let's shuffle the cards a bit more: when we examined how the LEs of our LVs were allocated, we saw that ''lvdata01'' (named lvdata1 at this time) consisted of 512 LEs or 512 PEs (because of the 1:1 mapping between those)  spread over two PVs. As we have extended it to use an additional PV, we should see it using 3 segments:
+
 
+
* Segment 1: located on the PV stored on /dev/loop0 (LE/PE #0 to #510)
+
* Segment 2: located on the PV stored on /dev/loop1 (LE/PE #511)
+
* Segment 3: located on the PV stored on /dev/loop1 (LE/PE #512 and followers)
+
 
+
Is it the case? Let's check:
+
 
+
<pre>
+
# lvdisplay -m  vgdata/lvdata01
+
  --- Logical volume ---
+
  LV Name                /dev/vgdata/lvdata01
+
  VG Name                vgdata
+
  LV UUID                fT22is-cmSL-uhwM-zwCd-jeIe-DWO7-Hkj4k3
+
  LV Write Access        read/write
+
  LV Status              available
+
  # open                1
+
  LV Size                3.00 GiB
+
  Current LE            767
+
  Segments              3
+
  Allocation            inherit
+
  Read ahead sectors    auto
+
  - currently set to    256
+
  Block device          253:0
+
 
+
  --- Segments ---
+
  Logical extent 0 to 510:
+
    Type                linear
+
    Physical volume    /dev/loop0
+
    Physical extents    0 to 510
+
 
+
  Logical extent 511 to 511:
+
    Type                linear
+
    Physical volume    /dev/loop1
+
    Physical extents    0 to 0
+
 
+
  Logical extent 512 to 766:
+
    Type                linear
+
    Physical volume    /dev/loop3
+
    Physical extents    0 to 254
+
</pre>
+
 
+
Bingo! Note that if it is true here (LVM uses linear allocation) would not be true in the general case.
+
 
+
{{fancywarning|'''Never mix a local storage device with a SAN disk within the same volume group''' and especially if that later is your system volume. It will bring you a lot of troubles if the SAN disk goes offline or bring weird performance fluctuations as PEs allocated on the SAN will get faster response times than those located on  a local disk. }}
+
 
+
== Shrinking a storage space ==
+
 
+
On some occasions it can be useful to reduce the size of a LV or the size of the VG itself. The principle is similar to what has been demonstrated in the previous section:
+
 
+
# umount the filesystem belong to the LV to be processed (if your filesystem does not support online shrinking)
+
# reduce the filesystem size (if the LV is not to be flushed)
+
# reduce the LV size - OR - remove the LV
+
# remove a PV from the volume group if no longer used to store extents
+
 
+
The simplest case to start with is how a LV can be removed: a good candidate for removal is ''lvdata03'', we failed to resize it and the better would be to scrap it. First unmount it:
+
 
+
<pre>
+
# lvs
+
  LV      VG    Attr  LSize Origin Snap%  Move Log Copy%  Convert
+
  lvdata01 vgdata -wi-ao 3.00g                                     
+
  lvdata02 vgdata -wi-ao 1.00g                                     
+
  lvdata03 vgdata -wi-ao 1.10g                                     
+
  lvdata04 vgdata -wi-ao 2.39g                                     
+
# umount /dev/vgdata/lvdata03
+
# lvs
+
  LV      VG    Attr  LSize Origin Snap%  Move Log Copy%  Convert
+
  lvdata01 vgdata -wi-ao 3.00g                                     
+
  lvdata02 vgdata -wi-ao 1.00g                                     
+
  lvdata03 vgdata -wi-a- 1.10g                                     
+
  lvdata04 vgdata -wi-ao 2.39g
+
</pre>
+
 
+
Noticed the little change with '''lvs'''? It lies in the ''Attr'' field: once the ''lvdata03'' has been unmounted, '''lvs''' tells us the LV is not '''o'''pened anymore (the little o at the rightmost position has been replaced by a dash). The LV still exists but nothing is using it.
+
 
+
To remove ''lvdata03'' use the command '''lvremove''' and confirm the removal by entering 'y' when asked:
+
 
+
<pre>
+
# lvremove vgdata/lvdata03
+
Do you really want to remove active logical volume lvdata03? [y/n]: y
+
  Logical volume "lvdata03" successfully removed
+
# lvs
+
  LV      VG    Attr  LSize Origin Snap%  Move Log Copy%  Convert
+
  lvdata01 vgdata -wi-ao 3.00g                                     
+
  lvdata02 vgdata -wi-ao 1.00g                                     
+
  lvdata04 vgdata -wi-ao 2.39g
+
# vgs
+
  VG    #PV #LV #SN Attr  VSize VFree
+
  vgdata  4  3  0 wz--n- 7.98g 1.60g
+
</pre>
+
 
+
Notice the 1.60 of space has been freed in the VG. What can we do next? Shrinking ''lvdata04'' by 50% giving roughly 1.2GB or 1228MB (1.2*1024) of its size could be a good idea so here we go. First we need to umount the filesystem from the VFS because ext3 '''does not support''' online shrinking.
+
 
+
<pre>
+
# umount /dev/vgdata/lvdata04
+
# e2fsck -f /dev/vgdata/lvdata04
+
e2fsck 1.42 (29-Nov-2011)
+
Pass 1: Checking inodes, blocks, and sizes
+
Pass 2: Checking directory structure
+
Pass 3: Checking directory connectivity
+
Pass 4: Checking reference counts
+
Pass 5: Checking group summary information
+
/dev/vgdata/lvdata04: 11/156800 files (0.0% non-contiguous), 27154/626688 blocks
+
# resize2fs -p /dev/vgdata/lvdata04 -L 1228M
+
# lvreduce /dev/vgdata/lvdata04 -L 1228
+
  WARNING: Reducing active logical volume to 1.20 GiB
+
  THIS MAY DESTROY YOUR DATA (filesystem etc.)
+
Do you really want to reduce lvdata04? [y/n]: y
+
  Reducing logical volume lvdata04 to 1.20 GiB
+
  Logical volume lvdata04 successfully resized
+
oxygen ~ # e2fsck -f /dev/vgdata/lvdata04
+
e2fsck 1.42 (29-Nov-2011)
+
Pass 1: Checking inodes, blocks, and sizes
+
Pass 2: Checking directory structure
+
Pass 3: Checking directory connectivity
+
Pass 4: Checking reference counts
+
Pass 5: Checking group summary information
+
/dev/vgdata/lvdata04: 11/78400 files (0.0% non-contiguous), 22234/314368 blocks
+
</pre>
+
 
+
Not very practical indeed, we can tell '''lvreduce''' to handle the underlying filesystem shrinkage for us. Let's shrink again this time giving a 1 GB volume (1024 MB) in absolute size:
+
 
+
<pre>
+
# lvreduce /dev/vgdata/lvdata04 -r -L 1024
+
fsck from util-linux 2.20.1
+
/dev/mapper/vgdata-lvdata04: clean, 11/78400 files, 22234/314368 blocks
+
resize2fs 1.42 (29-Nov-2011)
+
Resizing the filesystem on /dev/mapper/vgdata-lvdata04 to 262144 (4k) blocks.
+
The filesystem on /dev/mapper/vgdata-lvdata04 is now 262144 blocks long.
+
 
+
  Reducing logical volume lvdata04 to 1.00 GiB
+
  Logical volume lvdata04 successfully resized
+
# lvs
+
  LV      VG    Attr  LSize Origin Snap%  Move Log Copy%  Convert
+
  lvdata01 vgdata -wi-ao 3.00g                                     
+
  lvdata02 vgdata -wi-ao 1.00g                                     
+
  lvdata04 vgdata -wi-a- 1.00g
+
</pre>
+
 
+
{{fancynote|Notice the number of 4k blocks shown: 4096*262144/1024^2 gives 1,073,741,824 bytes either 1 GB.}}
+
 
+
Time to mount the volume again:
+
 
+
<pre>
+
# mount /dev/vgdata/lvdata04 /mnt/data04
+
# df -h | grep lvdata04
+
/dev/mapper/vgdata-lvdata04  1021M  79M  891M  9% /mnt/data04
+
</pre>
+
 
+
And what is going on at the VG level?
+
 
+
<pre>
+
# vgs
+
  VG    #PV #LV #SN Attr  VSize VFree
+
  vgdata  4  3  0 wz--n- 7.98g 2.99g
+
</pre>
+
 
+
Wow, we have near 3 GB of free space inside, a bit more than one of our PV. It could be great if we can free one of the those and of course LVM gives you the possibility to do that. Before going further, let's check what happened at the PVs level:
+
 
+
<pre>
+
# pvs   
+
  PV        VG    Fmt  Attr PSize PFree 
+
  /dev/loop0 vgdata lvm2 a-  2.00g      0
+
  /dev/loop1 vgdata lvm2 a-  2.00g 1016.00m
+
  /dev/loop2 vgdata lvm2 a-  2.00g 1020.00m
+
  /dev/loop3 vgdata lvm2 a-  2.00g    1.00g
+
</pre>
+
 
+
Did you noticed? 1 GB of space has been freed on the last PV (/dev/loop3) since ''lvdata04'' has been shrunk not counting the space freed on ''/dev/loop1'' and ''/dev/loop2'' after the removal of lvdata02.
+
 
+
 
+
Next steo: can we remove a PV directly (the command to remove a PV from a VG is '''vgreduce''')?
+
 
+
<pre>
+
# vgreduce vgdata /dev/loop0
+
  Physical volume "/dev/loop0" still in use
+
</pre>
+
 
+
Of course not, all of our PVs supports the content of our LVs and we must find a manner to move all of the PE (physical extents) actually hold by the PV /dev/loop0 elsewhere withing the VG. But wait a minute, the victory is there yet: we do have some free space in the  /dev/loop0 and we will get more and more free space in it as the displacement process will progress. What is going to happen if, from a concurrent session, we create others LV in ''vgdata'' at the same time the content of  /dev/loop0 is moved? Simple: it can be filled again with the PEs newly allocated.
+
 
+
So before proceeding to the displacement of what ''/dev/loop0'' contents, we must say to LVM: "please don't allocate anymore PEs on ''/dev/loop0''". This is achieved via the parameter ''-x'' of the command '''pvchange''':
+
<pre>
+
# pvchange -x n /dev/loop0
+
  Physical volume "/dev/loop0" changed
+
  1 physical volume changed / 0 physical volumes not changed
+
</pre>
+
 
+
The value ''n'' given to ''-x'' marks the PV as ''unallocable'' (i.e. not usable for future PE allocations). Let's check again the PVs with '''pvs''' and '''pvdisplay''':
+
 
+
<pre>
+
# pvs
+
  PV        VG    Fmt  Attr PSize PFree 
+
  /dev/loop0 vgdata lvm2 --  2.00g      0
+
  /dev/loop1 vgdata lvm2 a-  2.00g 1016.00m
+
  /dev/loop2 vgdata lvm2 a-  2.00g 1020.00m
+
  /dev/loop3 vgdata lvm2 a-  2.00g    1.00g
+
 
+
# pvdisplay /dev/loop0
+
  --- Physical volume ---
+
  PV Name              /dev/loop0
+
  VG Name              vgdata
+
  PV Size              2.00 GiB / not usable 4.00 MiB
+
  Allocatable          NO
+
  PE Size              4.00 MiB
+
  Total PE              511
+
  Free PE              0
+
  Allocated PE          511
+
  PV UUID              b9i1Hi-llka-egCF-2vU2-f7tp-wBqh-qV4qEk
+
</pre>
+
 
+
Great news here, the ''Attrs'' field shows a dash instead of 'a' at the leftmost position meaning the PV is effectively ''not allocatable''. However '''marking a PV not allocatable does not wipe the existing PEs stored on it'''. In other words, it means that data present on the PV remains '''absolutely intact'''. Another positive point lies the remaining capacities of the PVs composing ''vgdata'': the sum of free space available on ''/dev/loop1'', ''/dev/loop2'' and ''/dev/loop3'' is 3060MB (1016MB + 1020MB + 1024MB) so largely sufficient to hold the 2048 MB (2 GB) actually stored on the PV ''/dev/loop0''.
+
 
+
Now we have frozen the allocation of PEs on /dev/loop0 we can make LVM move all of PEs located in this PV on the others PVs composing the VG ''vgdata''. Again, we don't have to worry about the gory details like where LVM will precisely relocate the PEs actually hold by ''/dev/loop0'', our '''only''' concerns is to get all of them moved out of ''/dev/loop0''. That job gets done by:
+
 
+
<pre>
+
# pvmove /dev/loop0
+
  /dev/loop0: Moved: 5.9%
+
  /dev/loop0: Moved: 41.3%
+
  /dev/loop0: Moved: 50.1%
+
  /dev/loop0: Moved: 100.0%
+
</pre>
+
 
+
We don't have to tell LVM the VG name because it already knows that ''/dev/loop0'' belongs to ''vgdata'' and what are the others PVs belonging to that VG usable to host the PEs coming from ''/dev/loop0''. It is absolutely normal for the process to takes some minutes (real life cases can go up to several hours even with SAN disks located on high-end storage hardware which is much more faster than local SATA or even SAS drive).
+
 
+
At the end of the moving process, we can see that the PV ''/dev/loop0'' is totally free:
+
 
+
<pre>
+
# pvs
+
  PV        VG    Fmt  Attr PSize PFree 
+
  /dev/loop0 vgdata lvm2 a-  2.00g    2.00g
+
  /dev/loop1 vgdata lvm2 a-  2.00g 1016.00m
+
  /dev/loop2 vgdata lvm2 a-  2.00g      0
+
  /dev/loop3 vgdata lvm2 a-  2.00g      0
+
 
+
# pvdisplay /dev/loop0
+
  --- Physical volume ---
+
  PV Name              /dev/loop0
+
  VG Name              vgdata
+
  PV Size              2.00 GiB / not usable 4.00 MiB
+
  Allocatable          yes
+
  PE Size              4.00 MiB
+
  Total PE              511
+
  Free PE              511
+
  Allocated PE          0
+
  PV UUID              b9i1Hi-llka-egCF-2vU2-f7tp-wBqh-qV4qEk
+
</pre>
+
 
+
511 PEs free out of a maximum 511 PEs so all of its containt has been successfully spread on the others PVs (the volume is also still marked as "unallocatable", this is normal). Now it is ready to be detached from the VG ''vgdata'' with the help of '''vgreduce''' :
+
 
+
<pre>
+
# vgreduce vgdata /dev/loop0
+
  Removed "/dev/loop0" from volume group "vgdata"
+
</pre>
+
 
+
What happened to ''vgdata''?
+
<pre>
+
# vgs
+
  VG    #PV #LV #SN Attr  VSize VFree 
+
  vgdata  3  3  0 wz--n- 5.99g 1016.00m
+
</pre>
+
 
+
Its storage space falls to ~6GB! What would tell '''pvs'''?
+
 
+
<pre>
+
# pvs
+
  PV        VG    Fmt  Attr PSize PFree 
+
  /dev/loop0        lvm2 a-  2.00g    2.00g
+
  /dev/loop1 vgdata lvm2 a-  2.00g 1016.00m
+
  /dev/loop2 vgdata lvm2 a-  2.00g      0
+
  /dev/loop3 vgdata lvm2 a-  2.00g      0
+
</pre>
+
 
+
''/dev/loop0'' is now a standalone device detached from any VG. However it still contains some LVM metadata that remains to be wiped with the help of the '''pvremove''' command:
+
 
+
{{fancywarning|pvremove/pvmove '''do not destroy the disk content'''. Please *do* a secure erase of the storage device with ''shred'' or any similar tool before disposing of it. }}
+
 
+
<pre>
+
# pvdisplay /dev/loop0
+
  "/dev/loop0" is a new physical volume of "2.00 GiB"
+
  --- NEW Physical volume ---
+
  PV Name              /dev/loop0
+
  VG Name             
+
  PV Size              2.00 GiB
+
  Allocatable          NO
+
  PE Size              0 
+
  Total PE              0
+
  Free PE              0
+
  Allocated PE          0
+
  PV UUID              b9i1Hi-llka-egCF-2vU2-f7tp-wBqh-qV4qEk
+
 
+
# pvremove /dev/loop0
+
  Labels on physical volume "/dev/loop0" successfully wiped
+
# pvdisplay /dev/loop0
+
  No physical volume label read from /dev/loop0
+
  Failed to read physical volume "/dev/loop0"
+
</pre>
+
 
+
Great! Things are just simple than that. In their day to day reality, system administrators drive their show in a extremely close similar manner: they do additional tasks like taking backups of data located on the LVs before doing any risky operation or plan applications shutdown periods prior starting a manipulation with a LVM volume to take extra precautions.
+
 
+
== Replacing a PV (storage device) by another ==
+
 
+
The principle a mix of what has been said in the above sections. The principle is basically:
+
# Create a new PV
+
# Associate it to the VG
+
# Move the contents of the PV to be removed on the remaining PVs composing the VG
+
# Remove the PV from the VG and wipe it
+
 
+
The strategy in this paragraph is to reuse ''/dev/loop0'' and make it replace ''/dev/loop2'' (both devices are of the same size, however we also could have used a bigger ''/dev/loop0'' as well).
+
 
+
Here we go! First we need to (re-)create the LVM metadata to make ''/dev/loop0'' usable by LVM:
+
 
+
<pre>
+
# pvcreate /dev/loop0
+
  Physical volume "/dev/loop0" successfully created
+
</pre>
+
 
+
Then this brand new PV is added to the VG ''vgdata'' thus increasing its size of 2 GB:
+
 
+
<pre>
+
# vgextend vgdata  /dev/loop0
+
  Volume group "vgdata" successfully extended
+
# vgs
+
  VG    #PV #LV #SN Attr  VSize VFree
+
  vgdata  4  3  0 wz--n- 7.98g 2.99g
+
# pvs
+
  PV        VG    Fmt  Attr PSize PFree 
+
  /dev/loop0 vgdata lvm2 a-  2.00g    2.00g
+
  /dev/loop1 vgdata lvm2 a-  2.00g 1016.00m
+
  /dev/loop2 vgdata lvm2 a-  2.00g      0
+
  /dev/loop3 vgdata lvm2 a-  2.00g      0
+
</pre>
+
 
+
Now we have to suspend the allocation of PEs on ''/dev/loop2'' prior to moving its PEs (and freeing some space on it):
+
 
+
<pre>
+
# pvchange -x n /dev/loop2
+
  Physical volume "/dev/loop2" changed
+
  1 physical volume changed / 0 physical volumes not changed
+
# pvs
+
  PV        VG    Fmt  Attr PSize PFree 
+
  /dev/loop0 vgdata lvm2 a-  2.00g    2.00g
+
  /dev/loop1 vgdata lvm2 a-  2.00g 1016.00m
+
  /dev/loop2 vgdata lvm2 --  2.00g      0
+
  /dev/loop3 vgdata lvm2 a-  2.00g      0
+
</pre>
+
 
+
Then we move all of the the PEs on ''/dev/loop2'' to the rest of the VG:
+
 
+
<pre>
+
# pvmove /dev/loop2
+
  /dev/loop2: Moved: 49.9%
+
  /dev/loop2: Moved: 100.0%
+
# pvs
+
  PV        VG    Fmt  Attr PSize PFree 
+
  /dev/loop0 vgdata lvm2 a-  2.00g      0
+
  /dev/loop1 vgdata lvm2 a-  2.00g 1016.00m
+
  /dev/loop2 vgdata lvm2 --  2.00g    2.00g
+
  /dev/loop3 vgdata lvm2 a-  2.00g      0
+
</pre>
+
 
+
Then we remove ''/dev/loop2'' from the VG and we wipe its LVM metadata:
+
 
+
<pre>
+
# vgreduce vgdata /dev/loop2
+
  Removed "/dev/loop2" from volume group "vgdata"
+
# pvremove /dev/loop2
+
  Labels on physical volume "/dev/loop2" successfully wiped
+
</pre>
+
 
+
Final state of the PVs composing ''vgdata'':
+
<pre>
+
# pvs
+
  PV        VG    Fmt  Attr PSize PFree 
+
  /dev/loop0 vgdata lvm2 a-  2.00g      0
+
  /dev/loop1 vgdata lvm2 a-  2.00g 1016.00m
+
  /dev/loop3 vgdata lvm2 a-  2.00g      0
+
</pre>
+
 
+
''/dev/loop0'' took the place of ''/dev/loop2'' :-)
+
 
+
= More advanced topics =
+
 
+
== Backing up the layout ==
+
 
+
== Freezing a VG ==
+
 
+
== LVM snapshots ==
+
 
+
== Linear/Stripped/Mirrored Logical volumes ==
+
 
+
= LVM and Funtoo =
+
 
+
 
+
[[Category:Labs]]
+
[[Category:Filesystems]]
+
[[Category:Articles]]
+

Revision as of 14:15, 19 February 2014

This HOWTO will show you how to create your own zfs srm or download the prebuilt zfs srm module from my website.

Contents

Compiling a compatible kernel

The first thing you need to do is decide for which version of System Rescue CD you will be building for. Each System Rescue CD version has a different set of kernels. Specifically each version has a Stable kernel and an Alternate Kernel. The stable kernels get minor version increments every release (Sometimes major ones but usually doesn't happen too quickly), the Alternate kernels tend to move up much faster. For example System Rescue CD had the following stable/alternate kernels:

Version Stable Alternate
3.7.0 3.4.47 3.9.4
3.5.0 3.4.37 3.8.4

Download kernel and patches

Once you decide which version you want to use, you can go to http://kernel.sysresccd.org/ and pick the version you want. After you do this, download all the stuff in that directory and put them in their own folder (a work directory). The files that are named std-sources are the patches for the regular stable kernel, the patches named alt-sources are the patches for the alternate kernel. Once you have the patches, go to kernel.org and also download the initial release of the kernel sources you want. For example, if you want to build an srm for 3.4.2 which uses the 3.4.35 kernel, just download the 3.4 (3.4.0) kernel. You just need to download the initial release because one of the patches you downloaded earlier is the patch to update the 3.4.0 to 3.4.35.

Once you download the kernel you want, let's extract and patch it.

# tar xf linux-3.4.tar.bz2
# mv linux-3.4 linux-3.4.35-std342-amd64
# bzcat std-sources-3.4-01-stable-3.4.35.patch.bz2 | patch -p1 -d linux-3.4.35-std342-amd64
# bzcat std-sources-3.4-02-fc16.patch.bz2 | patch -p1 -d linux-3.4.35-std342-amd64
# bzcat std-sources-3.4-03-aufs.patch.bz2 | patch -p1 -d linux-3.4.35-std342-amd64

Retrieve kernel configuration

Once that is complete, you will need to get the kernel config specifically for the kernel and architecture you want. The easiest way I found to do this was to download the System Rescue CD you want, and then boot it inside a VM. Once you are inside the VM, go to the /root directory, and it will be a file named kernel-<version>.conf. Copy this file over SSH and put it in a safe location. Also copy it over into the linux kernel directory and rename it to .config.

Build the kernel

Once you copy the kernel config over, do a make menuconfig to begin. You will only need to change two settings inside the config, add the proper kernel name, and remove the old initramfs source files directory. This directory doesn't exist locally. This is a remnant of upstream's build process.

Follow the below tree structure, and make it equal to what is below:


(-std342-amd64) Local version - append to kernel release
      [*] Initial RAM filesystem and RAM disk (initramfs/initrd) support
      ( ) Initramfs source file(s)

The Local version above will end up making the kernel modules directory be 3.4.35-std342-amd64. The reason the kernel modules directory has to be equal to this is because our directory will be merged into the System Rescue CD later on, so the names need to be exactly the same so that the System Rescue CD system can find our kernel modules.

If you want to build an srm (which you do), you also need to change the following inside the kernel config so that squashfs-tools can work later on:


Miscellaneous Filesystems ->
      <*>   SquashFS 4.0 - Squashed file system support
       [*]     Squashfs XATTR support
       [*]     Include support for ZLIB compressed file systems
       [*]     Include support for LZO compressed file systems
       [*]     Include support for XZ compressed file systems
       [ ]     Use 4K device block size?
       [ ]     Additional option for memory-constrained systems

Now save and exit, and continue to build your kernel.

# make bzImage modules
# make modules_install
Note: If you have multiple cores, do a make -j<# of processors+1> to build faster. Example: make -j9 for an 8 core machine.

Using the portage tree

Inside the portage tree, there are ebuilds already prepared that will download the kernel, patch it, and perform any required substitutions. Afterwards you literally just go into your /usr/src/<kernel> directory, and run make. To see the available versions, run the following:

Versions available for standard kernel
# equery y std-sources

[I]3.4.37-r1 | o ~ o o o o o o o o o o o | o 3.4.37 | gentoo

Versions available for alternate kernel
# equery y alt-sources

[I]3.8.4 | o ~ o o o o o o o o o o o | o 3.8.4 | gentoo

I haven't written the ebuilds in a way where it's easy to tell (at first glance) what System Rescue CD version you need. However, you can check the ebuild and you will see the version of the CD inside there. You can also check the chart above.

If you wanted to install the standard kernel sources, you just emerge it like any other application:

# emerge std-sources

Once the kernel and the kernel module are installed, you need to make the SRM.

Creating the SRM

It's time to gather the required files and pack them together in order for it to become an SRM. An SRM is nothing more than a directory that has been compressed with squashfs. You can emerge the "Bliss Initramfs Creator" which has a feature that will automagically build the srm for you.


You also need to have spl, zfs, and zfs-kmod installed on your system before you try to make the srm. Emerging "bliss-initramfs" should automatically pull those packages. If it doesn't, make sure you install them.

# echo "sys-kernel/bliss-initramfs srm" >> /etc/portage/package.use
# emerge bliss-initramfs


You should now have the following directory layout inside the /opt/bliss-initramfs folder:

octopus bliss-initramfs # ls -l
total 117
-rw-r--r-- 1 root root 6718 May 23 18:05 CHANGES
-rw-r--r-- 1 root root  176 May 23 18:05 CREDITS
-rw-r--r-- 1 root root  431 May 23 18:05 HELP
-rw-r--r-- 1 root root 1397 May 23 18:05 LICENSE
-rw-r--r-- 1 root root 1852 May 23 18:05 README
-rw-r--r-- 1 root root 3194 May 23 18:05 USAGE
-rwxr-xr-x 1 root root 2891 May 23 18:05 createInit
drwxr-xr-x 3 root root    4 May 23 18:11 files
drwxr-xr-x 3 root root    6 May 23 18:11 hooks
drwxr-xr-x 2 root root    5 May 23 22:01 resources


Then run the createInit script and follow the instructions for the kernel you want to make the srm for (In this case it's 3.4.37-std350-amd64):

-----------------------------------
| Bliss Initramfs Creator - v1.8.1
| Author: Jonathan Vasquez <jvasquez1011@gmail.com>
| Distributed under the Simplified BSD License
-----------------------------------

>>> Which initramfs would you like to generate:
>> 1. ZFS
>> 2. Encrypted ZFS (LUKS + ZFS)
>> 3. More Options
>> 4. Exit Program

>>> Current choice [1]: 3 ↵

>>> More Options:

>> 1. ZFS - System Rescue Module
>> 2. Back
>> 3. Exit Program

>>> Current choice [1]: 

>>> Creating a ZFS System Rescue Module!

>>> Do you want to use the current kernel: 3.8.13-ALL? [Y/n]: n ↵

>>> Please enter the kernel name: 3.4.37-std350-amd64 ↵

>>> Detected 64 bit platform
>>> Checking to see if modules directory exists for 3.4.37-std350-amd64...
>>> Creating SRMs for 3.4.37-std350-amd64...
>>> Creating directory structure for initramfs...
>>> Checking preliminary binaries...
>>> Checking binaries...
>>> Using ZFS
>>> Checking modules...
>>> Copying binaries...
>>> Copying modules...
>>> Copying documentation...
>>> Copying udev rules...
>>> Compressing kernel modules...
>>> Getting dependencies...
>>> Copying dependencies...
>>> Configuring files...
>>> Creating and Packing SRMs...
>>> Complete :)


Now you should have two new files:

-rw-r--r-- 1 root root 2068480 May 24 01:05 zfs-core-3.4.37-std350-amd64.srm
-rw-r--r-- 1 root root  483328 May 24 01:05 zfs-kmod-3.4.37-std350-amd64.srm


Now all you need to do is put those two files in the root of your USB directory.

If you are making srms for both the standard and alternate kernels, you will end up with two zfs-core files and two zfs-kmod files (1 set for each kernel). You don't need to put the zfs-core that it makes for one of the kernels. The zfs-core srm only has zfs program binaries, man pages, udev rules, and a few other things. The zfs-kmod is the srm that has the kernel modules (and only the kernel modules). So you can easily just put two zfs-kmods (one for each kernel version) and just re-use the same zfs-core for both of them.

Using the prebuilt srm

Download the SRM

If you didn't build your own srm and want to use the prebuilt one, just emerge "zfs-srm" from the tree:

# emerge zfs-srm

You can check available versions just as you did above for the kernels:

Available versions of the zfs-srm
# equery y zfs-srm

[I]3.5.0 | o ~ o o o o o o o o o o o | o 3.5.0 | gentoo

(These versions match the version of the System Rescue CD).

Installing the zfs-srm will automatically pull the "bliss-isomaker" package which is just a script that assists you with rebuilding the System Rescue CD ISO so that the ISO includes the ZFS srms. The script lets you make a bootable ISO or a bootable USB.

Once it's installed, switch to the /opt/bliss-isomaker folder

# cd /opt/bliss-isomaker

You should now see a directory layout that looks similar to this:

octopus bliss-isomaker # ls -l
total 100
drwxr-xr-x 2 root root    5 May 24 01:41 3.5.0
-rw-r--r-- 1 root root 1397 May 24 01:31 LICENSE
-rw-r--r-- 1 root root  312 May 24 01:31 README
-rw-r--r-- 1 root root  576 May 24 01:31 USAGE
-rwxr-xr-x 1 root root 3228 May 24 01:31 create
drwxr-xr-x 2 root root    3 May 24 01:41 iso
drwxr-xr-x 2 root root    4 May 24 01:41 resources
drwxr-xr-x 2 root root    3 May 24 01:41 srm

The layout is as follows:

  • 3.5.0 - This folder contains the System Rescue CD 3.5.0 specific srms for both the standard and alternate kernel that were installed by emerge.
  • create - A script to automatically recreate the system rescue cd iso or usb with the zfs stuff included
  • iso - directory to place your system rescue cd iso in
  • srm - directory that has the srms
  • resources - the files in this folder contain function calls that the 'create' script uses. You don't need to worry about these.

Installing the SRM

There are a few ways to do this, you can either use one of the scripts, or you can do it manually. Before anything, make sure to copy the SRMs from the <Version> folder to the srm folder if you are using a prebuilt one:

# cp 3.5.0/* srm

Generating a new iso

If you just want to remake a new iso so that you can burn onto a cd or use in a virtual machine, just copy your iso over to the iso directory, and run the 'create' script. The new iso will be located in the iso directory as well with a -zfs ending.

Running the script with 1 (or iso) chooses to make an iso. You can also pass the version as well.
# ./create 1

or

# ./create iso 3.5.0

Creating a fresh usb

If you want to have zfs on a usb with system rescue cd, put the iso in your iso dir, and then run the usb script. It will ask you what usb you want to format (This will delete everything), and then install system rescue cd onto it. Once that is done it will copy the zfs stuff over.


Running the script with 2 (or usb) chooses to make an usb. You can also pass the version as well.
# ./create 2

or

# ./create usb 3.5.0

Manual installation onto a usb

Assuming that your installing to a flash drive and that the flash drive is mounted in /mnt

1. Mount your usb drive

# mount /dev/sdX# /mnt/usbstick

Where X# is the letter and number of your device. Immediately after you plug your usb in, type `dmesg | tail` in the console and you should see it.


2. Copy the zfs .srms to /mnt/usbstick

# cp zfs-core-3.4.37-std350-amd64.srm zfs-kmod-3.4.37-std350-amd64.srm zfs-kmod-3.8.4-alt350-amd64.srm /mnt/usbstick

3. List your /mnt/usbstick directory and you should see something similar to the following in your /mnt/usbstick

octopus 3.5.0 # ls -l /mnt/usbstick
total 305028
drwxr-xr-x 3 root root      4096 May 23 20:51 boot
drwxr-xr-x 2 root root      4096 May 23 20:51 bootdisk
drwxr-xr-x 2 root root      4096 May 23 20:51 bootprog
drwxr-xr-x 3 root root      4096 May 23 20:51 efi
drwxr-xr-x 2 root root      4096 May 23 20:51 ntpasswd
-rwxr-xr-x 1 root root      2349 May 23 20:51 readme.txt
drwxr-xr-x 3 root root      4096 May 23 20:52 syslinux
-rwxr-xr-x 1 root root 309252096 May 23 20:51 sysrcd.dat
-rwxr-xr-x 1 root root        45 May 23 20:51 sysrcd.md5
drwxr-xr-x 2 root root      4096 May 23 20:51 usb_inst
-rwxr-xr-x 1 root root     15889 May 23 20:51 usb_inst.sh
-rwxr-xr-x 1 root root       877 May 23 20:51 usbstick.htm
-rwxr-xr-x 1 root root         6 May 23 20:51 version
-rwxr-xr-x 1 root root   2068480 May 23 20:51 zfs-core-3.4.37-std350-amd64.srm
-rwxr-xr-x 1 root root    483328 May 23 20:51 zfs-kmod-3.4.37-std350-amd64.srm
-rwxr-xr-x 1 root root    483328 May 23 20:51 zfs-kmod-3.8.4-alt350-amd64.srm

Now un-mount your flash drive and boot it into the machine that you want to use ZFS on.

# umount /mnt/usbstick

Booting into the correct kernel

If you are using the standard srm:

C) Standard 64bit kernel (rescue64) with more choice... >
1. SystemRescueCd with default options

If you are using the alternative srm:

E) Alternative 64bit kernel (altker64) with more choice... >
1. SystemRescueCd with default options

2. Run 'depmod' so that the new modules can be seen by the kernel.

# depmod
Warning: You must run depmod. If you don't, then you will get failed to load the ZFS stack error!

3. Use ZFS as usual. If you type zpool status and then type dmesg | tail,

you should see something that says:

ZFS: Loaded module v0.6.1-1, ZFS pool version 5000, ZFS filesystem version 5

If you see the above, then the modules loaded successfully!

Enjoy System Rescue CD with ZFS :)


Using the premade iso

If you don't want to do any of the above stuff but just want the ISO with the ZFS SRMs already included, simply Download the ISO.