|
|
| Line 1: |
Line 1: |
| This HOWTO will show you how to create your own zfs srm or download the prebuilt zfs srm module from my website.
| | == Description == |
|
| |
|
| == Compiling a compatible kernel ==
| | Funtoo Linux has a number of forked packages, and sometimes these packages can get out-of-date relative to Gentoo Linux. Periodically, we need to update ebuilds in Funtoo Linux so that they are current. To make this task easier, we have an automated script that compares versions of forked ebuilds in Funtoo Linux to those in Gentoo Linux. |
|
| |
|
| The first thing you need to do is decide for which version of System Rescue CD you will be building for. Each System Rescue CD version has a different set of kernels. Specifically each version has a Stable kernel and an Alternate Kernel. The stable kernels get minor version increments every release (Sometimes major ones but usually doesn't happen too quickly), the Alternate kernels tend to move up much faster. For example System Rescue CD had the following stable/alternate kernels:
| | Here's how it works: Below, you can see a list of all ebuilds in Gentoo Linux that have higher version numbers than those in Funtoo Linux. This list is updated ''hourly''. |
| | {{#get_web_data:url=http://ports.funtoo.org/my.json|format=json|data=fcx8664=fcx8664}} |
| | == Funtoo / Gentoo Comparison (funtoo-current, x86-64bit) == |
| | |
| | {{#arraymaptemplate:{{#external_value:fcx8664}}|PkgCompareDisplay|,||}} |
|
| |
|
| {| class="wikitable"
| | Some of these ebuilds may be intentionally kept at earlier versions, although in general, we want to update ebuilds to the most recent version that works reliably. Here are some technical caveats: |
| |-
| |
| ! Version !! Stable !! Alternate
| |
| |-
| |
| | 3.7.0 || 3.4.47 || 3.9.4
| |
| |-
| |
| | 3.5.0 || 3.4.37 || 3.8.4
| |
| |}
| |
|
| |
|
| ===Download kernel and patches===
| | * Rev parts of ebuilds ("<tt>-rX</tt>") are not considered in version comparisons, since sometimes Gentoo and Funtoo revisions are not analogous. |
| Once you decide which version you want to use, you can go to http://kernel.sysresccd.org/ and pick the version you want. After you do this, download all the stuff in that directory and put them in their own folder (a work directory). The files that are named std-sources are the patches for the regular stable kernel, the patches named alt-sources are the patches for the alternate kernel. Once you have the patches, go to kernel.org and also download the initial release of the kernel sources you want. For example, if you want to build an srm for 3.4.2 which uses the 3.4.35 kernel, just download the 3.4 (3.4.0) kernel. You just need to download the initial release because one of the patches you downloaded earlier is the patch to update the 3.4.0 to 3.4.35.
| | * Masks are not used for this version comparison. The latest unmasked or masked version in Gentoo is compared against the latest masked or unmasked version in Funtoo. |
|
| |
|
| Once you download the kernel you want, let's extract and patch it.
| | == How To Help == |
|
| |
|
| <console>
| | Here's how you can help: If you see an package above that appears to be out-of-date, and you believe we should update to a newer version, [http://bugs.funtoo.org open a bug] on the Funtoo bug tracker. We can look into the ebuild and will give you feedback on whether we feel a new version of the ebuild should be added. |
| # ##i##tar xf linux-3.4.tar.bz2
| |
| # ##i##mv linux-3.4 linux-3.4.35-std342-amd64
| |
| # ##i##bzcat std-sources-3.4-01-stable-3.4.35.patch.bz2 | patch -p1 -d linux-3.4.35-std342-amd64
| |
| # ##i##bzcat std-sources-3.4-02-fc16.patch.bz2 | patch -p1 -d linux-3.4.35-std342-amd64
| |
| # ##i##bzcat std-sources-3.4-03-aufs.patch.bz2 | patch -p1 -d linux-3.4.35-std342-amd64
| |
| </console>
| |
|
| |
|
| ===Retrieve kernel configuration===
| | If there is some consensus that it should be updated, then generally Funtoo staff will update the ebuild for you, since generally we are familiar with why the ebuild was forked and be able to efficiently update the package to the most recent version. |
| Once that is complete, you will need to get the kernel config specifically for the kernel and architecture you want. The easiest way I found to do this was to download the System Rescue CD you want, and then boot it inside a VM. Once you are inside the VM, go to the /root directory, and it will be a file named kernel-<version>.conf. Copy this file over SSH and put it in a safe location. Also copy it over into the linux kernel directory and rename it to .config.
| |
|
| |
|
| ===Build the kernel===
| | However, in some cases it may be possible for you can also help with the update process, if you are so inclined, by forking the [https://github.com/funtoo/funtoo-overlay funtoo-overlay on GitHub] and working on updating the ebuild yourself. Reference your commits in a bug report on our bug tracker. If you are going to help with ebuild writing, it's important to first familiarize yourself with why we forked the ebuild in the first place. The best way to dig up information on the history of the forked Funtoo package is to type "<tt>git log .</tt>" in the ebuild directory. This will show log information for the package only, and you can look at the history of the ebuild. |
| Once you copy the kernel config over, do a make menuconfig to begin. You will only need to change two settings inside the config, add the proper kernel name, and remove the old initramfs source files directory. This directory doesn't exist locally. This is a remnant of upstream's build process.
| |
|
| |
|
| Follow the below tree structure, and make it equal to what is below:
| | When updating ebuilds, we do want to look at Gentoo's most recent changes. Some of our ebuilds are mild variations of Gentoo's ebuilds, while others are complete rewrites. In general, we like to make our work as easy as possible, so we don't want to duplicate work, although sometimes there are exceptions where we will rewrite complex core ebuilds to make them more ''maintainable'' and ''easy to understand'' (our <tt>sys-devel/gcc</tt> ebuilds are an example of this.) |
|
| |
|
| <console>
| | Of course, there's a reason we originally forked an ebuild, and we want to make the process of updating ebuilds as easy as possible. Sometimes, we will fork a package from Gentoo for a specific reason, such as a build breakage, and once the issue is resolved upstream, we will want to ''unfork'' the package by removing it from funtoo-overlay entirely. In general, we want to minimize the number of forked ebuilds to the minimum number necessary. |
| General Setup ->
| |
| (-std342-amd64) Local version - append to kernel release
| |
| [*] Initial RAM filesystem and RAM disk (initramfs/initrd) support () Initramfs source file(s)
| |
| </console>
| |
|
| |
|
| The Local version above will end up making the kernel modules directory be 3.4.35-std342-amd64. The reason the kernel modules directory has to be equal to this is because our directory will be merged into the System Rescue CD later on, so the names need to be exactly the same so that the System Rescue CD system can find our kernel modules.
| | [[Category:Activity]] |
| | |
| If you want to build an srm (which you do), you also need to change the following inside the kernel config so that squashfs-tools can work later on:
| |
| | |
| <console>
| |
| File Systems ->
| |
| Miscellaneous Filesystems ->
| |
| <*> SquashFS 4.0 - Squashed file system support
| |
| [*] Squashfs XATTR support
| |
| [*] Include support for ZLIB compressed file systems
| |
| [*] Include support for LZO compressed file systems
| |
| [*] Include support for XZ compressed file systems
| |
| [ ] Use 4K device block size?
| |
| [ ] Additional option for memory-constrained systems
| |
| </console>
| |
| | |
| Now save and exit, and continue to build your kernel.
| |
| | |
| <console>
| |
| # ##i##make bzImage modules
| |
| # ##i##make modules_install
| |
| </console>
| |
| | |
| {{fancynote|If you have multiple cores, do a make -j<# of processors+1> to build faster. Example: '''make -j9''' for an 8 core machine.}}
| |
| | |
| === Using the portage tree ===
| |
| Inside the portage tree, there are ebuilds already prepared that will download the kernel, patch it, and perform any required substitutions. Afterwards you literally just go into your /usr/src/<kernel> directory, and run make. To see the available versions, run the following:
| |
| | |
| <console>
| |
| Versions available for standard kernel
| |
| # ##i##equery y std-sources
| |
| | |
| [I]3.4.37-r1 | o ~ o o o o o o o o o o o | o 3.4.37 | gentoo
| |
| | |
| Versions available for alternate kernel
| |
| # ##i##equery y alt-sources
| |
| | |
| [I]3.8.4 | o ~ o o o o o o o o o o o | o 3.8.4 | gentoo
| |
| </console>
| |
| | |
| I haven't written the ebuilds in a way where it's easy to tell (at first glance) what System Rescue CD version you need. However, you can check the ebuild and you will see the version of the CD inside there. You can also check the chart above.
| |
| | |
| If you wanted to install the standard kernel sources, you just emerge it like any other application:
| |
| | |
| <console>
| |
| # ##i##emerge std-sources
| |
| </console>
| |
| | |
| Once the kernel and the kernel module are installed, you need to make the SRM.
| |
| | |
| == Creating the SRM ==
| |
| It's time to gather the required files and pack them together in order for it to become an SRM. An SRM is nothing more than a directory that has been compressed with squashfs. You can emerge the "Bliss Initramfs Creator" which has a feature that will automagically build the srm for you.
| |
| | |
| | |
| You also need to have spl, zfs, and zfs-kmod installed on your system before you try to make the srm. Emerging '''"bliss-initramfs"''' should automatically pull those packages. If it doesn't, make sure you install them.
| |
| | |
| <console>
| |
| # ##i##echo "sys-kernel/bliss-initramfs srm" >> /etc/portage/package.use
| |
| # ##i##emerge bliss-initramfs
| |
| </console>
| |
| | |
| | |
| You should now have the following directory layout inside the '''/opt/bliss-initramfs''' folder:
| |
| | |
| <console>
| |
| octopus bliss-initramfs # ls -l
| |
| total 117
| |
| -rw-r--r-- 1 root root 6718 May 23 18:05 CHANGES
| |
| -rw-r--r-- 1 root root 176 May 23 18:05 CREDITS
| |
| -rw-r--r-- 1 root root 431 May 23 18:05 HELP
| |
| -rw-r--r-- 1 root root 1397 May 23 18:05 LICENSE
| |
| -rw-r--r-- 1 root root 1852 May 23 18:05 README
| |
| -rw-r--r-- 1 root root 3194 May 23 18:05 USAGE
| |
| -rwxr-xr-x 1 root root 2891 May 23 18:05 createInit
| |
| drwxr-xr-x 3 root root 4 May 23 18:11 files
| |
| drwxr-xr-x 3 root root 6 May 23 18:11 hooks
| |
| drwxr-xr-x 2 root root 5 May 23 22:01 resources
| |
| </console>
| |
| | |
| | |
| Then run the '''createInit''' script and follow the instructions for the kernel you want to make the srm for (In this case it's '''3.4.37-std350-amd64'''):
| |
| | |
| <console>
| |
| -----------------------------------
| |
| | Bliss Initramfs Creator - v1.8.1
| |
| | Author: Jonathan Vasquez <jvasquez1011@gmail.com>
| |
| | Distributed under the Simplified BSD License
| |
| -----------------------------------
| |
| | |
| >>> Which initramfs would you like to generate:
| |
| >> 1. ZFS
| |
| >> 2. Encrypted ZFS (LUKS + ZFS)
| |
| >> 3. More Options
| |
| >> 4. Exit Program
| |
| | |
| >>> Current choice [1]: ##i##3 ↵
| |
| | |
| >>> More Options:
| |
| | |
| >> 1. ZFS - System Rescue Module
| |
| >> 2. Back
| |
| >> 3. Exit Program
| |
| | |
| >>> Current choice [1]: ##i##↵
| |
| | |
| >>> Creating a ZFS System Rescue Module!
| |
| | |
| >>> Do you want to use the current kernel: 3.8.13-ALL? [Y/n]: ##i##n ↵
| |
| | |
| >>> Please enter the kernel name: ##i##3.4.37-std350-amd64 ↵
| |
| | |
| >>> Detected 64 bit platform
| |
| >>> Checking to see if modules directory exists for 3.4.37-std350-amd64...
| |
| >>> Creating SRMs for 3.4.37-std350-amd64...
| |
| >>> Creating directory structure for initramfs...
| |
| >>> Checking preliminary binaries...
| |
| >>> Checking binaries...
| |
| >>> Using ZFS
| |
| >>> Checking modules...
| |
| >>> Copying binaries...
| |
| >>> Copying modules...
| |
| >>> Copying documentation...
| |
| >>> Copying udev rules...
| |
| >>> Compressing kernel modules...
| |
| >>> Getting dependencies...
| |
| >>> Copying dependencies...
| |
| >>> Configuring files...
| |
| >>> Creating and Packing SRMs...
| |
| >>> Complete :)
| |
| </console>
| |
| | |
| | |
| Now you should have two new files:
| |
| | |
| <console>
| |
| -rw-r--r-- 1 root root 2068480 May 24 01:05 zfs-core-3.4.37-std350-amd64.srm
| |
| -rw-r--r-- 1 root root 483328 May 24 01:05 zfs-kmod-3.4.37-std350-amd64.srm
| |
| </console>
| |
| | |
| | |
| Now all you need to do is put those two files in the root of your USB directory.
| |
| | |
| If you are making srms for both the standard and alternate kernels, you will end up with two zfs-core files and two zfs-kmod files (1 set for each kernel). You don't need to put the zfs-core that it makes for one of the kernels. The zfs-core srm only has zfs program binaries, man pages, udev rules, and a few other things. The zfs-kmod is the srm that has the kernel modules (and only the kernel modules). So you can easily just put two zfs-kmods (one for each kernel version) and just re-use the same zfs-core for both of them.
| |
| | |
| == Using the prebuilt srm ==
| |
| | |
| === Download the SRM ===
| |
| If you didn't build your own srm and want to use the prebuilt one, just emerge '''"zfs-srm"''' from the tree:
| |
| | |
| <console>
| |
| # ##i##emerge zfs-srm
| |
| </console>
| |
| | |
| You can check available versions just as you did above for the kernels:
| |
| | |
| <console>
| |
| Available versions of the zfs-srm
| |
| # ##i##equery y zfs-srm
| |
| | |
| [I]3.5.0 | o ~ o o o o o o o o o o o | o 3.5.0 | gentoo
| |
| </console>
| |
| | |
| (These versions match the version of the System Rescue CD).
| |
| | |
| Installing the zfs-srm will automatically pull the '''"bliss-isomaker"''' package which is just a script that assists you with rebuilding the System Rescue CD ISO so that the ISO includes the ZFS srms. The script lets you make a bootable ISO or a bootable USB.
| |
| | |
| Once it's installed, switch to the /opt/bliss-isomaker folder
| |
| | |
| <console>
| |
| # ##i##cd /opt/bliss-isomaker
| |
| </console>
| |
| | |
| You should now see a directory layout that looks similar to this:
| |
| | |
| <console>
| |
| octopus bliss-isomaker # ls -l
| |
| total 100
| |
| drwxr-xr-x 2 root root 5 May 24 01:41 3.5.0
| |
| -rw-r--r-- 1 root root 1397 May 24 01:31 LICENSE
| |
| -rw-r--r-- 1 root root 312 May 24 01:31 README
| |
| -rw-r--r-- 1 root root 576 May 24 01:31 USAGE
| |
| -rwxr-xr-x 1 root root 3228 May 24 01:31 create
| |
| drwxr-xr-x 2 root root 3 May 24 01:41 iso
| |
| drwxr-xr-x 2 root root 4 May 24 01:41 resources
| |
| drwxr-xr-x 2 root root 3 May 24 01:41 srm
| |
| </console>
| |
| | |
| The layout is as follows:
| |
| * 3.5.0 - This folder contains the System Rescue CD 3.5.0 specific srms for both the standard and alternate kernel that were installed by emerge.
| |
| * create - A script to automatically recreate the system rescue cd iso or usb with the zfs stuff included
| |
| * iso - directory to place your system rescue cd iso in
| |
| * srm - directory that has the srms
| |
| * resources - the files in this folder contain function calls that the 'create' script uses. You don't need to worry about these.
| |
| | |
| === Installing the SRM ===
| |
| There are a few ways to do this, you can either use one of the scripts, or you can do it manually. Before anything, make sure to copy the SRMs from the <Version> folder to the srm folder if you are using a prebuilt one:
| |
| | |
| <console>
| |
| # ##i##cp 3.5.0/* srm
| |
| </console>
| |
| | |
| ==== Generating a new iso ====
| |
| If you just want to remake a new iso so that you can burn onto a cd or use in a virtual machine, just copy your iso over to the iso directory, and run the 'create' script. The new iso will be located in the iso directory as well with a -zfs ending.
| |
| | |
| <console>
| |
| Running the script with 1 (or iso) chooses to make an iso. You can also pass the version as well.
| |
| # ##i##./create 1
| |
| | |
| or
| |
| | |
| # ##i##./create iso 3.5.0
| |
| </console>
| |
| | |
| ==== Creating a fresh usb ====
| |
| If you want to have zfs on a usb with system rescue cd, put the iso in your iso dir, and then run the usb script. It will ask you what usb you want to format (This will delete everything), and then install system rescue cd onto it. Once that is done it will copy the zfs stuff over.
| |
| | |
| | |
| <console>
| |
| Running the script with 2 (or usb) chooses to make an usb. You can also pass the version as well.
| |
| # ##i##./create 2
| |
| | |
| or
| |
| | |
| # ##i##./create usb 3.5.0
| |
| </console>
| |
| | |
| ==== Manual installation onto a usb ====
| |
| Assuming that your installing to a flash drive and that the flash drive is mounted in /mnt
| |
| | |
| 1. Mount your usb drive
| |
| | |
| <console>
| |
| # ##i##mount /dev/sdX# /mnt/usbstick
| |
| </console>
| |
| | |
| '''Where X# is the letter and number of your device. Immediately after you plug your usb in, type `dmesg | tail` in the console and you should see it.'''
| |
| | |
| | |
| 2. Copy the zfs .srms to /mnt/usbstick
| |
| | |
| <console>
| |
| # ##i##cp zfs-core-3.4.37-std350-amd64.srm zfs-kmod-3.4.37-std350-amd64.srm zfs-kmod-3.8.4-alt350-amd64.srm /mnt/usbstick
| |
| </console>
| |
| | |
| 3. List your /mnt/usbstick directory and you should see something similar to the following in your /mnt/usbstick
| |
| | |
| <console>
| |
| octopus 3.5.0 # ls -l /mnt/usbstick
| |
| total 305028
| |
| drwxr-xr-x 3 root root 4096 May 23 20:51 boot
| |
| drwxr-xr-x 2 root root 4096 May 23 20:51 bootdisk
| |
| drwxr-xr-x 2 root root 4096 May 23 20:51 bootprog
| |
| drwxr-xr-x 3 root root 4096 May 23 20:51 efi
| |
| drwxr-xr-x 2 root root 4096 May 23 20:51 ntpasswd
| |
| -rwxr-xr-x 1 root root 2349 May 23 20:51 readme.txt
| |
| drwxr-xr-x 3 root root 4096 May 23 20:52 syslinux
| |
| -rwxr-xr-x 1 root root 309252096 May 23 20:51 sysrcd.dat
| |
| -rwxr-xr-x 1 root root 45 May 23 20:51 sysrcd.md5
| |
| drwxr-xr-x 2 root root 4096 May 23 20:51 usb_inst
| |
| -rwxr-xr-x 1 root root 15889 May 23 20:51 usb_inst.sh
| |
| -rwxr-xr-x 1 root root 877 May 23 20:51 usbstick.htm
| |
| -rwxr-xr-x 1 root root 6 May 23 20:51 version
| |
| -rwxr-xr-x 1 root root 2068480 May 23 20:51 zfs-core-3.4.37-std350-amd64.srm
| |
| -rwxr-xr-x 1 root root 483328 May 23 20:51 zfs-kmod-3.4.37-std350-amd64.srm
| |
| -rwxr-xr-x 1 root root 483328 May 23 20:51 zfs-kmod-3.8.4-alt350-amd64.srm
| |
| </console>
| |
| | |
| Now un-mount your flash drive and boot it into the machine that you want to use ZFS on.
| |
| | |
| <console>
| |
| # ##i##umount /mnt/usbstick
| |
| </console>
| |
| | |
| == Booting into the correct kernel ==
| |
| '''If you are using the standard srm:'''
| |
| <console>
| |
| C) Standard 64bit kernel (rescue64) with more choice... >
| |
| 1. SystemRescueCd with default options
| |
| </console>
| |
| | |
| '''If you are using the alternative srm:'''
| |
| <console>
| |
| E) Alternative 64bit kernel (altker64) with more choice... >
| |
| 1. SystemRescueCd with default options
| |
| </console>
| |
| | |
| 2. Run 'depmod' so that the new modules can be seen by the kernel.
| |
| | |
| <console>
| |
| # ##i##depmod
| |
| </console>
| |
| | |
| {{fancywarning|You must run '''depmod'''. If you don't, then you will get failed to load the ZFS stack error!}}
| |
| | |
| 3. Use ZFS as usual. If you type '''zpool status''' and then type '''dmesg | tail''',
| |
| | |
| you should see something that says:
| |
| | |
| <console>
| |
| ZFS: Loaded module v0.6.1-1, ZFS pool version 5000, ZFS filesystem version 5
| |
| </console>
| |
| | |
| If you see the above, then the modules loaded successfully!
| |
| | |
| Enjoy System Rescue CD with ZFS :)
| |
| | |
| | |
| == Using the premade iso ==
| |
| | |
| <big>If you don't want to do any of the above stuff but just want the ISO with the ZFS SRMs already included, simply [http://ftp.osuosl.org/pub/funtoo/distfiles/sysresccd/ Download the ISO].</big>
| |
| | |
| [[Category:HOWTO]] | |
Description
Funtoo Linux has a number of forked packages, and sometimes these packages can get out-of-date relative to Gentoo Linux. Periodically, we need to update ebuilds in Funtoo Linux so that they are current. To make this task easier, we have an automated script that compares versions of forked ebuilds in Funtoo Linux to those in Gentoo Linux.
Here's how it works: Below, you can see a list of all ebuilds in Gentoo Linux that have higher version numbers than those in Funtoo Linux. This list is updated hourly.
Error while fetching data from URL http://ports.funtoo.org/my.json: $2.
Error fetching URL: Failed to connect to ports.funtoo.org port 80: No route to host
There was a problem during the HTTP request: 0 Error
Could not get URL http://ports.funtoo.org/my.json after 3 tries.
Funtoo / Gentoo Comparison (funtoo-current, x86-64bit)
Some of these ebuilds may be intentionally kept at earlier versions, although in general, we want to update ebuilds to the most recent version that works reliably. Here are some technical caveats:
- Rev parts of ebuilds ("-rX") are not considered in version comparisons, since sometimes Gentoo and Funtoo revisions are not analogous.
- Masks are not used for this version comparison. The latest unmasked or masked version in Gentoo is compared against the latest masked or unmasked version in Funtoo.
How To Help
Here's how you can help: If you see an package above that appears to be out-of-date, and you believe we should update to a newer version, open a bug on the Funtoo bug tracker. We can look into the ebuild and will give you feedback on whether we feel a new version of the ebuild should be added.
If there is some consensus that it should be updated, then generally Funtoo staff will update the ebuild for you, since generally we are familiar with why the ebuild was forked and be able to efficiently update the package to the most recent version.
However, in some cases it may be possible for you can also help with the update process, if you are so inclined, by forking the funtoo-overlay on GitHub and working on updating the ebuild yourself. Reference your commits in a bug report on our bug tracker. If you are going to help with ebuild writing, it's important to first familiarize yourself with why we forked the ebuild in the first place. The best way to dig up information on the history of the forked Funtoo package is to type "git log ." in the ebuild directory. This will show log information for the package only, and you can look at the history of the ebuild.
When updating ebuilds, we do want to look at Gentoo's most recent changes. Some of our ebuilds are mild variations of Gentoo's ebuilds, while others are complete rewrites. In general, we like to make our work as easy as possible, so we don't want to duplicate work, although sometimes there are exceptions where we will rewrite complex core ebuilds to make them more maintainable and easy to understand (our sys-devel/gcc ebuilds are an example of this.)
Of course, there's a reason we originally forked an ebuild, and we want to make the process of updating ebuilds as easy as possible. Sometimes, we will fork a package from Gentoo for a specific reason, such as a build breakage, and once the issue is resolved upstream, we will want to unfork the package by removing it from funtoo-overlay entirely. In general, we want to minimize the number of forked ebuilds to the minimum number necessary.