Difference between pages "Repository Configuration" and "Whole disk ZFS Install Guide"

(Difference between pages)
(Undo revision 9202 by Duncan.britton (talk))
 
(Introduction)
 
Line 1: Line 1:
{{Warning|This article is a work-in-progress referring to a future Portage version. It does not apply to the current Funtoo Portage version. Please do not update your configuration yet.}}
+
== Introduction ==
  
Starting with Portage-2.3.8, a switch to a new repository configuration framework is complete and users may want to update their configuration files. This document aims to describe the goals for the new framework and how to use it.
+
ZFS seams to be one of the future filesystems. At the time of writing (31.08.2015) booting on ZFS is still a headache. Some linux distributions boot on ext2 partition and run zfs as rootfs like [[ZFS_Install_Guide]] and some are able to boot from zfs, some do support zfs only as a filesystem.
  
== Multiple repository layout ==
+
Funtoo linux supports ZFS as filesystem and as rootfs, however it does not support it as boot/grub filesystem. For easier search by search engines, it is referred to in this document as whole disk ZFS.
One of the most important changes is the switch from the old ''overlay'' layout to a new cleaner ''repository'' system. The new layout is more flexible and more predictable. For example, repositories can now use resources (eclasses, for example) provided by other repositories.
+
  
The old layout was based on the concept of one ''main tree'' and optionally a number of overlays. The main tree provided base system ebuilds, eclasses, profiles, while overlays mostly were able to provide their own ebuilds. The ebuild provided by overlays overrode the ebuilds in main tree to the extend of making it impossible to install the main tree version. Overlays could also provide eclasses for their own ebuilds and package.* entries that applied to all overlays and to the main tree. The Package Manager is responsible for updating the main tree, while overlays are managed externally.
+
Funtoo linux uses the grub bootloader. So it has been not really understandable, why whole disk ZFS could not be supported by funtoo, as grub is able to do so. Or better, some linux distribution, using the grub bootloader run "whole disk ZFS". What they can, ... yes, we can! ... as well.
  
The new layout is based on the concept of one or more configurable repositories. Each repository can either be stand-alone or depend upon other repositories. The distribution provides a repository called ''funtoo'' (a drop-in replacement for Gentoo's ''gentoo'' repository). Users can install more repositories at they will, the repositories providing their own ebuilds, eclasses and profiles as necessary and/or using them from other repositories. Users can explicitly choose the repository they want to install packages from. The Package Manager can update all repositories.
+
This guide is based on many trial and errors - caused by lack of knowledge - the actual [[ZFS_Install_Guide]] and a guide for whole disk ZFS for ubuntu. Ubuntu runs als grub bootloader, so some ideas are adapted from there.
 +
== Disclaimers ==
  
== Portage configuration ==
+
{{fancywarning|This guide is a working pretty well on one computer - that is mine! If it does not run on yours, or if it breaks on yours, than you should try to sort the issue out and report it to this page respectively.  
=== New repository layout ===
+
The repository configuration should be stored in <code>/etc/portage/repos.conf</code>. It can be either a single file or a directory containing one or more ''.conf'' files.
+
  
The default configuration is installed as <code>/usr/share/portage/config/repos.conf</code>. This file is internal configuration file installed with portage ebuild and should '''not''' be modified. Instead, the configuration in <code>/etc/portage/repos.conf</code> can override the defaults specified there.
+
So, you may expect that it MAY work! ... and you should be aware that it MAY break your installation. This guide is not developed enough to ENSURE a stable production environment.
  
The configuration uses format similar to Windows .ini files. Each section heading (repository name in square brackets) signifies a single repository, followed by one or more key-value option pairs. For example, the following file copies default configuration for Funtoo repository:
+
... however, I use it for that! ;-) - crazzy as I am.
 
+
{{file|name=/etc/portage/repos.conf/funtoo.conf|desc=Example configuration override for Funtoo repository to move it to non-standard location|body=
+
[funtoo]
+
# moved to non-standard location!
+
location = /var/db/repos/funtoo
+
sync-type = git
+
sync-uri = git://github.com/funtoo/ports-2015.git
+
auto-sync = yes
+
 
}}
 
}}
  
The most useful repository configuration options are listed below:
+
{{fancyimportant|ZFS will run properly only on 64Bit machines. If you plan to run ZFS with 32Bit, you may also try russian roulette with six bullets. The outcome is clear, and sure not that what you want!}}
;location: ''Obligatory.'' Specifies the directory where repository is/will be stored. If Portage knows how to sync the repository and the location does not exist, it will be created on next ''emerge --sync''. Otherwise, the directory must exist.
+
;priority: Specifies the priority used for ordering ebuilds from different repositories. If two repositories provide an ebuild with matching versions, the repository with higher priority will be used.
+
;auto-sync: Specifies whether ''emerge --sync'' should update the repository. Defaults to ''yes'' if ''sync-type'' is specified, ''no'' otherwise.
+
;sync-depth: Specifies ''--depth'' for git clone. Used only on initial sync. Defaults to 1. Can be set to 0 to force full clone (not pass ''--depth'' at all).
+
;sync-type: Specifies syncing/update method. Can be one of: ''cvs'', ''git'', ''rsync'', ''svn''.
+
;sync-umask: Specifies the umask used when updating/syncing the repository.
+
;sync-uri: Specifies remote URI from which the repository will be cloned/synced. Can use any syntax valid for a particular syncing method.
+
;sync-user: Specifies the user[:group] used to update/sync the repository. If ''FEATURES=usersync'' is used, defaults to the credentials of directory owner.
+
  
Additionally a <code>[DEFAULT]</code> section may be specified. Options in this section are used as defaults for all repositories.
+
== Downloading the ISO (With ZFS) ==
  
=== Migrating existing configurations ===
+
This is a copy by the date of 31.08.2015 from the [[ZFS_Install_Guide]].
The new configuration format provides replacement for existing configuration done through <code>/etc/portage/make.conf</code> and environment variables. While the variables are still supported for backwards compatibility, users are recommended to move to the new configuration scheme. Funtoo portage ebuild is planned to  make the migration unattended (repos.conf installed automatically to ease the config steps) with the following file:
+
+
{{file|name=/etc/portage/repos.conf/funtoo.conf|body=
+
[funtoo]
+
location = /usr/portage
+
sync-type = git
+
sync-uri = git://github.com/funtoo/ports-2015.git
+
auto-sync = yes
+
}}
+
The following replacements are provided for existing variables:
+
;PORTDIR: Used to specify main tree location. Replaced by ''location'' key in the section corresponding to the default repository (<code>[funtoo]</code> by default).
+
;PORTDIR_OVERLAY: Used to specify locations of overlays. Each of the paths needs to be replaced with a separate repository section, with the path placed in ''location'' key. Additionally, ''priority'' may be used to force specific ordering of ebuild overrides.
+
;SYNC: Used to specify URI for syncing the main repository, also implied a protocol for doing that. Replaced by the ''sync-uri'' and ''sync-type'' keys in the default repository section.
+
;SYNC_UMASK: Used to specify umask for syncing repositories. Replaced by ''sync-umask'' key in repository configuration. Can be specified in <code>[DEFAULT]</code> section to apply to all repositories.
+
;SYNC_USER: Used to specify user credentials for syncing repositories. Replaced by ''sync-user'' key in repository configuration. Can be specified in <code>[DEFAULT]</code> section to apply to all repositories.
+
  
{{file|name=/etc/portage/make.conf|desc=Example old make.conf file|body=
+
In order for us to install Funtoo on ZFS, you will need an environment that already provides the ZFS tools. Therefore we will download a customized version of System Rescue CD with ZFS included.  
# user changed PORTDIR location
+
PORTDIR="/var/db/repos/funtoo"
+
PORTDIR_OVERLAY="/var/db/repos/foo /var/db/repos/bar"
+
  
SYNC="git://github.com/funtoo/ports-2015.git"
+
<pre>
SYNC_USER="oleg"
+
Name: sysresccd-4.2.0_zfs_0.6.2.iso  (545 MB)
SYNC_UMASK="022"
+
Release Date: 2014-02-25
}}
+
md5sum 01f4e6929247d54db77ab7be4d156d85
 +
</pre>
  
{{file|name=/etc/portage/repos.conf|desc=Replacement repos.conf file|body=
 
[DEFAULT]
 
sync-user = oleg
 
sync-umask = 022
 
  
[funtoo]
+
'''[http://ftp.osuosl.org/pub/funtoo/distfiles/sysresccd/ Download System Rescue CD with ZFS]'''<br />
location = /var/db/repos/funtoo
+
sync-type = git
+
sync-uri = git://github.com/funtoo/ports-2015.git
+
  
[foo]
+
== Creating a bootable USB from ISO (From a Linux Environment) ==
location = /var/db/repos/foo
+
priority = 1
+
  
[bar]
+
This is a copy by the date of 31.08.2015 from the [[ZFS_Install_Guide]].
location = /var/db/repos/bar
+
priority = 2
+
}}
+
  
The <code>repos.conf</code> configuration can be further extended with ''sync-type'' and ''sync-uri'' for overlays to get ''emerge --sync'' updating them automatically.
+
After you download the iso, you can do the following steps to create a bootable USB:
  
let's see a real example of tree and overlays added.
+
<console>
{{file|name=/etc/portage/repos.conf|desc=Replacement repos.conf file|body=
+
Make a temporary directory
 +
# ##i##mkdir /tmp/loop
  
[gentoo]
+
Mount the iso
location = /usr/portage
+
# ##i##mount -o ro,loop /root/sysresccd-4.2.0_zfs_0.6.2.iso /tmp/loop
sync-type = git
+
 
sync-uri = git://github.com/funtoo/ports-2012.git
+
Run the usb installer
+
# ##i##/tmp/loop/usb_inst.sh
[funtoo-overlay]
+
</console>
location = /root/git/funtoo-overlay
+
 
+
That should be all you need to do to get your flash drive working.
[funtoo-gnome]
+
 
location = /root/git/funtoo-gnome-overlay
+
== Booting the ISO ==
}}
+
While booting the above mentioned iso image - you should use exactly that one, unless you are writing another guide - you shoud use the option '''E) Alternative 64bit kernel (altker64) with more choice...'''. 64Bit, as mentioned above.
funtoo-overlay and funtoo-gnome-overlay are an overlays added on top of regular portage tree.
+
 
[[Category:Portage]]
+
On the next page you should select '''7. SystemRecsueCd with the default graphic environment'''. The system will come up with a small graphical environment and some tools. One of them is a graphical version of gparted, which will be used in this guide.
 +
 
 +
== Cleaning the disks ==
 +
 
 +
This guide was developed using GPT partition layout. Any other layout type is not included here.
 +
 
 +
All below this note is a copy by the date of 31.08.2015 from the respective section of the [[ZFS_Install_Guide]].
 +
 
 +
First lets make sure that the disk is completely wiped from any previous disk labels and partitions.
 +
We will also assume that <tt>/dev/sda</tt> is the target drive.<br />
 +
 
 +
<console>
 +
# ##i##sgdisk -Z /dev/sda
 +
</console>
 +
 
 +
{{fancywarning|This is a destructive operation and the program will not ask you for confirmation! Make sure you really don't want anything on this disk.}}

Revision as of 15:07, August 31, 2015

Introduction

ZFS seams to be one of the future filesystems. At the time of writing (31.08.2015) booting on ZFS is still a headache. Some linux distributions boot on ext2 partition and run zfs as rootfs like ZFS_Install_Guide and some are able to boot from zfs, some do support zfs only as a filesystem.

Funtoo linux supports ZFS as filesystem and as rootfs, however it does not support it as boot/grub filesystem. For easier search by search engines, it is referred to in this document as whole disk ZFS.

Funtoo linux uses the grub bootloader. So it has been not really understandable, why whole disk ZFS could not be supported by funtoo, as grub is able to do so. Or better, some linux distribution, using the grub bootloader run "whole disk ZFS". What they can, ... yes, we can! ... as well.

This guide is based on many trial and errors - caused by lack of knowledge - the actual ZFS_Install_Guide and a guide for whole disk ZFS for ubuntu. Ubuntu runs als grub bootloader, so some ideas are adapted from there.

Disclaimers

Warning

This guide is a working pretty well on one computer - that is mine! If it does not run on yours, or if it breaks on yours, than you should try to sort the issue out and report it to this page respectively.

So, you may expect that it MAY work! ... and you should be aware that it MAY break your installation. This guide is not developed enough to ENSURE a stable production environment.

... however, I use it for that! ;-) - crazzy as I am.

Important

ZFS will run properly only on 64Bit machines. If you plan to run ZFS with 32Bit, you may also try russian roulette with six bullets. The outcome is clear, and sure not that what you want!

Downloading the ISO (With ZFS)

This is a copy by the date of 31.08.2015 from the ZFS_Install_Guide.

In order for us to install Funtoo on ZFS, you will need an environment that already provides the ZFS tools. Therefore we will download a customized version of System Rescue CD with ZFS included.

Name: sysresccd-4.2.0_zfs_0.6.2.iso  (545 MB)
Release Date: 2014-02-25
md5sum 01f4e6929247d54db77ab7be4d156d85


Download System Rescue CD with ZFS

Creating a bootable USB from ISO (From a Linux Environment)

This is a copy by the date of 31.08.2015 from the ZFS_Install_Guide.

After you download the iso, you can do the following steps to create a bootable USB:

Make a temporary directory
# mkdir /tmp/loop

Mount the iso
# mount -o ro,loop /root/sysresccd-4.2.0_zfs_0.6.2.iso /tmp/loop

Run the usb installer
# /tmp/loop/usb_inst.sh

That should be all you need to do to get your flash drive working.

Booting the ISO

While booting the above mentioned iso image - you should use exactly that one, unless you are writing another guide - you shoud use the option E) Alternative 64bit kernel (altker64) with more choice.... 64Bit, as mentioned above.

On the next page you should select 7. SystemRecsueCd with the default graphic environment. The system will come up with a small graphical environment and some tools. One of them is a graphical version of gparted, which will be used in this guide.

Cleaning the disks

This guide was developed using GPT partition layout. Any other layout type is not included here.

All below this note is a copy by the date of 31.08.2015 from the respective section of the ZFS_Install_Guide.

First lets make sure that the disk is completely wiped from any previous disk labels and partitions. We will also assume that /dev/sda is the target drive.

# sgdisk -Z /dev/sda
Warning

This is a destructive operation and the program will not ask you for confirmation! Make sure you really don't want anything on this disk.