https://www.funtoo.org/api.php?action=feedcontributions&user=Oleg&feedformat=atomFuntoo - User contributions [en]2024-03-19T12:39:16ZUser contributionsMediaWiki 1.36.2https://www.funtoo.org/index.php?title=Funtoo:Metro/Evolved_Bootstrap&diff=43315Funtoo:Metro/Evolved Bootstrap2023-11-02T06:04:23Z<p>Oleg: this saves space, as there is full meta-repo tree within ffs stage1root</p>
<hr />
<div>This page documents how to use Funtoo from Scratch, aka [[Funtoo:Evolved Bootstrap|Evolved Boostrap]] to create a stage1 that is then fed to metro to build a stage3. This is an advanced functionality that ties together our Funtoo from Scratch effort under Evolved Bootstrap with our official Metro stage builder, allowing a Funtoo stage3 to be built that is not based on a pre-existing Funtoo system.<br />
<br />
== Evolved Bootstrap ==<br />
<br />
The [[Funtoo:Evolved Bootstrap|Evolved Bootstrap]] project allows for the creation of a stage1 Funtoo tarball completely from a non-Funtoo/non-Gentoo environment, fully cross-compiled and from scratch. Once the stage1 is created, it can be fed to Metro which will turn it into a stage3. But first, we need the bootstrapped stage1.<br />
<br />
In order to create a stage1 tarball using Evolved Bootstrap, you will want to follow the instructions located at https://code.funtoo.org/bitbucket/projects/CORE/repos/ffs/browse/README.rst. Use the LXD setup<br />
instructions to successfully complete a GNU build, as follows:<br />
<br />
{{console|body=<br />
$ ##i##cd ~/ffs<br />
$ ##i##ci/lxd-baremetal/bin/ffs gnu x86-64bit<br />
}}<br />
<br />
Once the build completes successfully, a stage1 will have been built, but is not (yet) automatically captured into a tarball. To do this, you will need to enter the container to grab it:<br />
<br />
{{console|body=<br />
$ ##i##lxc exec ffs-drobbins-x86-64bit-gnu-test -- /bin/bash --login<br />
%container% ##i##cd /root/ffs-repo/tmp/stage1root<br />
%container% ##i##tar cvf /var/tmp/stage1.tar --xattrs --acls . --exclude="var/git"<br />
%container% ##i##exit<br />
$ ##i##lxc file pull ffs-drobbins-x86-64bit-gnu-test/var/tmp/stage1.tar .<br />
$ ##i##scp -C /var/tmp/stage1.tar root@metro-host:/var/tmp/<br />
}}<br />
<br />
You now have a {{c|stage1.tar}} next-release tarball outside of the container, which can then be copied to a system set up with [[Funtoo:Metro|Metro]], and used to seed a Metro build, which we will cover next.<br />
<br />
== Seeding Metro ==<br />
<br />
You will need a system set up to run Metro. The typical way you want to use this is to follow the [[Funtoo:Metro/Initial Setup|Initial Setup]] instructions, and initialize Metro to build a {{c|generic_64}} next-release build for {{c|x86-64bit}}. Assuming you have this set up in {{c|/home/mirror/funtoo/next/x86-64bit/generic_64}}, which is the default location, here is how you will set up Metro to use your stage1. Replace {{c|2022-10-10}} with the current date on your Linux system:<br />
<br />
{{console|body=<br />
# ##i##cd /home/mirror/funtoo/next/x86-64bit/generic_64<br />
# ##i##mkdir 2022-10-10 <br />
# ##i##mv /var/tmp/stage1.tar stage1-generic_64-next-2022-10-10.tar<br />
}}<br />
<br />
For this to work, the name of the stage1 must match the specific expected name above, which is in the format {{c|stage1-subarch-release-date.tar}}. That's it! What we have done is set up Metro so that it thinks it has already built a valid stage1, so if we run the build script, this artifact will not be rebuilt -- and it will proceed to use the stage1 to build a stage2, and use the stage2 to build a stage3. Perform this build by running the following command:<br />
<br />
{{console|body=<br />
# ##i##cd /root/metro/scripts<br />
# ##i##./ezbuild.sh next x86-64bit generic_64 full<br />
}}<br />
<br />
Metro will proceed to build the stage2 and stage3, and you can follow the logs at {{c|/home/mirror/funtoo/next/x86-64bit/generic_64/2022-10-10/log/stage2.log}}.</div>Oleghttps://www.funtoo.org/index.php?title=Package:Mysql-communtiy&diff=26747Package:Mysql-communtiy2019-01-30T05:37:58Z<p>Oleg: Oleg moved page Package:Mysql-communtiy to Package:Mysql-community</p>
<hr />
<div>#REDIRECT [[Package:Mysql-community]]</div>Oleghttps://www.funtoo.org/index.php?title=Package:Mysql-community&diff=26746Package:Mysql-community2019-01-30T05:37:58Z<p>Oleg: Oleg moved page Package:Mysql-communtiy to Package:Mysql-community</p>
<hr />
<div>{{Ebuild<br />
|Summary=MySQL is a fast, multi-threaded, multi-user SQL database server.<br />
|CatPkg=dev-db/mysql-community<br />
|Maintainer=<br />
|Homepage=http://www.mysql.com/<br />
}}<br />
{{Note|mysql-community is what known as MySQL, version 8. The name mysql-community is just a distinct ebuild name to point a significant changes compared to version 5.x and that mysql-community is an open source version, not enterprise commercial software.}}<br />
MySQL is the M in the lamp/lemp/llmp stack. It is a popular, & common database that accepts SQL statements. {{package|dev-db/mariadb}} & {{package|dev-db/percona-server}} are drop in replacements for MySQL.<br />
<br />
=== Installation ===<br />
<br />
{{console|body=###i## emerge mysql-community}}<br />
<br />
==== First Run ====<br />
MySQL requires configuration upon instillation.<br />
<br />
To deploy MySQL:<br />
{{console|body=###i## mysqld --initialize-insecure --default_authentication_plugin=mysql_native_password --datadir=/var/lib/mysql}}<br />
This will create MySQL base directory, by default {{c|/var/lib/mysql}} as well as an empty password for your initial database setup. This is preliminary step before setting up your password, see below<br />
<br />
==== Init ====<br />
To start mysql:<br />
{{console|body=###i## rc-service mysql start}}<br />
<br />
To start upon boot:<br />
{{console|body=###i## rc-update add mysql default}}<br />
<br />
==== Secure installation ====<br />
Now, run {{c|mysql_secure_installation}}. You will be promted for setting your password, as well as other required steps.<br />
{{console|body=###i## mysql_secure_installation}}<br />
<br />
=== Usage ===<br />
To use mysql you will need to run MySQL as the funtoo root user, logging in as the mysql database root user. The system root user is not the same as the database root user, and passwords for both should be different. <br />
{{console|body=###i## mysql -u root -p<br />
Enter password: <br />
mysql><br />
}}<br />
mysql is now blinking at you ready for SQL statements.<br />
<br />
Root use is discouraged, for every database you require, create a user, and a database specific for the application, then allow the user access with as stripped down as possible permissions. We will use oleg examples of creating a oleg user & database.<br />
<br />
{{console|body=###i## mysql -u root -p<br />
Enter password: <br />
mysql> ##i##CREATE USER 'oleg'@'localhost' IDENTIFIED BY 'password';<br />
mysql> ##i##GRANT ALL PRIVILEGES ON *.* TO 'oleg'@'localhost'<br />
->##i##WITH GRANT OPTION;<br />
mysql> ##i##\q<br />
}}<br />
Let's login with "oleg" account we created above. And let's try creating a database.<br />
{{console|body=###i## mysql -u oleg -p<br />
mysql> ##i##CREATE DATABASE betelgeuse;<br />
}}<br />
<br />
==== Logging ====<br />
<br />
By default MySQL logs every action, including leaving plain text passwords in its history file.<br />
<br />
To remove the history file:<br />
{{console|body=###i## rm /root/.mysql_history}}<br />
<br />
To automatically remove future history:<br />
{{console|body=###i## ln -s /dev/null /root/.mysql_history}}<br />
For further references, please, follow this great guide: https://downloads.mysql.com/docs/refman-8.0-en.pdf</div>Oleghttps://www.funtoo.org/index.php?title=Package:Mysql-community&diff=26744Package:Mysql-community2019-01-29T14:48:00Z<p>Oleg: /* Usage */</p>
<hr />
<div>{{Ebuild<br />
|Summary=MySQL is a fast, multi-threaded, multi-user SQL database server.<br />
|CatPkg=dev-db/mysql-community<br />
|Maintainer=<br />
|Homepage=http://www.mysql.com/<br />
}}<br />
{{Note|mysql-community is what known as MySQL, version 8. The name mysql-community is just a distinct ebuild name to point a significant changes compared to version 5.x and that mysql-community is an open source version, not enterprise commercial software.}}<br />
MySQL is the M in the lamp/lemp/llmp stack. It is a popular, & common database that accepts SQL statements. {{package|dev-db/mariadb}} & {{package|dev-db/percona-server}} are drop in replacements for MySQL.<br />
<br />
=== Installation ===<br />
<br />
{{console|body=###i## emerge mysql-community}}<br />
<br />
==== First Run ====<br />
MySQL requires configuration upon instillation.<br />
<br />
To deploy MySQL:<br />
{{console|body=###i## mysqld --initialize-insecure --default_authentication_plugin=mysql_native_password --datadir=/var/lib/mysql}}<br />
This will create MySQL base directory, by default {{c|/var/lib/mysql}} as well as an empty password for your initial database setup. This is preliminary step before setting up your password, see below<br />
<br />
==== Init ====<br />
To start mysql:<br />
{{console|body=###i## rc-service mysql start}}<br />
<br />
To start upon boot:<br />
{{console|body=###i## rc-update add mysql default}}<br />
<br />
==== Secure installation ====<br />
Now, run {{c|mysql_secure_installation}}. You will be promted for setting your password, as well as other required steps.<br />
{{console|body=###i## mysql_secure_installation}}<br />
<br />
=== Usage ===<br />
To use mysql you will need to run MySQL as the funtoo root user, logging in as the mysql database root user. The system root user is not the same as the database root user, and passwords for both should be different. <br />
{{console|body=###i## mysql -u root -p<br />
Enter password: <br />
mysql><br />
}}<br />
mysql is now blinking at you ready for SQL statements.<br />
<br />
Root use is discouraged, for every database you require, create a user, and a database specific for the application, then allow the user access with as stripped down as possible permissions. We will use oleg examples of creating a oleg user & database.<br />
<br />
{{console|body=###i## mysql -u root -p<br />
Enter password: <br />
mysql> ##i##CREATE USER 'oleg'@'localhost' IDENTIFIED BY 'password';<br />
mysql> ##i##GRANT ALL PRIVILEGES ON *.* TO 'oleg'@'localhost'<br />
->##i##WITH GRANT OPTION;<br />
mysql> ##i##\q<br />
}}<br />
Let's login with "oleg" account we created above. And let's try creating a database.<br />
{{console|body=###i## mysql -u oleg -p<br />
mysql> ##i##CREATE DATABASE betelgeuse;<br />
}}<br />
<br />
==== Logging ====<br />
<br />
By default MySQL logs every action, including leaving plain text passwords in its history file.<br />
<br />
To remove the history file:<br />
{{console|body=###i## rm /root/.mysql_history}}<br />
<br />
To automatically remove future history:<br />
{{console|body=###i## ln -s /dev/null /root/.mysql_history}}<br />
For further references, please, follow this great guide: https://downloads.mysql.com/docs/refman-8.0-en.pdf</div>Oleghttps://www.funtoo.org/index.php?title=Package:Mysql-community&diff=26743Package:Mysql-community2019-01-29T14:25:41Z<p>Oleg: /* Usage */</p>
<hr />
<div>{{Ebuild<br />
|Summary=MySQL is a fast, multi-threaded, multi-user SQL database server.<br />
|CatPkg=dev-db/mysql-community<br />
|Maintainer=<br />
|Homepage=http://www.mysql.com/<br />
}}<br />
{{Note|mysql-community is what known as MySQL, version 8. The name mysql-community is just a distinct ebuild name to point a significant changes compared to version 5.x and that mysql-community is an open source version, not enterprise commercial software.}}<br />
MySQL is the M in the lamp/lemp/llmp stack. It is a popular, & common database that accepts SQL statements. {{package|dev-db/mariadb}} & {{package|dev-db/percona-server}} are drop in replacements for MySQL.<br />
<br />
=== Installation ===<br />
<br />
{{console|body=###i## emerge mysql-community}}<br />
<br />
==== First Run ====<br />
MySQL requires configuration upon instillation.<br />
<br />
To deploy MySQL:<br />
{{console|body=###i## mysqld --initialize-insecure --default_authentication_plugin=mysql_native_password --datadir=/var/lib/mysql}}<br />
This will create MySQL base directory, by default {{c|/var/lib/mysql}} as well as an empty password for your initial database setup. This is preliminary step before setting up your password, see below<br />
<br />
==== Init ====<br />
To start mysql:<br />
{{console|body=###i## rc-service mysql start}}<br />
<br />
To start upon boot:<br />
{{console|body=###i## rc-update add mysql default}}<br />
<br />
==== Secure installation ====<br />
Now, run {{c|mysql_secure_installation}}. You will be promted for setting your password, as well as other required steps.<br />
{{console|body=###i## mysql_secure_installation}}<br />
<br />
=== Usage ===<br />
To use mysql you will need to run MySQL as the funtoo root user, logging in as the mysql database root user. The system root user is not the same as the database root user, and passwords for both should be different. <br />
{{console|body=###i## mysql -u root -p<br />
Enter password: <br />
mysql><br />
}}<br />
mysql is now blinking at you ready for SQL statements.<br />
<br />
Root use is discouraged, for every database you require, create a user, and a database specific for the application, then allow the user access with as stripped down as possible permissions. We will use oleg examples of creating a oleg user & database.<br />
<br />
{{console|body=###i## mysql -u root -p<br />
Enter password: <br />
mysql> ##i##CREATE USER 'oleg'@'localhost' IDENTIFIED BY 'password';<br />
mysql> ##i##GRANT ALL PRIVILEGES ON *.* TO 'oleg'@'localhost'<br />
->##i##WITH GRANT OPTION;<br />
mysql> ##i##<br />
}}<br />
<br />
==== Logging ====<br />
<br />
By default MySQL logs every action, including leaving plain text passwords in its history file.<br />
<br />
To remove the history file:<br />
{{console|body=###i## rm /root/.mysql_history}}<br />
<br />
To automatically remove future history:<br />
{{console|body=###i## ln -s /dev/null /root/.mysql_history}}</div>Oleghttps://www.funtoo.org/index.php?title=Package:Mysql-community&diff=26694Package:Mysql-community2019-01-27T16:01:42Z<p>Oleg: /* First Run */</p>
<hr />
<div>{{Ebuild<br />
|Summary=MySQL is a fast, multi-threaded, multi-user SQL database server.<br />
|CatPkg=dev-db/mysql-community<br />
|Maintainer=<br />
|Homepage=http://www.mysql.com/<br />
}}<br />
{{Note|mysql-community is what known as MySQL, version 8. The name mysql-community is just a distinct ebuild name to point a significant changes compared to version 5.x and that mysql-community is an open source version, not enterprise commercial software.}}<br />
MySQL is the M in the lamp/lemp/llmp stack. It is a popular, & common database that accepts SQL statements. {{package|dev-db/mariadb}} & {{package|dev-db/percona-server}} are drop in replacements for MySQL.<br />
<br />
=== Installation ===<br />
<br />
{{console|body=###i## emerge mysql-community}}<br />
<br />
==== First Run ====<br />
MySQL requires configuration upon instillation.<br />
<br />
To deploy MySQL:<br />
{{console|body=###i## mysqld --initialize-insecure --default_authentication_plugin=mysql_native_password --datadir=/var/lib/mysql}}<br />
This will create MySQL base directory, by default {{c|/var/lib/mysql}} as well as an empty password for your initial database setup. This is preliminary step before setting up your password, see below<br />
<br />
==== Init ====<br />
To start mysql:<br />
{{console|body=###i## rc-service mysql start}}<br />
<br />
To start upon boot:<br />
{{console|body=###i## rc-update add mysql default}}<br />
<br />
==== Secure installation ====<br />
Now, run {{c|mysql_secure_installation}}. You will be promted for setting your password, as well as other required steps.<br />
{{console|body=###i## mysql_secure_installation}}<br />
<br />
=== Usage ===<br />
To use mysql you will need to run MySQL as the funtoo root user, logging in as the mysql database root user. The system root user is not the same as the database root user, and passwords for both should be different. <br />
{{console|body=###i## mysql -u root -p<br />
Enter password: <br />
mysql><br />
}}<br />
mysql is now blinking at you ready for SQL statements.<br />
<br />
Root use is discouraged, for every database you require, create a user, and a database specific for the application, then allow the user access with as stripped down as possible permissions. We will use phpBB examples of creating a phpBB user & database.<br />
<br />
{{console|body=###i## mysql -u root -p<br />
Enter password: <br />
mysql> ##i##CREATE USER 'phpbb'@'localhost' IDENTIFIED BY 'changeme';<br />
mysql> ##i##CREATE DATABASE IF NOT EXISTS `phpbb` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`;<br />
mysql> ##i##GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER ON `phpbb`.* TO 'phpbb'@'localhost' IDENTIFIED BY 'changeme';<br />
mysql> ##i##\q<br />
}}<br />
<br />
==== Logging ====<br />
<br />
By default MySQL logs every action, including leaving plain text passwords in its history file.<br />
<br />
To remove the history file:<br />
{{console|body=###i## rm /root/.mysql_history}}<br />
<br />
To automatically remove future history:<br />
{{console|body=###i## ln -s /dev/null /root/.mysql_history}}</div>Oleghttps://www.funtoo.org/index.php?title=Package:Mysql-community&diff=26693Package:Mysql-community2019-01-27T13:29:01Z<p>Oleg: /* Secure instalation */</p>
<hr />
<div>{{Ebuild<br />
|Summary=MySQL is a fast, multi-threaded, multi-user SQL database server.<br />
|CatPkg=dev-db/mysql-community<br />
|Maintainer=<br />
|Homepage=http://www.mysql.com/<br />
}}<br />
{{Note|mysql-community is what known as MySQL, version 8. The name mysql-community is just a distinct ebuild name to point a significant changes compared to version 5.x and that mysql-community is an open source version, not enterprise commercial software.}}<br />
MySQL is the M in the lamp/lemp/llmp stack. It is a popular, & common database that accepts SQL statements. {{package|dev-db/mariadb}} & {{package|dev-db/percona-server}} are drop in replacements for MySQL.<br />
<br />
=== Installation ===<br />
<br />
{{console|body=###i## emerge mysql-community}}<br />
<br />
==== First Run ====<br />
MySQL requires configuration upon instillation.<br />
<br />
To deploy MySQL:<br />
{{console|body=###i## mysqld --initialize-insecure --default_authentication_plugin=mysql_native_password}}<br />
This will create MySQL base directory, by default {{c|/var/lib/mysql}} as well as an empty password for your initial database setup. This is preliminary step before setting up your password, see below<br />
<br />
==== Init ====<br />
To start mysql:<br />
{{console|body=###i## rc-service mysql start}}<br />
<br />
To start upon boot:<br />
{{console|body=###i## rc-update add mysql default}}<br />
<br />
==== Secure installation ====<br />
Now, run {{c|mysql_secure_installation}}. You will be promted for setting your password, as well as other required steps.<br />
{{console|body=###i## mysql_secure_installation}}<br />
<br />
=== Usage ===<br />
To use mysql you will need to run MySQL as the funtoo root user, logging in as the mysql database root user. The system root user is not the same as the database root user, and passwords for both should be different. <br />
{{console|body=###i## mysql -u root -p<br />
Enter password: <br />
mysql><br />
}}<br />
mysql is now blinking at you ready for SQL statements.<br />
<br />
Root use is discouraged, for every database you require, create a user, and a database specific for the application, then allow the user access with as stripped down as possible permissions. We will use phpBB examples of creating a phpBB user & database.<br />
<br />
{{console|body=###i## mysql -u root -p<br />
Enter password: <br />
mysql> ##i##CREATE USER 'phpbb'@'localhost' IDENTIFIED BY 'changeme';<br />
mysql> ##i##CREATE DATABASE IF NOT EXISTS `phpbb` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`;<br />
mysql> ##i##GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER ON `phpbb`.* TO 'phpbb'@'localhost' IDENTIFIED BY 'changeme';<br />
mysql> ##i##\q<br />
}}<br />
<br />
==== Logging ====<br />
<br />
By default MySQL logs every action, including leaving plain text passwords in its history file.<br />
<br />
To remove the history file:<br />
{{console|body=###i## rm /root/.mysql_history}}<br />
<br />
To automatically remove future history:<br />
{{console|body=###i## ln -s /dev/null /root/.mysql_history}}</div>Oleghttps://www.funtoo.org/index.php?title=Package:Mysql-community&diff=26692Package:Mysql-community2019-01-27T13:25:07Z<p>Oleg: Created page with "{{Ebuild |Summary=MySQL is a fast, multi-threaded, multi-user SQL database server. |CatPkg=dev-db/mysql-community |Maintainer= |Homepage=http://www.mysql.com/ }} {{Note|mysql-..."</p>
<hr />
<div>{{Ebuild<br />
|Summary=MySQL is a fast, multi-threaded, multi-user SQL database server.<br />
|CatPkg=dev-db/mysql-community<br />
|Maintainer=<br />
|Homepage=http://www.mysql.com/<br />
}}<br />
{{Note|mysql-community is what known as MySQL, version 8. The name mysql-community is just a distinct ebuild name to point a significant changes compared to version 5.x and that mysql-community is an open source version, not enterprise commercial software.}}<br />
MySQL is the M in the lamp/lemp/llmp stack. It is a popular, & common database that accepts SQL statements. {{package|dev-db/mariadb}} & {{package|dev-db/percona-server}} are drop in replacements for MySQL.<br />
<br />
=== Installation ===<br />
<br />
{{console|body=###i## emerge mysql-community}}<br />
<br />
==== First Run ====<br />
MySQL requires configuration upon instillation.<br />
<br />
To deploy MySQL:<br />
{{console|body=###i## mysqld --initialize-insecure --default_authentication_plugin=mysql_native_password}}<br />
This will create MySQL base directory, by default {{c|/var/lib/mysql}} as well as an empty password for your initial database setup. This is preliminary step before setting up your password, see below<br />
<br />
==== Init ====<br />
To start mysql:<br />
{{console|body=###i## rc-service mysql start}}<br />
<br />
To start upon boot:<br />
{{console|body=###i## rc-update add mysql default}}<br />
<br />
==== Secure instalation ====<br />
Now, run {{c|mysql_secure_installation}}. You will be promted for setting your password, as well as other required steps.<br />
{{console|body=###i## mysql_secure_installation}}<br />
<br />
=== Usage ===<br />
To use mysql you will need to run MySQL as the funtoo root user, logging in as the mysql database root user. The system root user is not the same as the database root user, and passwords for both should be different. <br />
{{console|body=###i## mysql -u root -p<br />
Enter password: <br />
mysql><br />
}}<br />
mysql is now blinking at you ready for SQL statements.<br />
<br />
Root use is discouraged, for every database you require, create a user, and a database specific for the application, then allow the user access with as stripped down as possible permissions. We will use phpBB examples of creating a phpBB user & database.<br />
<br />
{{console|body=###i## mysql -u root -p<br />
Enter password: <br />
mysql> ##i##CREATE USER 'phpbb'@'localhost' IDENTIFIED BY 'changeme';<br />
mysql> ##i##CREATE DATABASE IF NOT EXISTS `phpbb` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`;<br />
mysql> ##i##GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER ON `phpbb`.* TO 'phpbb'@'localhost' IDENTIFIED BY 'changeme';<br />
mysql> ##i##\q<br />
}}<br />
<br />
==== Logging ====<br />
<br />
By default MySQL logs every action, including leaving plain text passwords in its history file.<br />
<br />
To remove the history file:<br />
{{console|body=###i## rm /root/.mysql_history}}<br />
<br />
To automatically remove future history:<br />
{{console|body=###i## ln -s /dev/null /root/.mysql_history}}</div>Oleghttps://www.funtoo.org/index.php?title=LXD&diff=26583LXD2019-01-16T07:07:30Z<p>Oleg: /* Running systemd container on a non-systemd host */</p>
<hr />
<div>== PART I - Introduction ==<br />
LXD is a container "hypervisor" it should provide user with a new and fresh experience using [[LXC]] technology.<br />
{{#layout:doc}}<br />
__TOC__<br />
<br />
LXD consists of three components:<br />
<br />
* A system-wide daemon (lxd)<br />
* A command line client (lxc)<br />
* An OpenStack Nova plugin (nova-compute-lxd)<br />
<br />
A REST API that is accesible both locally and if enabled, over the network is provided from the lxd daemon.<br />
<br />
The command line tool is designed to be a very simple, yet very powerful tool to manage all your containers. It can handle connections to multiple container hosts and easily give you an overview of all the containers on your network, let you create some more where you want them and even move them around while they're running.<br />
<br />
The OpenStack plugin then allows you to use your lxd hosts as compute nodes, running workloads on containers rather than virtual machines.<br />
<br />
The LXD project was founded and is currently led by Canonical Ltd and Ubuntu with contributions from a range of other companies and individual contributors.<br />
<br />
=== Features ===<br />
<br />
Some of the biggest features of LXD are:<br />
<br />
* Secure by design (unprivileged containers, resource restrictions and much more)<br />
* Scalable (from containers on your laptop to thousand of compute nodes)<br />
* Intuitive (simple, clear API and crisp command line experience)<br />
* Image based (no more distribution templates, only good, trusted images)<br />
* Live migration<br />
<br />
==== Unprivileged Containers ====<br />
<br />
LXD uses unprivileged containers by default. The difference between an unprivileged container and a privileged one is whether the root user in the container is the “real” root user (uid 0 at the kernel level).<br />
<br />
The way unprivileged containers are created is by taking a set of normal UIDs and GIDs from the host, usually at least 65536 of each (to be POSIX compliant) and mapping those into the container.<br />
<br />
The most common example and what most LXD users will end up with by default is a map of 65536 UIDs and GIDs, with a host base id of 100000. This means that root in the container (uid 0) will be mapped to the host uid 100000 and uid 65535 in the container will be mapped to uid 165535 on the host. UID/GID 65536 and higher in the container aren’t mapped and will return an error if you attempt to use them.<br />
<br />
From a security point of view, that means that anything which is not owned by the users and groups mapped into the container will be inaccessible. Any such resource will show up as being owned by uid/gid “-1” (rendered as 65534 or nobody/nogroup in userspace). It also means that should there be a way to escape the container, even root in the container would find itself with just as much privileges on the host as a nobody user.<br />
<br />
LXD does offer a number of options related to unprivileged configuration:<br />
<br />
* Increasing the size of the default uid/gid map<br />
* Setting up per-container maps<br />
* Punching holes into the map to expose host users and groups<br />
<br />
=== Relationship with LXC ===<br />
LXD isn't a rewrite of LXC, in fact it's building on top of LXC to provide a new, better user experience. Under the hood, LXD uses LXC through liblxc and its Go binding<br />
to create and manage the containers.<br />
<br />
It's basically an alternative to LXC's tools and distribution template system with the added features that come from being controllable over the network.<br />
<br />
=== Licensing ===<br />
LXD is free software and is developed under the Apache 2 license.<br />
<br />
<hr><br />
<hr><br />
Let's break this tutorial into smaller parts. Please click on the headings to go to the section's text.<br />
<br />
<hr><br />
<hr><br />
<br />
== [[LXD/LXD Installation|PART II - LXD Installation]] ==<br />
<br />
== [[LXD/LXD Setup|PART III - LXD Setup]] ==<br />
<br />
== Containers, snapshots and images ==<br />
'''Containers''' in LXD are made of:<br />
* A filesystem (rootfs)<br />
* A list of configuration options, including resource limits, environment, security options and more<br />
* A bunch of devices like disks, character/block unix devices and network interfaces<br />
* A set of profiles the container inherits configuration from (see below)<br />
* Some properties (container architecture, ephemeral or persistent and the name)<br />
* Some runtime state (when using CRIU for checkpoint/restore)<br />
<br />
Container '''snapshots''' as the name states snapshots of the container in time and cannot be modified in any way. It is worth noting that because snapshots can store the container runtime state, which gives us ability of “stateful” snapshots. That is, the ability to rollback the container including its cpu and memory state at the time of the snapshot.<br />
<br />
LXD is '''image''' based, all LXD containers come from an image. Images are typically clean Linux distribution images similar to what you would use for a virtual machine or cloud instance. It is possible to “publish” a container, making an image from it which can then be used by the local or remote LXD hosts.<br />
<br />
=== Our first image ===<br />
Let's get our hands even more dirty and create our first image. We will be using a generic 64 bit Funtoo Linux image. <br />
<br />
{{note|The Funtoo's default build host is building only westmere stage for now.}}<br />
<br />
Grab the image here:<br />
https://build.funtoo.org/funtoo-current/x86-64bit/intel64-westmere/lxd-latest.tar.xz<br />
<br />
Grab also the hash file:<br />
https://build.funtoo.org/funtoo-current/x86-64bit/intel64-westmere/lxd-latest.tar.xz.hash.txt<br />
<br />
{{tip|Check the hash of the downloaded file against the one from server. Proceed if they match. }}<br />
<br />
==== Import the image ====<br />
After we have successfully downloaded the archive we can now finally import it into LXD and start using it as our "seed" image for all our containers.<br />
{{console|body=<br />
###i## lxc image import lxd-latest.tar.xz --alias funtoo<br />
Image imported with fingerprint: 6c2ca3af0222d503656f5a1838885f1b9b6aed2c1994f1d7ef94e2efcb7233c4<br />
###i## lxc image ls<br />
<nowiki>+--------+--------------+--------+------------------------------------+--------+----------+-----------------------------+<br />
| ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCH | SIZE | UPLOAD DATE |<br />
+--------+--------------+--------+------------------------------------+--------+----------+-----------------------------+<br />
| funtoo | 6c2ca3af0222 | no | Funtoo Current Generic Pure 64-bit | x86_64 |227.99MB | Dec 13, 2017 at 11:01pm (UTC) |<br />
+--------+--------------+--------+------------------------------------+--------+----------+-----------------------------+<br />
</nowiki>}}<br />
And there we have our very first Funtoo Linux image imported inside LXD. You can reference the image through the alias or through the fingerprint. Aliases can be added also later.<br />
<br />
Let me show you some basic usage then.<br />
<br />
=== Creating your first container ===<br />
So now we can launch our first container. That is done using this command:<br />
<br />
{{console|body=<br />
###i## lxc launch funtoo fun-1<br />
Creating fun-1<br />
Starting fun-1<br />
###i## lxc ls<br />
<nowiki>+-------+---------+------+-----------------------------------------------+------------+-----------+<br />
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |<br />
+-------+---------+------+-----------------------------------------------+------------+-----------+<br />
| fun-1 | RUNNING | | fd42:156d:4593:a619:216:3eff:fef7:c1c2 (eth0) | PERSISTENT | 0 |<br />
+-------+---------+------+-----------------------------------------------+------------+-----------+<br />
</nowiki>}}<br />
<br />
{{tip|lxc launch is a shortcut for lxc init and lxc start, lxc init creates the container without starting it. }}<br />
<br />
==== Profiles intermezzo ====<br />
LXD has the ability to change quite a few container settings, including resource limitation, control of container startup and a variety of device pass-through options using what is called profiles. Multiple profiles can be applied to a single container, and the last profile overrides the other ones it the resources being configured is the same for multiple profiles. Let me show you how can this be used.<br />
<br />
This is the default profile that gets inherited by all containers.<br />
{{console|body=<br />
###i## lxc profile list<br />
<nowiki>+---------+---------+<br />
| NAME | USED BY |<br />
+---------+---------+<br />
| default | 1 |<br />
+---------+---------+<br />
</nowiki><br />
###i## lxc profile show default<br />
config: {}<br />
description: Default LXD profile<br />
devices:<br />
eth0:<br />
nictype: bridged<br />
parent: lxdbr0<br />
type: nic<br />
root:<br />
path: /<br />
pool: default<br />
type: disk<br />
name: default<br />
used_by:<br />
- /1.0/containers/fun-1<br />
}}<br />
<br />
Now let's edit this profile for our funtoo containers. It will include some useful stuff.<br />
<br />
{{console|body=<br />
###i## lxc profile set default raw.lxc "lxc.mount.entry = none dev/shm tmpfs rw,nosuid,nodev,create=dir"<br />
###i## lxc profile set default environment.LANG "en_US.UTF-8"<br />
###i## lxc profile set default environment.LC_ALL "en_US.UTF-8"<br />
###i## lxc profile set default environment.LC_COLLATE "POSIX"<br />
}}<br />
<br />
Profiles can store any configuration that a container can (key/value or devices) and any number of profiles can be applied to a container. Profiles are applied in the order they are specified so the last profile to specify a specific key wins. In any case, resource-specific configuration always overrides that coming from the profiles.<br />
<br />
The default profile is set for any new container created which doesn't specify a different profiles list.<br />
<br />
{{note|LXD supports simple instance types. Those are represented as a string which can be passed at container creation time. [https://github.com/lxc/lxd/blob/master/doc/containers.md#instance-types containers.md#instance-types]}}<br />
<br />
=== Using our first container ===<br />
After we have done all these customizations we can now start using our container.<br />
The next command will give us shell inside the container.<br />
<br />
{{console|body=<br />
###i## lxc exec fun-1 bash<br />
}}<br />
<br />
Now you should see a different prompt starting with<br />
<br />
{{console|body=<br />
fun-1 ~ #<br />
}}<br />
<br />
If we run top or ps for example we will see only the processes of the container.<br />
<br />
{{console|body=<br />
fun-1 ~ # ps aux<br />
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND<br />
root 1 0.0 0.0 4248 748 ? Ss+ 13:20 0:00 init [3]<br />
root 266 0.0 0.0 30488 472 ? Ss 13:20 0:00 /usr/sbin/sshd<br />
root 312 0.2 0.0 17996 3416 ? Ss 13:29 0:00 bash<br />
root 317 0.0 0.0 19200 2260 ? R+ 13:29 0:00 ps aux<br />
}}<br />
<br />
As you can see only the container's processes are shown. User running the processes is root here. What happens if we search for all sshd processes for example on the host box?<br />
<br />
{{console|body=<br />
###i## <nowiki>ps aux|grep ssh<br />
root 14505 0.0 0.0 30564 1508 ? Ss Sep07 0:00 /usr/sbin/sshd <br />
100000 25863 0.0 0.0 30488 472 ? Ss 15:20 0:00 /usr/sbin/sshd <br />
root 29487 0.0 0.0 8324 828 pts/2 S+ 15:30 0:00 grep --colour=auto sshd</nowiki><br />
###i##<br />
}}<br />
<br />
So as you can see, the sshd process is running under user with uid 100000 on the host machine and has a different PID.<br />
<br />
=== Basic actions with containers ===<br />
==== Listing containers ====<br />
{{console|body=<br />
###i## lxc ls<br />
<nowiki>+-------+---------+-----------------------+------------------------------------------------+------------+-----------+<br />
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |<br />
+-------+---------+-----------------------+------------------------------------------------+------------+-----------+<br />
| fun-1 | RUNNING | 10.214.101.187 (eth0) | fd42:156d:4593:a619:a5ad:edaf:7270:e6c4 (eth0) | PERSISTENT | 0 |<br />
| | | | fd42:156d:4593:a619:216:3eff:fef7:c1c2 (eth0) | | |<br />
+-------+---------+-----------------------+------------------------------------------------+------------+-----------+<br />
</nowiki>}}<br />
lxc ls also accepts arguments as filters. For example lxc ls web will list all containers that have web in their name.<br />
<br />
==== Container details ====<br />
{{console|body=<br />
###i## lxc info c1<br />
Name: c1<br />
Remote: unix://<br />
Architecture: x86_64<br />
Created: 2017/09/08 02:07 UTC<br />
Status: Running<br />
Type: persistent<br />
Profiles: default, prf-funtoo<br />
Pid: 6366<br />
Ips:<br />
eth0: inet 10.214.101.79 vethFG4HXG<br />
eth0: inet6 fd42:156d:4593:a619:8619:546e:43f:2089 vethFG4HXG<br />
eth0: inet6 fd42:156d:4593:a619:216:3eff:fe4a:3d4f vethFG4HXG<br />
eth0: inet6 fe80::216:3eff:fe4a:3d4f vethFG4HXG<br />
lo: inet 127.0.0.1<br />
lo: inet6 ::1<br />
Resources:<br />
Processes: 6<br />
CPU usage:<br />
CPU usage (in seconds): 25<br />
Memory usage:<br />
Memory (current): 69.01MB<br />
Memory (peak): 258.92MB<br />
Network usage:<br />
eth0:<br />
Bytes received: 83.65kB<br />
Bytes sent: 9.44kB<br />
Packets received: 188<br />
Packets sent: 93<br />
lo:<br />
Bytes received: 0B<br />
Bytes sent: 0B<br />
Packets received: 0<br />
Packets sent: 0<br />
}}<br />
<br />
==== Container configuration ====<br />
{{console|body=<br />
###i## lxc config edit c1<br />
### This is a yaml representation of the configuration.<br />
### Any line starting with a '# will be ignored.<br />
###<br />
### A sample configuration looks like:<br />
### name: container1<br />
### profiles:<br />
### - default<br />
### config:<br />
### volatile.eth0.hwaddr: 00:16:3e:e9:f8:7f<br />
### devices:<br />
### homedir:<br />
### path: /extra<br />
### source: /home/user<br />
### type: disk<br />
### ephemeral: false<br />
###<br />
### Note that the name is shown but cannot be changed<br />
<br />
architecture: x86_64<br />
config:<br />
image.architecture: x86_64<br />
image.description: Funtoo Current Generic Pure 64-bit<br />
image.name: funtoo-generic_64-pure64-funtoo-current-2016-12-10<br />
image.os: funtoo<br />
image.release: "1.0"<br />
image.variant: current<br />
volatile.base_image: e279c16d1a801b2bd1698df95e148e0a968846835f4769b24988f2eb3700100f<br />
volatile.eth0.hwaddr: 00:16:3e:4a:3d:4f<br />
volatile.eth0.name: eth0<br />
volatile.idmap.base: "0"<br />
volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'<br />
volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'<br />
volatile.last_state.power: RUNNING<br />
devices: {}<br />
ephemeral: false<br />
profiles:<br />
- default<br />
- prf-funtoo<br />
stateful: false<br />
description: ""<br />
}}<br />
<br />
One can also add environment variables.<br />
{{console|body=<br />
###i## lxc config set <container> environment.LANG en_US.UTF-8<br />
###i## lxc config set <container> environment.LC_COLLATE POSIX<br />
}}<br />
<br />
=== Managing files ===<br />
<br />
=== Snapshots ===<br />
<br />
=== Cloning, copying and moving containers ===<br />
<br />
=== Resource control ===<br />
LXD offers a variety of resource limits. Some of those are tied to the container itself, like memory quotas, CPU limits and I/O priorities. Some are tied to a particular device instead, like I/O bandwidth or disk usage limits.<br />
<br />
As with all LXD configuration, resource limits can be dynamically changed while the container is running. Some may fail to apply, for example if setting a memory value smaller than the current memory usage, but LXD will try anyway and report back on failure.<br />
<br />
All limits can also be inherited through profiles in which case each affected container will be constrained by that limit. That is, if you set limits.memory=256MB in the default profile, every container using the default profile (typically all of them) will have a memory limit of 256MB.<br />
<br />
==== Disk ====<br />
Setting a size limit on the container’s filesystem and have it enforced against the container. Right now LXD only supports disk limits if you’re using the ZFS or btrfs storage backend.<br />
<br />
To set a disk limit (requires btrfs or ZFS):<br />
<br />
{{console|body=<br />
###i## lxc config device set c1 root size 20GB<br />
}}<br />
<br />
==== CPU ====<br />
To just limit a container to any 2 CPUs, do:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.cpu 2<br />
}}<br />
<br />
To pin to specific CPU cores, say the second and fourth:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.cpu 1,3<br />
}}<br />
<br />
More complex pinning ranges like this works too:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.cpu 0-3,7-11<br />
}}<br />
<br />
==== Memory ====<br />
To apply a straightforward memory limit run:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.memory 256MB<br />
}}<br />
<br />
(The supported suffixes are kB, MB, GB, TB, PB and EB)<br />
<br />
To turn swap off for the container (defaults to enabled):<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.memory.swap false<br />
}}<br />
To tell the kernel to swap this container’s memory first:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.memory.swap.priority 0<br />
}}<br />
And finally if you don’t want hard memory limit enforcement:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.memory.enforce soft<br />
}}<br />
<br />
==== Network ====<br />
==== Block I/O ====<br />
<br />
=== Resource limits using profile - Funtoo Containers example ===<br />
So I am going to create 3 profiles to mimic the resource limits for current Funtoo Containers. <br />
<br />
{{TableStart}}<br />
<tr class="danger"><th>Price</th><th>RAM</th><th>CPU Threads</th><th>Disk Space</th><th>Sign Up</th></tr><br />
<tr><td>'''$15/mo'''</td><td>4GB</td><td>6 CPU Threads</td><td>50GB</td><td>[https://funtoo.chargebee.com/hosted_pages/plans/container_small Sign Up! (small)]</td></tr><br />
<tr><td>'''$30/mo'''</td><td>12GB</td><td>12 CPU Threads</td><td>100GB</td><td>[https://funtoo.chargebee.com/hosted_pages/plans/container_medium Sign Up! (medium)]</td></tr><br />
<tr><td>'''$45/mo'''</td><td>48GB</td><td>24 CPU Threads</td><td>200GB</td><td>[https://funtoo.chargebee.com/hosted_pages/plans/container_large Sign Up! (large)]</td></tr><br />
{{TableEnd}}<br />
<br />
I am going to create one profile and copy/edit it for the remaining two options.<br />
<br />
{{console|body=<br />
###i## lxc profile create res-small<br />
###i## lxc profile edit res-small<br />
config:<br />
limits.cpu: "6"<br />
limits.memory: 4GB<br />
description: Small Variant of Funtoo Containers<br />
devices:<br />
root:<br />
path: /<br />
pool: default<br />
size: 50GB<br />
type: disk<br />
name: small<br />
used_by: []<br />
###i## lxc profile copy res-small res-medium<br />
###i## lxc profile copy res-small res-large<br />
###i## lxc profile set res-medium limits.cpu 12<br />
###i## lxc profile set res-medium limits.memory 12GB<br />
###i## lxc profile device set res-medium root size 100GB<br />
###i## lxc profile set res-large limits.cpu 24<br />
###i## lxc profile set res-large limits.memory 48GB<br />
###i## lxc profile device set res-large root size 200GB<br />
}}<br />
Now let's create a container and assign the res-small and funtoo profiles to it.<br />
{{console|body=<br />
###i## lxc init funtoo c-small<br />
###i## lxc profile assign c-small res-small<br />
###i## lxc profile add c-small funtoo<br />
}}<br />
<br />
=== Image manipulations ===<br />
<br />
=== Remote hosts ===<br />
<br />
== Running systemd container on a non-systemd host ==<br />
To use systemd in the container, a recent enough (>=4.6) kernel version with support for cgroup namespaces is needed. Funtoo's openrc has the fix to mount systemd cgroups, which is sufficient to run systemd based distributions lxd containers. <br />
<br />
If you want to get <code>systemd</code> hierarchy mounted automatically on system startup, using <code>/etc/fstab</code> will not work, but the <br />
{{Package|dev-libs/libcgroup}} can be used for this. First you needed to edit the <code>/etc/cgroup/cgconfig.conf</code> and add:<br />
{{file|name=/etc/cgroup/cgconfig.conf|body=mount {<br />
"name=systemd" = /sys/fs/cgroup/systemd;<br />
}<br />
}}<br />
Then you need to start the cgconfig daemon:<br />
{{console|body=<br />
###i## rc-service cgconfig start<br />
}}<br />
The daemon can be started as needed, or automatically at system start by simply adding it to default group:<br />
{{console|body=<br />
###i## rc-update add cgconfig default<br />
}}<br />
<br />
<hr><br />
<hr><br />
<br />
== [[LXD/LXD in LXD|PART X - LXD in LXD]] ==<br />
== [[LXD/Docker in LXD|PART Y - Docker in LXD]] ==<br />
<br />
== [[LXD/FAQ|PART Z - LXD FAQ]] ==<br />
<br />
== List of tested and working images ==<br />
These are images from the https://images.linuxcontainers.org repository available by default in lxd. You can <br />
list all available images by typing following command (beware the list is very long):<br />
{{console|body=<br />
###i## lxc image list images:<br />
<nowiki>+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCH | SIZE | UPLOAD DATE |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.3 (3 more) | ef69c8dc37f6 | yes | Alpine 3.3 amd64 (20171018_17:50) | x86_64 | 2.00MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.3/armhf (1 more) | 5ce4c80edcf3 | yes | Alpine 3.3 armhf (20170103_17:50) | armv7l | 1.53MB | Jan 3, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.3/i386 (1 more) | cd1700cb7c97 | yes | Alpine 3.3 i386 (20171018_17:50) | i686 | 1.84MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.4 (3 more) | bd4f1ccfabb5 | yes | Alpine 3.4 amd64 (20171018_17:50) | x86_64 | 2.04MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.4/armhf (1 more) | 9fe7c201924c | yes | Alpine 3.4 armhf (20170111_20:27) | armv7l | 1.58MB | Jan 11, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.4/i386 (1 more) | 188a31315773 | yes | Alpine 3.4 i386 (20171018_17:50) | i686 | 1.88MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.5 (3 more) | 63bebc672163 | yes | Alpine 3.5 amd64 (20171018_17:50) | x86_64 | 1.70MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.5/i386 (1 more) | 48045e297515 | yes | Alpine 3.5 i386 (20171018_17:50) | i686 | 1.73MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
...<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| | fd95a7a754a0 | yes | Alpine 3.5 amd64 (20171016_17:50) | x86_64 | 1.70MB | Oct 16, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| | fef66668f5a2 | yes | Debian stretch arm64 (20171016_22:42) | aarch64 | 96.56MB | Oct 16, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| | ff18aa2c11d7 | yes | Opensuse 42.3 amd64 (20171017_00:53) | x86_64 | 58.92MB | Oct 17, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| | ff4ef0d824b6 | yes | Ubuntu zesty s390x (20171017_03:49) | s390x | 86.88MB | Oct 17, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
</nowiki>}}<br />
<br />
These are the images that are known to work with current LXD setup on Funtoo Linux:<br />
{| class="wikitable sortable"<br />
|-<br />
! Image !! Init !! Status<br />
|-<br />
| CentOS 7 || systemd || Working<br />
|-<br />
| Debian Jessie (8) - EOL April/May 2020|| systemd || Working (systemd - no failed units)<br />
|-<br />
| Debian Stretch (9) - EOL June 2022|| systemd || Working<br />
|-<br />
| Fedora 26 || systemd with cgroup v2|| Not Working<br />
|-<br />
| Fedora 25 || systemd || Working<br />
|-<br />
| Fedora 24 || systemd || Working<br />
|-<br />
| Oracle 7 || systemd || Working (systemd - no failed units)<br />
|-<br />
| OpenSUSE 42.2 || systemd || Working<br />
|-<br />
| OpenSUSE 42.3 || systemd || Working<br />
|-<br />
| Ubuntu Xenial (16.04 LTS) - EOL 2021-04 || systemd || Working<br />
|-<br />
| Ubuntu Zesty (17.04) - EOL 2018-01 || systemd || Working<br />
|-<br />
| Alpine 3.3 || OpenRC || Working<br />
|-<br />
| Alpine 3.4 || OpenRC || Working<br />
|-<br />
| Alpine 3.5 || OpenRC || Working<br />
|-<br />
| Alpine 3.6 || OpenRC || Working<br />
|-<br />
| Alpine Edge || OpenRC || Working<br />
|-<br />
| Archlinux || systemd with cgroup v2 || Not Working<br />
|-<br />
| CentOS 6 || upstart || Working (systemd - no failed units)<br />
|-<br />
| Debian Buster || systemd with cgroup v2 || Not Working<br />
|-<br />
| Debian Sid || systemd with cgroup v2 || Not working<br />
|-<br />
| Debian Wheezy (7) - EOL May 2018 || ? || ? (more testing needed)<br />
|-<br />
| Gentoo || OpenRC || Working (all services started)<br />
|-<br />
| Oracle 6 || upstart || ? (mount outputs nothing)<br />
|-<br />
| Plamo 5 || ? || ?<br />
|-<br />
| Plamo 6 || ? || ?<br />
|-<br />
| Sabayon || systemd with cgroup v2 || Not Working<br />
|-<br />
| Ubuntu Artful (17.10) - EOL 2018-07|| systemd with cgroup v2 || Not Working<br />
|-<br />
| Ubuntu Core 16 || ? || ?<br />
|-<br />
| Ubuntu Trusty (14.04 LTS) - EOL 2019-04 || upstart || Working<br />
|}<br />
<br />
[[Category:Virtualization]]</div>Oleghttps://www.funtoo.org/index.php?title=LXD&diff=26582LXD2019-01-16T07:06:59Z<p>Oleg: /* Running systemd container on a non-systemd host */</p>
<hr />
<div>== PART I - Introduction ==<br />
LXD is a container "hypervisor" it should provide user with a new and fresh experience using [[LXC]] technology.<br />
{{#layout:doc}}<br />
__TOC__<br />
<br />
LXD consists of three components:<br />
<br />
* A system-wide daemon (lxd)<br />
* A command line client (lxc)<br />
* An OpenStack Nova plugin (nova-compute-lxd)<br />
<br />
A REST API that is accesible both locally and if enabled, over the network is provided from the lxd daemon.<br />
<br />
The command line tool is designed to be a very simple, yet very powerful tool to manage all your containers. It can handle connections to multiple container hosts and easily give you an overview of all the containers on your network, let you create some more where you want them and even move them around while they're running.<br />
<br />
The OpenStack plugin then allows you to use your lxd hosts as compute nodes, running workloads on containers rather than virtual machines.<br />
<br />
The LXD project was founded and is currently led by Canonical Ltd and Ubuntu with contributions from a range of other companies and individual contributors.<br />
<br />
=== Features ===<br />
<br />
Some of the biggest features of LXD are:<br />
<br />
* Secure by design (unprivileged containers, resource restrictions and much more)<br />
* Scalable (from containers on your laptop to thousand of compute nodes)<br />
* Intuitive (simple, clear API and crisp command line experience)<br />
* Image based (no more distribution templates, only good, trusted images)<br />
* Live migration<br />
<br />
==== Unprivileged Containers ====<br />
<br />
LXD uses unprivileged containers by default. The difference between an unprivileged container and a privileged one is whether the root user in the container is the “real” root user (uid 0 at the kernel level).<br />
<br />
The way unprivileged containers are created is by taking a set of normal UIDs and GIDs from the host, usually at least 65536 of each (to be POSIX compliant) and mapping those into the container.<br />
<br />
The most common example and what most LXD users will end up with by default is a map of 65536 UIDs and GIDs, with a host base id of 100000. This means that root in the container (uid 0) will be mapped to the host uid 100000 and uid 65535 in the container will be mapped to uid 165535 on the host. UID/GID 65536 and higher in the container aren’t mapped and will return an error if you attempt to use them.<br />
<br />
From a security point of view, that means that anything which is not owned by the users and groups mapped into the container will be inaccessible. Any such resource will show up as being owned by uid/gid “-1” (rendered as 65534 or nobody/nogroup in userspace). It also means that should there be a way to escape the container, even root in the container would find itself with just as much privileges on the host as a nobody user.<br />
<br />
LXD does offer a number of options related to unprivileged configuration:<br />
<br />
* Increasing the size of the default uid/gid map<br />
* Setting up per-container maps<br />
* Punching holes into the map to expose host users and groups<br />
<br />
=== Relationship with LXC ===<br />
LXD isn't a rewrite of LXC, in fact it's building on top of LXC to provide a new, better user experience. Under the hood, LXD uses LXC through liblxc and its Go binding<br />
to create and manage the containers.<br />
<br />
It's basically an alternative to LXC's tools and distribution template system with the added features that come from being controllable over the network.<br />
<br />
=== Licensing ===<br />
LXD is free software and is developed under the Apache 2 license.<br />
<br />
<hr><br />
<hr><br />
Let's break this tutorial into smaller parts. Please click on the headings to go to the section's text.<br />
<br />
<hr><br />
<hr><br />
<br />
== [[LXD/LXD Installation|PART II - LXD Installation]] ==<br />
<br />
== [[LXD/LXD Setup|PART III - LXD Setup]] ==<br />
<br />
== Containers, snapshots and images ==<br />
'''Containers''' in LXD are made of:<br />
* A filesystem (rootfs)<br />
* A list of configuration options, including resource limits, environment, security options and more<br />
* A bunch of devices like disks, character/block unix devices and network interfaces<br />
* A set of profiles the container inherits configuration from (see below)<br />
* Some properties (container architecture, ephemeral or persistent and the name)<br />
* Some runtime state (when using CRIU for checkpoint/restore)<br />
<br />
Container '''snapshots''' as the name states snapshots of the container in time and cannot be modified in any way. It is worth noting that because snapshots can store the container runtime state, which gives us ability of “stateful” snapshots. That is, the ability to rollback the container including its cpu and memory state at the time of the snapshot.<br />
<br />
LXD is '''image''' based, all LXD containers come from an image. Images are typically clean Linux distribution images similar to what you would use for a virtual machine or cloud instance. It is possible to “publish” a container, making an image from it which can then be used by the local or remote LXD hosts.<br />
<br />
=== Our first image ===<br />
Let's get our hands even more dirty and create our first image. We will be using a generic 64 bit Funtoo Linux image. <br />
<br />
{{note|The Funtoo's default build host is building only westmere stage for now.}}<br />
<br />
Grab the image here:<br />
https://build.funtoo.org/funtoo-current/x86-64bit/intel64-westmere/lxd-latest.tar.xz<br />
<br />
Grab also the hash file:<br />
https://build.funtoo.org/funtoo-current/x86-64bit/intel64-westmere/lxd-latest.tar.xz.hash.txt<br />
<br />
{{tip|Check the hash of the downloaded file against the one from server. Proceed if they match. }}<br />
<br />
==== Import the image ====<br />
After we have successfully downloaded the archive we can now finally import it into LXD and start using it as our "seed" image for all our containers.<br />
{{console|body=<br />
###i## lxc image import lxd-latest.tar.xz --alias funtoo<br />
Image imported with fingerprint: 6c2ca3af0222d503656f5a1838885f1b9b6aed2c1994f1d7ef94e2efcb7233c4<br />
###i## lxc image ls<br />
<nowiki>+--------+--------------+--------+------------------------------------+--------+----------+-----------------------------+<br />
| ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCH | SIZE | UPLOAD DATE |<br />
+--------+--------------+--------+------------------------------------+--------+----------+-----------------------------+<br />
| funtoo | 6c2ca3af0222 | no | Funtoo Current Generic Pure 64-bit | x86_64 |227.99MB | Dec 13, 2017 at 11:01pm (UTC) |<br />
+--------+--------------+--------+------------------------------------+--------+----------+-----------------------------+<br />
</nowiki>}}<br />
And there we have our very first Funtoo Linux image imported inside LXD. You can reference the image through the alias or through the fingerprint. Aliases can be added also later.<br />
<br />
Let me show you some basic usage then.<br />
<br />
=== Creating your first container ===<br />
So now we can launch our first container. That is done using this command:<br />
<br />
{{console|body=<br />
###i## lxc launch funtoo fun-1<br />
Creating fun-1<br />
Starting fun-1<br />
###i## lxc ls<br />
<nowiki>+-------+---------+------+-----------------------------------------------+------------+-----------+<br />
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |<br />
+-------+---------+------+-----------------------------------------------+------------+-----------+<br />
| fun-1 | RUNNING | | fd42:156d:4593:a619:216:3eff:fef7:c1c2 (eth0) | PERSISTENT | 0 |<br />
+-------+---------+------+-----------------------------------------------+------------+-----------+<br />
</nowiki>}}<br />
<br />
{{tip|lxc launch is a shortcut for lxc init and lxc start, lxc init creates the container without starting it. }}<br />
<br />
==== Profiles intermezzo ====<br />
LXD has the ability to change quite a few container settings, including resource limitation, control of container startup and a variety of device pass-through options using what is called profiles. Multiple profiles can be applied to a single container, and the last profile overrides the other ones it the resources being configured is the same for multiple profiles. Let me show you how can this be used.<br />
<br />
This is the default profile that gets inherited by all containers.<br />
{{console|body=<br />
###i## lxc profile list<br />
<nowiki>+---------+---------+<br />
| NAME | USED BY |<br />
+---------+---------+<br />
| default | 1 |<br />
+---------+---------+<br />
</nowiki><br />
###i## lxc profile show default<br />
config: {}<br />
description: Default LXD profile<br />
devices:<br />
eth0:<br />
nictype: bridged<br />
parent: lxdbr0<br />
type: nic<br />
root:<br />
path: /<br />
pool: default<br />
type: disk<br />
name: default<br />
used_by:<br />
- /1.0/containers/fun-1<br />
}}<br />
<br />
Now let's edit this profile for our funtoo containers. It will include some useful stuff.<br />
<br />
{{console|body=<br />
###i## lxc profile set default raw.lxc "lxc.mount.entry = none dev/shm tmpfs rw,nosuid,nodev,create=dir"<br />
###i## lxc profile set default environment.LANG "en_US.UTF-8"<br />
###i## lxc profile set default environment.LC_ALL "en_US.UTF-8"<br />
###i## lxc profile set default environment.LC_COLLATE "POSIX"<br />
}}<br />
<br />
Profiles can store any configuration that a container can (key/value or devices) and any number of profiles can be applied to a container. Profiles are applied in the order they are specified so the last profile to specify a specific key wins. In any case, resource-specific configuration always overrides that coming from the profiles.<br />
<br />
The default profile is set for any new container created which doesn't specify a different profiles list.<br />
<br />
{{note|LXD supports simple instance types. Those are represented as a string which can be passed at container creation time. [https://github.com/lxc/lxd/blob/master/doc/containers.md#instance-types containers.md#instance-types]}}<br />
<br />
=== Using our first container ===<br />
After we have done all these customizations we can now start using our container.<br />
The next command will give us shell inside the container.<br />
<br />
{{console|body=<br />
###i## lxc exec fun-1 bash<br />
}}<br />
<br />
Now you should see a different prompt starting with<br />
<br />
{{console|body=<br />
fun-1 ~ #<br />
}}<br />
<br />
If we run top or ps for example we will see only the processes of the container.<br />
<br />
{{console|body=<br />
fun-1 ~ # ps aux<br />
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND<br />
root 1 0.0 0.0 4248 748 ? Ss+ 13:20 0:00 init [3]<br />
root 266 0.0 0.0 30488 472 ? Ss 13:20 0:00 /usr/sbin/sshd<br />
root 312 0.2 0.0 17996 3416 ? Ss 13:29 0:00 bash<br />
root 317 0.0 0.0 19200 2260 ? R+ 13:29 0:00 ps aux<br />
}}<br />
<br />
As you can see only the container's processes are shown. User running the processes is root here. What happens if we search for all sshd processes for example on the host box?<br />
<br />
{{console|body=<br />
###i## <nowiki>ps aux|grep ssh<br />
root 14505 0.0 0.0 30564 1508 ? Ss Sep07 0:00 /usr/sbin/sshd <br />
100000 25863 0.0 0.0 30488 472 ? Ss 15:20 0:00 /usr/sbin/sshd <br />
root 29487 0.0 0.0 8324 828 pts/2 S+ 15:30 0:00 grep --colour=auto sshd</nowiki><br />
###i##<br />
}}<br />
<br />
So as you can see, the sshd process is running under user with uid 100000 on the host machine and has a different PID.<br />
<br />
=== Basic actions with containers ===<br />
==== Listing containers ====<br />
{{console|body=<br />
###i## lxc ls<br />
<nowiki>+-------+---------+-----------------------+------------------------------------------------+------------+-----------+<br />
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |<br />
+-------+---------+-----------------------+------------------------------------------------+------------+-----------+<br />
| fun-1 | RUNNING | 10.214.101.187 (eth0) | fd42:156d:4593:a619:a5ad:edaf:7270:e6c4 (eth0) | PERSISTENT | 0 |<br />
| | | | fd42:156d:4593:a619:216:3eff:fef7:c1c2 (eth0) | | |<br />
+-------+---------+-----------------------+------------------------------------------------+------------+-----------+<br />
</nowiki>}}<br />
lxc ls also accepts arguments as filters. For example lxc ls web will list all containers that have web in their name.<br />
<br />
==== Container details ====<br />
{{console|body=<br />
###i## lxc info c1<br />
Name: c1<br />
Remote: unix://<br />
Architecture: x86_64<br />
Created: 2017/09/08 02:07 UTC<br />
Status: Running<br />
Type: persistent<br />
Profiles: default, prf-funtoo<br />
Pid: 6366<br />
Ips:<br />
eth0: inet 10.214.101.79 vethFG4HXG<br />
eth0: inet6 fd42:156d:4593:a619:8619:546e:43f:2089 vethFG4HXG<br />
eth0: inet6 fd42:156d:4593:a619:216:3eff:fe4a:3d4f vethFG4HXG<br />
eth0: inet6 fe80::216:3eff:fe4a:3d4f vethFG4HXG<br />
lo: inet 127.0.0.1<br />
lo: inet6 ::1<br />
Resources:<br />
Processes: 6<br />
CPU usage:<br />
CPU usage (in seconds): 25<br />
Memory usage:<br />
Memory (current): 69.01MB<br />
Memory (peak): 258.92MB<br />
Network usage:<br />
eth0:<br />
Bytes received: 83.65kB<br />
Bytes sent: 9.44kB<br />
Packets received: 188<br />
Packets sent: 93<br />
lo:<br />
Bytes received: 0B<br />
Bytes sent: 0B<br />
Packets received: 0<br />
Packets sent: 0<br />
}}<br />
<br />
==== Container configuration ====<br />
{{console|body=<br />
###i## lxc config edit c1<br />
### This is a yaml representation of the configuration.<br />
### Any line starting with a '# will be ignored.<br />
###<br />
### A sample configuration looks like:<br />
### name: container1<br />
### profiles:<br />
### - default<br />
### config:<br />
### volatile.eth0.hwaddr: 00:16:3e:e9:f8:7f<br />
### devices:<br />
### homedir:<br />
### path: /extra<br />
### source: /home/user<br />
### type: disk<br />
### ephemeral: false<br />
###<br />
### Note that the name is shown but cannot be changed<br />
<br />
architecture: x86_64<br />
config:<br />
image.architecture: x86_64<br />
image.description: Funtoo Current Generic Pure 64-bit<br />
image.name: funtoo-generic_64-pure64-funtoo-current-2016-12-10<br />
image.os: funtoo<br />
image.release: "1.0"<br />
image.variant: current<br />
volatile.base_image: e279c16d1a801b2bd1698df95e148e0a968846835f4769b24988f2eb3700100f<br />
volatile.eth0.hwaddr: 00:16:3e:4a:3d:4f<br />
volatile.eth0.name: eth0<br />
volatile.idmap.base: "0"<br />
volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'<br />
volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'<br />
volatile.last_state.power: RUNNING<br />
devices: {}<br />
ephemeral: false<br />
profiles:<br />
- default<br />
- prf-funtoo<br />
stateful: false<br />
description: ""<br />
}}<br />
<br />
One can also add environment variables.<br />
{{console|body=<br />
###i## lxc config set <container> environment.LANG en_US.UTF-8<br />
###i## lxc config set <container> environment.LC_COLLATE POSIX<br />
}}<br />
<br />
=== Managing files ===<br />
<br />
=== Snapshots ===<br />
<br />
=== Cloning, copying and moving containers ===<br />
<br />
=== Resource control ===<br />
LXD offers a variety of resource limits. Some of those are tied to the container itself, like memory quotas, CPU limits and I/O priorities. Some are tied to a particular device instead, like I/O bandwidth or disk usage limits.<br />
<br />
As with all LXD configuration, resource limits can be dynamically changed while the container is running. Some may fail to apply, for example if setting a memory value smaller than the current memory usage, but LXD will try anyway and report back on failure.<br />
<br />
All limits can also be inherited through profiles in which case each affected container will be constrained by that limit. That is, if you set limits.memory=256MB in the default profile, every container using the default profile (typically all of them) will have a memory limit of 256MB.<br />
<br />
==== Disk ====<br />
Setting a size limit on the container’s filesystem and have it enforced against the container. Right now LXD only supports disk limits if you’re using the ZFS or btrfs storage backend.<br />
<br />
To set a disk limit (requires btrfs or ZFS):<br />
<br />
{{console|body=<br />
###i## lxc config device set c1 root size 20GB<br />
}}<br />
<br />
==== CPU ====<br />
To just limit a container to any 2 CPUs, do:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.cpu 2<br />
}}<br />
<br />
To pin to specific CPU cores, say the second and fourth:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.cpu 1,3<br />
}}<br />
<br />
More complex pinning ranges like this works too:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.cpu 0-3,7-11<br />
}}<br />
<br />
==== Memory ====<br />
To apply a straightforward memory limit run:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.memory 256MB<br />
}}<br />
<br />
(The supported suffixes are kB, MB, GB, TB, PB and EB)<br />
<br />
To turn swap off for the container (defaults to enabled):<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.memory.swap false<br />
}}<br />
To tell the kernel to swap this container’s memory first:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.memory.swap.priority 0<br />
}}<br />
And finally if you don’t want hard memory limit enforcement:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.memory.enforce soft<br />
}}<br />
<br />
==== Network ====<br />
==== Block I/O ====<br />
<br />
=== Resource limits using profile - Funtoo Containers example ===<br />
So I am going to create 3 profiles to mimic the resource limits for current Funtoo Containers. <br />
<br />
{{TableStart}}<br />
<tr class="danger"><th>Price</th><th>RAM</th><th>CPU Threads</th><th>Disk Space</th><th>Sign Up</th></tr><br />
<tr><td>'''$15/mo'''</td><td>4GB</td><td>6 CPU Threads</td><td>50GB</td><td>[https://funtoo.chargebee.com/hosted_pages/plans/container_small Sign Up! (small)]</td></tr><br />
<tr><td>'''$30/mo'''</td><td>12GB</td><td>12 CPU Threads</td><td>100GB</td><td>[https://funtoo.chargebee.com/hosted_pages/plans/container_medium Sign Up! (medium)]</td></tr><br />
<tr><td>'''$45/mo'''</td><td>48GB</td><td>24 CPU Threads</td><td>200GB</td><td>[https://funtoo.chargebee.com/hosted_pages/plans/container_large Sign Up! (large)]</td></tr><br />
{{TableEnd}}<br />
<br />
I am going to create one profile and copy/edit it for the remaining two options.<br />
<br />
{{console|body=<br />
###i## lxc profile create res-small<br />
###i## lxc profile edit res-small<br />
config:<br />
limits.cpu: "6"<br />
limits.memory: 4GB<br />
description: Small Variant of Funtoo Containers<br />
devices:<br />
root:<br />
path: /<br />
pool: default<br />
size: 50GB<br />
type: disk<br />
name: small<br />
used_by: []<br />
###i## lxc profile copy res-small res-medium<br />
###i## lxc profile copy res-small res-large<br />
###i## lxc profile set res-medium limits.cpu 12<br />
###i## lxc profile set res-medium limits.memory 12GB<br />
###i## lxc profile device set res-medium root size 100GB<br />
###i## lxc profile set res-large limits.cpu 24<br />
###i## lxc profile set res-large limits.memory 48GB<br />
###i## lxc profile device set res-large root size 200GB<br />
}}<br />
Now let's create a container and assign the res-small and funtoo profiles to it.<br />
{{console|body=<br />
###i## lxc init funtoo c-small<br />
###i## lxc profile assign c-small res-small<br />
###i## lxc profile add c-small funtoo<br />
}}<br />
<br />
=== Image manipulations ===<br />
<br />
=== Remote hosts ===<br />
<br />
== Running systemd container on a non-systemd host ==<br />
To use systemd in the container, a recent enough (>=4.6) kernel version with support for cgroup namespaces is needed. Funtoo's openrc has the fix to mount systemd cgroups, which is sufficient to run systemd based distributions lxd images. <br />
<br />
If you want to get <code>systemd</code> hierarchy mounted automatically on system startup, using <code>/etc/fstab</code> will not work, but the <br />
{{Package|dev-libs/libcgroup}} can be used for this. First you needed to edit the <code>/etc/cgroup/cgconfig.conf</code> and add:<br />
{{file|name=/etc/cgroup/cgconfig.conf|body=mount {<br />
"name=systemd" = /sys/fs/cgroup/systemd;<br />
}<br />
}}<br />
Then you need to start the cgconfig daemon:<br />
{{console|body=<br />
###i## rc-service cgconfig start<br />
}}<br />
The daemon can be started as needed, or automatically at system start by simply adding it to default group:<br />
{{console|body=<br />
###i## rc-update add cgconfig default<br />
}}<br />
<br />
<hr><br />
<hr><br />
<br />
== [[LXD/LXD in LXD|PART X - LXD in LXD]] ==<br />
== [[LXD/Docker in LXD|PART Y - Docker in LXD]] ==<br />
<br />
== [[LXD/FAQ|PART Z - LXD FAQ]] ==<br />
<br />
== List of tested and working images ==<br />
These are images from the https://images.linuxcontainers.org repository available by default in lxd. You can <br />
list all available images by typing following command (beware the list is very long):<br />
{{console|body=<br />
###i## lxc image list images:<br />
<nowiki>+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCH | SIZE | UPLOAD DATE |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.3 (3 more) | ef69c8dc37f6 | yes | Alpine 3.3 amd64 (20171018_17:50) | x86_64 | 2.00MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.3/armhf (1 more) | 5ce4c80edcf3 | yes | Alpine 3.3 armhf (20170103_17:50) | armv7l | 1.53MB | Jan 3, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.3/i386 (1 more) | cd1700cb7c97 | yes | Alpine 3.3 i386 (20171018_17:50) | i686 | 1.84MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.4 (3 more) | bd4f1ccfabb5 | yes | Alpine 3.4 amd64 (20171018_17:50) | x86_64 | 2.04MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.4/armhf (1 more) | 9fe7c201924c | yes | Alpine 3.4 armhf (20170111_20:27) | armv7l | 1.58MB | Jan 11, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.4/i386 (1 more) | 188a31315773 | yes | Alpine 3.4 i386 (20171018_17:50) | i686 | 1.88MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.5 (3 more) | 63bebc672163 | yes | Alpine 3.5 amd64 (20171018_17:50) | x86_64 | 1.70MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.5/i386 (1 more) | 48045e297515 | yes | Alpine 3.5 i386 (20171018_17:50) | i686 | 1.73MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
...<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| | fd95a7a754a0 | yes | Alpine 3.5 amd64 (20171016_17:50) | x86_64 | 1.70MB | Oct 16, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| | fef66668f5a2 | yes | Debian stretch arm64 (20171016_22:42) | aarch64 | 96.56MB | Oct 16, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| | ff18aa2c11d7 | yes | Opensuse 42.3 amd64 (20171017_00:53) | x86_64 | 58.92MB | Oct 17, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| | ff4ef0d824b6 | yes | Ubuntu zesty s390x (20171017_03:49) | s390x | 86.88MB | Oct 17, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
</nowiki>}}<br />
<br />
These are the images that are known to work with current LXD setup on Funtoo Linux:<br />
{| class="wikitable sortable"<br />
|-<br />
! Image !! Init !! Status<br />
|-<br />
| CentOS 7 || systemd || Working<br />
|-<br />
| Debian Jessie (8) - EOL April/May 2020|| systemd || Working (systemd - no failed units)<br />
|-<br />
| Debian Stretch (9) - EOL June 2022|| systemd || Working<br />
|-<br />
| Fedora 26 || systemd with cgroup v2|| Not Working<br />
|-<br />
| Fedora 25 || systemd || Working<br />
|-<br />
| Fedora 24 || systemd || Working<br />
|-<br />
| Oracle 7 || systemd || Working (systemd - no failed units)<br />
|-<br />
| OpenSUSE 42.2 || systemd || Working<br />
|-<br />
| OpenSUSE 42.3 || systemd || Working<br />
|-<br />
| Ubuntu Xenial (16.04 LTS) - EOL 2021-04 || systemd || Working<br />
|-<br />
| Ubuntu Zesty (17.04) - EOL 2018-01 || systemd || Working<br />
|-<br />
| Alpine 3.3 || OpenRC || Working<br />
|-<br />
| Alpine 3.4 || OpenRC || Working<br />
|-<br />
| Alpine 3.5 || OpenRC || Working<br />
|-<br />
| Alpine 3.6 || OpenRC || Working<br />
|-<br />
| Alpine Edge || OpenRC || Working<br />
|-<br />
| Archlinux || systemd with cgroup v2 || Not Working<br />
|-<br />
| CentOS 6 || upstart || Working (systemd - no failed units)<br />
|-<br />
| Debian Buster || systemd with cgroup v2 || Not Working<br />
|-<br />
| Debian Sid || systemd with cgroup v2 || Not working<br />
|-<br />
| Debian Wheezy (7) - EOL May 2018 || ? || ? (more testing needed)<br />
|-<br />
| Gentoo || OpenRC || Working (all services started)<br />
|-<br />
| Oracle 6 || upstart || ? (mount outputs nothing)<br />
|-<br />
| Plamo 5 || ? || ?<br />
|-<br />
| Plamo 6 || ? || ?<br />
|-<br />
| Sabayon || systemd with cgroup v2 || Not Working<br />
|-<br />
| Ubuntu Artful (17.10) - EOL 2018-07|| systemd with cgroup v2 || Not Working<br />
|-<br />
| Ubuntu Core 16 || ? || ?<br />
|-<br />
| Ubuntu Trusty (14.04 LTS) - EOL 2019-04 || upstart || Working<br />
|}<br />
<br />
[[Category:Virtualization]]</div>Oleghttps://www.funtoo.org/index.php?title=Upgrade_Instructions/1.3-release&diff=26581Upgrade Instructions/1.3-release2019-01-16T05:46:57Z<p>Oleg: add category</p>
<hr />
<div><languages/><br />
<translate><br />
<!--T:1--><br />
{{Warning|Upgrading to 1.3 will remove any 32-bit compatibility on your system, as we have deprecated multilib support! Please be aware of this before starting the upgrade process.}}<br />
{{Important|The goal of these instructions is to provide Funtoo Linux users with a reliable, consistent set of instructions for upgrading Funtoo Linux from 1.2 to 1.3. Please assist in ensuring that these instructions are complete and guide users through any potential complications. Since this is a wiki, make changes to the page that are needed to make these instructions 100% reliable. Thank you!}}<br />
<br />
=== Set Release and Sync ===<br />
<br />
{{Important|Make sure you have {{c|ego 2.7.2}} or later installed!}}<br />
<br />
In {{f|/etc/ego.conf}}, set the release to 1.3:<br />
<br />
{{file|name=/etc/ego.conf|body=<br />
[global]<br />
<br />
release = 1.3<br />
}}<br />
<br />
Then, run {{c|ego sync}}:<br />
<br />
{{console|body=<br />
# ##i##ego sync<br />
}}<br />
<!--T:2--><br />
<br />
=== Optionally Remove Xorg-Server ===<br />
<br />
1.3-release contains a completely reworked xorg-server, and as such, it is best to ''remove'' your old xorg-server prior to updating:<br />
<br />
{{console|body=<br />
# ##i##emerge -C xorg-server<br />
# ##i##emerge -aC $(qlist -IC x11-proto)<br />
# ##i##emerge --oneshot x11-base/xorg-proto<br />
}}<br />
<!--T:3--><br />
<br />
=== Optionally Retarget Pure64 ===<br />
<br />
If you are running a {{c|pure64}} build of Funtoo Linux, you will need to run the following command to update your {{f|/etc/portage/make.profile/parent}} file to no longer reference the {{c|pure64}} arch profile, as it has been deprecated.<br />
Run the following command:<br />
<br />
{{console|body=<br />
# ##i##epro arch x86-64bit<br />
##y##WARNING: Previous value: x86-64bit -- typically, user should not change this.<br />
<br />
=== Enabled Profiles: ===<br />
<br />
arch: ##c##x86-64bit<br />
build: ##c##current<br />
subarch: ##c##intel64-westmere<br />
flavor: ##c##core<br />
mix-ins: ##c##mediaformat-gfx-common<br />
mix-ins: ##c##mediaformat-gfx-extra<br />
<br />
>>> Set arch to x86-64bit.<br />
Updating profiles at /etc/portage/make.profile/parent...<br />
}}<br />
<!--T:4--><br />
=== Relax Deps and Rebuild ===<br />
<br />
Before upgrading, it is a good idea to perform the following commands to relax any existing 32-bit ABI deps so that installed packages don't block necessary updates. Be sure to back up {{f|/var/db/pkg}}, as included in the instructions below, and specify the {{c|find}} commands below exactly -- best to copy and paste:<br />
<br />
{{console|body=<br />
# ##i##cd /var/db<br />
# ##i##cp -a pkg /var/tmp/pkg.bak<br />
# ##i##cd pkg<br />
# ##i##find -iname RDEPEND -exec sed -i -e 's/\[abi_x86_32(-),abi_x86_64(-)]//g' {} \;<br />
# ##i##find -iname RDEPEND -exec sed -i -e 's/,abi_x86_32(-),abi_x86_64(-)]/]/g' {} \;<br />
}}<br />
<br />
Any critical installed packages should no longer depend on ebuilds providing 32-bit ABIs.<br />
<br />
Now, proceed to perform a system upgrade followed by a world upgrade:<br />
<br />
{{console|body=<br />
# ##i##emerge -u1 gcc<br />
# ##i##emerge -u1 glibc<br />
}}<br />
<br />
=== Perl Rebuild ===<br />
With 1.3-release {{c|dev-lang/perl}} updated from 5.24 to 5.26 version. It is necessary to rebuild perl modules installed with such major version update, which can be done with:<br />
{{console|body=<br />
# ##i##emerge -v1 --nodeps dev-lang/perl<br />
# ##i##perl-cleaner --all<br />
}}<br />
*Note: Some systems may require perl-cleaner --reallyall<br />
<br />
{{console|body=<br />
# ##i##emerge -auDN @system --ignore-world<br />
# ##i##emerge -auDN @world<br />
}}<br />
<br />
{{Important|It appears that {{c|dev-lang/go}} will keep rebuilding against a preserved 32-bit glibc, so to fully remove multilib on a system that has {{c|dev-lang/go}} installed, you will need to perform the following steps: {{c|emerge -C dev-lang/go; emerge dev-lang/go}}.}}<br />
<br />
Now, it should be possible to rebuild any necessary packages to get rid of preserved libraries, paying particular attention to any old versions of glibc:<br />
<br />
{{console|body=<br />
# ##i##emerge -av @preserved-rebuild<br />
}}<br />
<!--T:5--><br />
<br />
Now, after update and rebuild you will want to either run {{c|etc-update}} or {{c|dispatch-conf}} to perform changes to the configuration files that may happen with ebuild updates:<br />
{{console|body=<br />
# ##i##etc-update<br />
}}<br />
<!--T:6--><br />
<br />
=== Optionally Update Kernel ===<br />
<br />
If you were not using {{c|debian-sources-lts}} before, you may want to upgrade to this kernel. Do this as follows:<br />
<br />
{{console|body=<br />
# ##i##emerge -av debian-sources-lts<br />
# ##i##ego boot update<br />
}}<br />
<br />
Depending on your {{f|/etc/boot.conf}} settings, you may need to tweak the file in order to have {{c|debian-sources-lts}} selected by default.<br />
<br />
{{Important|Remember to rebuild any necessary kernel modules!}}<br />
This can be achieved with:<br />
{{console|body=<br />
# ##i##emerge -av @module-rebuild --exclude debian-sources-lts<br />
}}<br />
<!--T:7--><br />
=== Reboot ===<br />
<br />
At this point, we recommend rebooting the system to ensure you are running in the new environment:<br />
<br />
{{console|body=<br />
# ##i##reboot<br />
}}<br />
<br />
</translate><br />
<br />
[[Category:Official Documentation]]<br />
[[Category:Upgrade Instructions]]</div>Oleghttps://www.funtoo.org/index.php?title=Funtoo:Networking/Configuration&diff=26550Funtoo:Networking/Configuration2019-01-08T17:34:08Z<p>Oleg: /* Configuration Variables */</p>
<hr />
<div><blockquote>This document explains how to configure your network settings by explaining the network configuration functionality available in Funtoo Linux. Also covered is <tt>dhcpcd 5.x</tt>, Wi-Fi (IEEE 802.11) configuration, and the OpenResolv framework.<br />
</blockquote><br />
<br />
== Introduction ==<br />
__TOC__<br />
Funtoo Linux has its own core network configuration system that differs somewhat from upstream network configuration systems used in [http://www.gentoo.org Gentoo Linux] and [http://roy.marples.name/projects/openrc OpenRC].<br />
<br />
In this document, I will explain the unique additions and changes to the Funtoo network configuration and show you how to use this system to configure your network.<br />
<br />
I'll also explain how to use <code>dhcpcd</code> for managing network interfaces on DHCP-based networks, and will also cover OpenRC stacked runlevel configuration, ''Wi-Fi'' (IEEE 802.11) configuration, and the OpenResolv framework, which is enabled in Funtoo Linux by default.<br />
<br />
== A Gentle Introduction to Funtoo Network Configuration ==<br />
<br />
Before I get into the technical details of configuring your network, it's important to understand that Funtoo Linux has a number of different options available to you for network configuration, with more likely to be added in the future. Each approach is different and has its own strengths and weaknesses, and this is, in my opinion, a good thing.<br />
<br />
=== The Easy (Dynamic) Way ===<br />
<br />
When configuring your network, one option is to skip traditional network configuration and simply rely on DHCP. This is by far the simplest method of configuring your network. If you are on a wired network, no other steps are typically required beyond enabling a DHCP client, and Funtoo Linux includes {{c|dhcpcd 7.x}} by default. <br />
<br />
==== Network Manager, Wicd ====<br />
<br />
If you are going to use a third party package such as [[Network Manager]] or [[Wicd]] to manage your network then you do not need to configure DHCP at all. These packages configure DHCP for you. Simply emerge the package you want to use and start using it.<br />
<br />
==== DHCP-Only Systems ====<br />
<br />
If you are not planning to use a third-party package to manage your network interfaces, it is still extremely easy to set up DHCP networking, especially if you always use DHCP to connect to networks, which is common for desktops or laptops. In this scenario, we can simply enable {{c|dhcpcd}} to run at system startup. It will run in the background and automatically look for DHCP servers on all your network interfaces, and will attempt to lease an IP address from any DHCP servers found. <br />
<br />
If this sounds like what you want to do, then add {{c|dhcpcd}} to your default runlevel as follows:<br />
<br />
<console># ##i##rc-update add dhcpcd default</console><br />
<br />
To enable DHCP immediately, you would follow the previous command with an {{c|openrc}} command, which would start the {{c|dhcpcd}} client you just added:<br />
<br />
{{console|body=<br />
###i## openrc<br />
}}<br />
<br />
If you're on a wired network and have the necessary drivers in your kernel, then this should get you going. For wireless networks, more steps are required to utilize your wireless hardware to associate with an access point, which will be covered later in this document. <br />
<br />
===== Tweaking Dhcpcd =====<br />
<br />
For now, it's important to note that {{c|dhcpcd 7.x}} will manage ''all'' available network interfaces by default. If you want to run a DHCP client on ''all but one'' interface, or some other subset of interfaces, you can add the appropriate {{c|denyinterfaces}} or {{c|allowinterfaces}} [[glob pattern]] to {{c|/etc/dhcpcd.conf}}:<br />
<br />
{{file|name=/etc/dhcpcd.conf|lang=bash|body=<br />
# manage all interfaces but eth0 with dhcpcd<br />
denyinterfaces eth0<br />
}}<br />
<br />
This can also be accomplished by modifying {{c|tc/init.d/dhcpcd}} directly and adding {{c|-Z ''ifglob''}} or {{c|-z ''ifglob''}} (the equivalent command-line parameters) to {{c|command_args}}.<br />
<br />
==== Using Funtoo Scripts for DHCP ====<br />
<br />
You can also use the Funtoo Linux networking scripts to start a DHCP client just on a specific interface. This approach is best if you are planning to also do some advanced bridging, bonding or VLAN configuration on your machine along with DHCP, since you will be using the Funtoo Linux networking scripts for that too. <br />
<br />
To use this variant approach, ''don't'' enable {{c|/etc/init.d/dhcpcd}} directly. Instead, use the Funtoo Linux {{c|dhcpcd}} template which will start dhcpcd on only one interface. Below, you will see the steps to do this. This is very similar to how we set up advanced network interfaces, which will be covered later in this documentation:<br />
<br />
{{console|body=<br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##rc-update add net.eth0 default<br />
# ##i##echo template=dhcpcd > /etc/conf.d/net.eth0<br />
# ##i##openrc<br />
}}<br />
<br />
The last command, {{c|openrc}} causes {{c|net.eth0}} to be started.<br />
<br />
=== Server Network Configuration ===<br />
<br />
For servers and advanced networking scenarios, Funtoo Linux offers its own modular, template-based network configuration system. This system offers a lot of flexibility for configuring network interfaces, essentially serving as a &quot;network interface construction kit.&quot; This system can be used by itself, or even combined with <tt>dhcpcd</tt>, as shown in the previous section.<br />
<br />
Here are the key components of the template-based network configuration system:<br />
<br />
;{{c|/etc/init.d/loopback}}: An init script that configures the localhost interface. This script is always enabled and is part of the boot process.<br />
;{{c|/etc/netif.d}}: This is a directory that contains various network configuration templates. Each of these templates is focused on configuring a particular type of network interface, such as a general static IP-based interface, a bridge interface, a bond interface, etc.<br />
;{{c|/etc/init.d/netif.tmpl}}: This is the master init script for the template-based network configuration system. New interfaces are added to your system by creating '''symbolic links''' to this file in {{c|/etc/init.d}}.<br />
<br />
So, if you wanted to use this system to configure {{c|eth0}} with a static IP address, you would create a {{c|net.eth0}} symlink to {{c|netif.tmpl}} as follows:<br />
<br />
{{console|body=<br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth0<br />
}}<br />
<br />
Then, you would create an {{c|/etc/conf.d/net.eth0}} configuration file that would specify which template to use from the {{c|/etc/netif.d}} directory:<br />
<br />
{{file|name=/etc/conf.d/net.eth0|lang=bash|body=<br />
template="interface"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
}}<br />
<br />
To complete our static IP network configuration we would need to:<br />
<br />
{{console|body=# ##i##rc-update add net.eth0 default}}<br />
When configuring your own static network interface, one of {{c|ipaddr}} or {{c|ipaddrs}} is required and should specify the IP address(es) to configure for this interface, in &quot;a.b.c.d/netmask&quot; format. Optional parameters include {{c|gateway}}, which defines a default gateway for your entire network, and if set should specify the gateway's IP address. In addition, {{c|domain}} and {{c|nameservers}} (space-separated if more than one) can be used to specify DNS information for this interface.<br />
<br />
=== Configuration Variables ===<br />
<br />
==== Interface Variables ====<br />
<br />
The {{c|ipaddr}} and {{c|ipaddrs}} variables are supported by the {{c|interface}} and {{c|bridge}} templates, and are used to specify a single or multiple IPv4 or IPv6 address(es) for the interface. IP addresses should be specified in 'IP/netmask' format, such as {{c|10.0.0.1/24}}. Multiple IP addresses can be specified delimited by whitespace:<br />
<br />
<pre>ipaddrs=&quot;10.0.0.1/24 10.0.0.2/24&quot;</pre><br />
<br />
===== Broadcast Address =====<br />
<br />
By default, a broadcast address will be calculated based on the IP address and network mask. If you need to manually specify a broadcast address, use the following format for your IP address:<br />
<br />
<pre><br />
ipaddrs="10.0.0.1/24;broadcast=10.0.1.255 10.0.0.2/24"<br />
</pre><br />
<br />
===== Not Specifying An Address =====<br />
<br />
Note that in some cases, you may choose to '''not''' specify <tt>ipaddr</tt> or <tt>ipaddrs</tt> for a <tt>bridge</tt> template. That is allowed. If you don't want to specify an IP address for a regular interface, you can choose to use the <tt>interface</tt> template without an IP address specified in the config, or use the <tt>interface-noip</tt> template instead, for the sake of clarity.<br />
<br />
===== Viewing All Configured IP Addresses =====<br />
<br />
Also note that if you specify multiple IPv4 addresses, <tt>ifconfig</tt> will only show the first IP address. To view all IP addresses associated with the interface, use the <tt>ip addr show</tt> command.<br />
<br />
=== General Variables ===<br />
<br />
The following variables are enabled by default for all network scripts, and if specified will trigger a corresponding configuration action:<br />
<br />
;<tt>nameservers</tt>: Set DNS nameservers using OpenResolv. Specify multiple IPv4 or IPv6 nameservers like this: &quot;1.2.3.4 1.2.3.5 1.2.3.6&quot;. Please note that OpenResolv treats <tt>127.0.0.1</tt> specially, and it indicates that you are running a local name resolver like <tt>dnsmasq</tt> or <tt>bind</tt>. OpenResolv will ignore all other name servers specified alongside <tt>127.0.0.1</tt>. See <tt>man resolvconf</tt> and <tt>man resolvconf.conf</tt> for additional setup information.<br />
;<tt>search</tt>: Set DNS search information using OpenResolv.<br />
;<tt>domain</tt>: Set DNS domain using OpenResolv.<br />
;<tt>gateway</tt>: Define a default IPv4 gateway on this interface.<br />
;<tt>gateway6</tt>: Define a default IPv6 gateway on this interface.<br />
;<tt>route</tt>: Specify a semi-colon delimited list of IPv4 routes to apply when this interface is brought up. Will be appended to <tt>ip -4 route add</tt>.<br />
;<tt>route6</tt>: Specify a semi-colon delimited list of IPv6 routes to apply when this interface is brought up. Will be appended to <tt>ip -6 route add</tt>.<br />
;<tt>mtu</tt>: Set Maximum Transmit Unit for the interface<br />
;<tt>mac_replace</tt>: Replace existing MAC address with MAC address specified in this variable.<br />
<br />
==== VLAN Variables ====<br />
<br />
VLAN support is enabled by default for all network configuration scripts. If a network script has a name in the format <tt>net.ethX.Y</tt>, then it is assumed to be a VLAN interface referencing trunk <tt>ethX</tt> and VLAN ID <tt>Y</tt>. If you desire a custom name for your VLAN interface, you can name your interface whatever you'd like and specify the following variables in your interface config:<br />
<br />
;<tt>trunk</tt>: VLAN trunk interface, e.g. &quot;net.eth0&quot;<br />
;<tt>vlan</tt>: VLAN id, e.g. &quot;32&quot;<br />
<br />
==== Bridge / Tap Variables ====<br />
<br />
The following variables for configuring a functional bridge interface with optional tap interfaces:<br />
<br />
;<tt>slaves</tt>: Set slave interfaces of this interface (for bridges, etc.) All slaves will automatically be depended upon, and will also automatically have their <tt>mtu</tt> set to that of the current interface, if an <tt>mtu</tt> is specified for the current interface. This setting is required for the <tt>bond</tt> template and optional for the <tt>bridge</tt> template.<br />
;<tt>stp</tt>: Enables Spanning Tree Protocol on a bridge interface like this &quot;stp=on&quot;<br />
;<tt>forwarding</tt>: Enables forwarding on a bridge interface by calling sysctl; as this interface does not exist when sysctl is called by init, we do it here. If this is disabled, your bridge will not forward traffic back out onto the network. useage: &quot;forwarding=1&quot;<br />
<br />
=== OpenResolv and resolv.conf ===<br />
<br />
OpenResolv will be used to set DNS information provided by the <tt>nameservers</tt>, <tt>domain</tt> and <tt>search</tt> variables when an interface is brought up. The OpenResolv framework will add entries to <tt>/etc/resolv.conf</tt>, and will also handle removing these entries when the interface is brought down. This way, <tt>/etc/resolv.conf</tt> should always contain current information and should not need to be manually edited by the system administrator. <tt>dhcpcd</tt> will use OpenResolv for updating system DNS information as well.<br />
<br />
=== Multiple Network Configurations ===<br />
<br />
For information on how to have multiple, independent network configurations, please see [[Stacked Runlevels]].<br />
<br />
=== Alternate Configs ===<br />
If you need to run the same service with different configuration parameters depending upon runlevel, then you'll be happy to know that you can specify runlevel-specific conf.d files by appending a <tt>.<br />
&lt;runlevel&gt;</tt> suffix. In this particular example, we could imagine a situation where we had two child runlevels named <tt>home</tt> and <tt>work</tt>:<br />
<br />
<pre>/etc/conf.d/net.eth0.home<br />
/etc/conf.d/net.eth0.work</pre>Note that this feature works for all init scripts, not just network configuration scripts.<br />
<br />
=== Interface Renaming ===<br />
<br />
Funtoo network scripts now support interface renaming, so you can create an interface called <tt>lan</tt> if you would like. To do this, simply specify the MAC address of the interface you would like to rename using the <tt>macaddr</tt> variable:<br />
<pre>macaddr=&quot;00:15:17:19:b6:a3&quot;</pre>If this MAC address is part of the <tt>net.lan</tt> configuration file, then when this interface starts, whatever interface currently has the MAC address of 00:15:17:19:b6:a3 (i.e. <tt>eth5</tt>) will be renamed to <tt>lan</tt> prior to the interface being brought up, and will show up in <tt>ifconfig</tt> and <tt>ip</tt> commands as being an interface named <tt>lan</tt>. It is possible to combine this with the <tt>mac_replace</tt> variable to set a new MAC address, if desired.<br />
<br />
=== Basic VLAN Configuration ===<br />
<br />
The standard <tt>interface</tt> template supports VLANs. To use VLAN support, first ensure that your kernel was compiled with VLAN support (the module name is <tt>8021q</tt>) :<br />
<br />
<console><br />
# ##i##grep CONFIG_VLAN /usr/src/linux/.config<br />
CONFIG_VLAN_8021Q=m<br />
CONFIG_VLAN_8021Q_GVRP=y<br />
</console><br />
<br />
Then, configure the trunk interface using the <tt>interface-noip</tt> template. Assuming <tt>eth1</tt> is trunked, you would create the file <tt>/etc/conf.d/net.eth1</tt> with the following contents:<br />
<br />
<pre>template=&quot;interface-noip&quot;</pre><br />
<br />
Then, create a network interface symlink for the trunk and add it to your default runlevel:<br />
<br />
<console><br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth1<br />
# ##i##rc-update add net.eth1 default<br />
</console><br />
<br />
Now, assuming you wanted to configure a VLAN of 32, you would create a config file named <tt>/etc/conf.d/net.eth1.32</tt> that looks something like this:<br />
<br />
<pre><br />
template=&quot;interface&quot;<br />
ipaddr=&quot;1.2.3.4/24&quot;<br />
gateway=&quot;1.2.3.1&quot;# etc...<br />
</pre><br />
<br />
Then, create a VLAN network interface symlink and add it to your default runlevel:<br />
<br />
<console><br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth1.32<br />
# ##i##rc-update add net.eth1.32 default<br />
</console><br />
<br />
The Funtoo network configuration scripts will automatically recognize the filename <tt>net.eth1.32</tt> as being VLAN 32 of trunk interface <tt>net.eth1</tt>.<br />
<br />
When the VLAN interface is brought up, it will be named <tt>eth1.32</tt>.<br />
<br />
=== Custom VLAN Names ===<br />
<br />
However, sometimes you may want to turn off automatic file-based VLAN naming and give your VLAN interface a custom name, such as <tt>mgmt</tt>. To do this, you would set up the trunk interface in the exact same way as described above, but instead of creating a <tt>net.eth1.32</tt> interface, you would create a <tt>net.mgmt</tt> interface, and specify <tt>vlan</tt> and <tt>trunk</tt> in the <tt>/etc/conf.d/net.mgmt</tt> config file, as follows:<br />
<br />
<pre>template=&quot;interface&quot;<br />
vlan=&quot;32&quot;<br />
trunk=&quot;net.eth1&quot;<br />
ipaddr=&quot;1.2.3.4/24&quot;<br />
gateway=&quot;1.2.3.1&quot;<br />
# etc...</pre><br />
When you specify <tt>trunk</tt> and <tt>vlan</tt> in the interface config file, filename-based auto-detecting of VLAN ID and trunk is disabled. Both <tt>trunk</tt> and <tt>vlan</tt> must be specified -- you can't specify just one.<br />
<br />
Then you would simply create a VLAN network interface symlink for <tt>net.mgmt</tt>:<br />
<br />
<console># ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.mgmt<br />
# ##i##rc-update add net.mgmt default</console><br />
When the VLAN interface is brought up, it will be named <tt>mgmt</tt>.<br />
<br />
=== Bonding Configuration ===<br />
<br />
Bonding allows you to aggregate multiple network interfaces into a single logical network interface, allowing for benefits in throughput as well as resiliency in the case that an individual interface may go down. This example shows how you would create a bonding interface (<tt>mybond</tt>) with a simple static ip setup, containing two slave devices (<tt>eth0</tt> and <tt>eth1</tt>).<br />
<br />
First, ensure that your kernel is configured to support bonding (the module name is <tt>bonding</tt>) :<br />
<br />
<console><br />
$ ##i##grep CONFIG_BONDING /usr/src/linux/.config<br />
CONFIG_BONDING=m<br />
</console><br />
<br />
You'l want to ensure that CONFIG_BONDING is set to "m" or "y". You can find this kernel configuration option tucked under "Device Drivers" -> "Network Device Support" -> "Bonding driver support".<br />
Be sure that ifenslave is emerged (this package included in Funtoo stage3):<br />
<br />
<console><br />
# ##i##emerge ifenslave<br />
</console><br />
Once bonding is enabled in the kernel, you will need to choose at least two devices to bond together. These will be set up as "slave" interfaces with no IP address.<br />
<br />
<console><br />
# ##i##cd /etc/init.d/<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##ln -s netif.tmpl net.eth1<br />
</console><br />
<br />
Then, configure the slave interfaces by creating <tt>/etc/conf.d/net.eth0</tt> and <tt>/etc/conf.d/net.eth1</tt> with the following contents:<br />
<br />
<pre><br />
template="interface-noip"<br />
</pre><br />
<br />
Now, we will create the bond interface and make <tt>net.eth0</tt> and <tt>net.eth1</tt> slaves of this interface. Note that our bond interface can have any name. To demonstrate this, we will give it the name of "mybond" below:<br />
<br />
<console><br />
# ##i##ln -s netif.tmpl net.mybond<br />
# ##i##rc-update add net.mybond default<br />
</console><br />
<br />
Now we can configure "mybond" using its configuration file <tt>/etc/conf.d/net.mybond</tt>, just as we would a regular interface, except that we specify <tt>slaves</tt>:<br />
<br />
<pre><br />
template="bond"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
slaves="net.eth0 net.eth1"<br />
</pre><br />
<br />
In a bonded configuration, it is common to set the MTU to the maximum possible value supported by hardware to maximize throughput. In order to do this, simply set the MTU option in <tt>/etc/conf.d/net.mybond</tt> to the maximum value supported by your hardware. The network scripts will ensure that this MTU setting is applied to all slave interfaces:<br />
<br />
<pre><br />
mtu=9000<br />
</pre><br />
<br />
=== Bridge Configuration ===<br />
<br />
When hosting virtual machines, it can be convenient to use a bridge setup. This example shows how you would create a bridge (br0) with a simple static ip setup, containing two slave devices (eth0, tap0).<br />
<br />
First, ensure that your kernel is configured to support bridging (the module name is <tt>bridge</tt>) :<br />
<br />
<console><br />
$ ##i##grep CONFIG_BRIDGE /usr/src/linux/.config<br />
CONFIG_BRIDGE=m<br />
CONFIG_BRIDGE_IGMP_SNOOPING=y<br />
</console><br />
<br />
Second, make sure you have the required software installed:<br />
<br />
<console><br />
# ##i##emerge -av bridge-utils usermode-utilities<br />
</console><br />
<br />
Then, create the necessary symlinks for the interfaces and add them to your default runlevel :<br />
<br />
<console><br />
# ##i##cd /etc/init.d/<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##ln -s netif.tmpl net.br0<br />
# ##i##ln -s netif.tmpl net.tap0<br />
# ##i##rc-update add net.br0 default<br />
# ##i##rc-update add net.tap0 default<br />
</console><br />
<br />
Then, configure the slave interface <tt>/etc/conf.d/net.eth0</tt> :<br />
<br />
<pre><br />
template="interface-noip"<br />
</pre><br />
<br />
Then, configure the slave interface <tt>/etc/conf.d/net.tap0</tt> - note you only require group OR user, not both :<br />
<br />
<pre><br />
template="tap"<br />
group="kvm" <br />
user="kvm"<br />
mac_replace="10:20:30:40:50:66"<br />
</pre><br />
<br />
... and the bridge interface <tt>/etc/conf.d/net.br0</tt> :<br />
<br />
<pre><br />
template="bridge"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
slaves="net.eth0 net.tap0"<br />
stp="on"<br />
forwarding=1<br />
</pre><br />
<br />
If you are using dhcpcd, you should ensure that it does not attempt to configure <tt>eth0</tt> or <tt>br0</tt> by adding the following to <tt>/etc/dhcpcd.conf</tt> :<br />
<br />
<pre><br />
# don't attempt to pull an ip address for br0 or its slave device<br />
denyinterfaces eth0 br0<br />
</pre><br />
<br />
=== More Complex Network Configuration ===<br />
<br />
If the standard templates don't work for your needs, simply create a new template -- I recommend starting from the <tt>interface</tt> template for most things:<br />
<br />
<console># ##i##cd /etc/netif.d<br />
# ##i##cp interface custom</console><br />
You can now call whatever commands you need to <tt>/etc/netif.d/custom</tt>. The following shell functions can be defined in a network script:<br />
<br />
==== netif_create ====<br />
<br />
In <tt>netif_create</tt>, you should call any commands to create the interface if it does not yet exist.<br />
<br />
==== netif_depend ====<br />
<br />
In <tt>netif_depend</tt>, you can define dependencies, using the functions <tt>need</tt> and <tt>use</tt>.<br />
<br />
==== netif_pre_up ====<br />
<br />
In <tt>netif_pre_up</tt>, you can define network configuration actions to perform prior to bringing the interface up. You can also ensure certain variables are specified by calling <tt>require var1 [var2...]</tt> here.<br />
<br />
==== netif_post_up====<br />
<br />
In <tt>netif_post_up</tt>, you can define network configuration actions to perform after bringing the interface up.<br />
<br />
==== netif_pre_down ====<br />
<br />
In <tt>netif_pre_down</tt>, you can define network configuration actions to perform prior to bringing the interface down.<br />
<br />
==== netif_post_down ====<br />
<br />
In <tt>netif_post_down</tt>, you can define network configuration actions to perform after bringing the interface down.<br />
<br />
==== netif_destroy ====<br />
<br />
In <tt>netif_destroy</tt>, you can call any commands necessary to destroy/delete the interface if it is dynamic in nature (tun/tap, etc.)<br />
<br />
==== How It Works ====<br />
<br />
You do not specify a function for actually bringing up the interface, because the template-based system does this for you. The template-based system also performs all normal actions required to bring an interface down, so you only need to specify atypical actions that must be performed - such as removing child interfaces or destroying a bridge using <tt>brctl</tt>.<br />
<br />
When you create your own network configuration template, the following capabilities are available for use automatically, as long as the appropriate variables are set in the <tt>/etc/conf.d/netif.&lt;ifname&gt;</tt> file, without requiring any explicit steps on your part:<br />
<br />
* DNS configuration using <tt>domain</tt> and <tt>nameservers</tt> config settings. OpenResolv is used automatically.<br />
* VLAN configuration using auto-naming (<tt>net.ethX.Y</tt>) or via custom naming with <tt>trunk</tt> and <tt>vlan</tt> config settings.<br />
* Default IPv4 gateway and route configuration using the <tt>gateway</tt> and <tt>route</tt> settings.<br />
* Default IPv6 gateway and route configuration using the <tt>gateway6</tt> and <tt>route6</tt> settings.<br />
* MTU configuration using the <tt>mtu</tt> setting.<br />
* Auto-depend (and auto-MTU configuration) of slave interfaces specified using <tt>slaves</tt> setting.<br />
* Renaming of existing network interface (specify MAC address using <tt>macaddr</tt> setting).<br />
<br />
To take advantage of this functionality, simply enable the appropriate variables.<br />
<br />
All other necessary network configuration and dependency behavior should be defined using the <tt>netif_</tt>-prefix functions described above.<br />
<br />
== Wireless Configuration ==<br />
<br />
The recommended approach for setting up Wi-Fi under Funtoo Linux is to use NetworkManager. Steps are provided in the [[Funtoo Linux Installation#Wi-Fi|Wi-Fi section of the Funtoo Linux Installation Guide]].<br />
<br />
== Other Network Configurations ==<br />
<br />
If you have a network configuration template that might be useful to others, please post it to the [https://bugs.funtoo.org bugtracker] so we can review it and possibly incorporate it into Funtoo.<br />
<br />
== License ==<br />
<br />
Funtoo Linux networking scripts are released under the following license:<br />
<br />
{{BSD2 Funtoo|src=http://github.com/funtoo/corenetwork}}<br />
<br />
[[Category:HOWTO]]<br />
[[Category:Projects]]<br />
[[Category:Networking]]<br />
[[Category:Install]]<br />
[[Category:Funtoo features]]<br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Funtoo:Networking/Configuration&diff=26549Funtoo:Networking/Configuration2019-01-08T17:27:28Z<p>Oleg: /* Server Network Configuration */</p>
<hr />
<div><blockquote>This document explains how to configure your network settings by explaining the network configuration functionality available in Funtoo Linux. Also covered is <tt>dhcpcd 5.x</tt>, Wi-Fi (IEEE 802.11) configuration, and the OpenResolv framework.<br />
</blockquote><br />
<br />
== Introduction ==<br />
__TOC__<br />
Funtoo Linux has its own core network configuration system that differs somewhat from upstream network configuration systems used in [http://www.gentoo.org Gentoo Linux] and [http://roy.marples.name/projects/openrc OpenRC].<br />
<br />
In this document, I will explain the unique additions and changes to the Funtoo network configuration and show you how to use this system to configure your network.<br />
<br />
I'll also explain how to use <code>dhcpcd</code> for managing network interfaces on DHCP-based networks, and will also cover OpenRC stacked runlevel configuration, ''Wi-Fi'' (IEEE 802.11) configuration, and the OpenResolv framework, which is enabled in Funtoo Linux by default.<br />
<br />
== A Gentle Introduction to Funtoo Network Configuration ==<br />
<br />
Before I get into the technical details of configuring your network, it's important to understand that Funtoo Linux has a number of different options available to you for network configuration, with more likely to be added in the future. Each approach is different and has its own strengths and weaknesses, and this is, in my opinion, a good thing.<br />
<br />
=== The Easy (Dynamic) Way ===<br />
<br />
When configuring your network, one option is to skip traditional network configuration and simply rely on DHCP. This is by far the simplest method of configuring your network. If you are on a wired network, no other steps are typically required beyond enabling a DHCP client, and Funtoo Linux includes {{c|dhcpcd 7.x}} by default. <br />
<br />
==== Network Manager, Wicd ====<br />
<br />
If you are going to use a third party package such as [[Network Manager]] or [[Wicd]] to manage your network then you do not need to configure DHCP at all. These packages configure DHCP for you. Simply emerge the package you want to use and start using it.<br />
<br />
==== DHCP-Only Systems ====<br />
<br />
If you are not planning to use a third-party package to manage your network interfaces, it is still extremely easy to set up DHCP networking, especially if you always use DHCP to connect to networks, which is common for desktops or laptops. In this scenario, we can simply enable {{c|dhcpcd}} to run at system startup. It will run in the background and automatically look for DHCP servers on all your network interfaces, and will attempt to lease an IP address from any DHCP servers found. <br />
<br />
If this sounds like what you want to do, then add {{c|dhcpcd}} to your default runlevel as follows:<br />
<br />
<console># ##i##rc-update add dhcpcd default</console><br />
<br />
To enable DHCP immediately, you would follow the previous command with an {{c|openrc}} command, which would start the {{c|dhcpcd}} client you just added:<br />
<br />
{{console|body=<br />
###i## openrc<br />
}}<br />
<br />
If you're on a wired network and have the necessary drivers in your kernel, then this should get you going. For wireless networks, more steps are required to utilize your wireless hardware to associate with an access point, which will be covered later in this document. <br />
<br />
===== Tweaking Dhcpcd =====<br />
<br />
For now, it's important to note that {{c|dhcpcd 7.x}} will manage ''all'' available network interfaces by default. If you want to run a DHCP client on ''all but one'' interface, or some other subset of interfaces, you can add the appropriate {{c|denyinterfaces}} or {{c|allowinterfaces}} [[glob pattern]] to {{c|/etc/dhcpcd.conf}}:<br />
<br />
{{file|name=/etc/dhcpcd.conf|lang=bash|body=<br />
# manage all interfaces but eth0 with dhcpcd<br />
denyinterfaces eth0<br />
}}<br />
<br />
This can also be accomplished by modifying {{c|tc/init.d/dhcpcd}} directly and adding {{c|-Z ''ifglob''}} or {{c|-z ''ifglob''}} (the equivalent command-line parameters) to {{c|command_args}}.<br />
<br />
==== Using Funtoo Scripts for DHCP ====<br />
<br />
You can also use the Funtoo Linux networking scripts to start a DHCP client just on a specific interface. This approach is best if you are planning to also do some advanced bridging, bonding or VLAN configuration on your machine along with DHCP, since you will be using the Funtoo Linux networking scripts for that too. <br />
<br />
To use this variant approach, ''don't'' enable {{c|/etc/init.d/dhcpcd}} directly. Instead, use the Funtoo Linux {{c|dhcpcd}} template which will start dhcpcd on only one interface. Below, you will see the steps to do this. This is very similar to how we set up advanced network interfaces, which will be covered later in this documentation:<br />
<br />
{{console|body=<br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##rc-update add net.eth0 default<br />
# ##i##echo template=dhcpcd > /etc/conf.d/net.eth0<br />
# ##i##openrc<br />
}}<br />
<br />
The last command, {{c|openrc}} causes {{c|net.eth0}} to be started.<br />
<br />
=== Server Network Configuration ===<br />
<br />
For servers and advanced networking scenarios, Funtoo Linux offers its own modular, template-based network configuration system. This system offers a lot of flexibility for configuring network interfaces, essentially serving as a &quot;network interface construction kit.&quot; This system can be used by itself, or even combined with <tt>dhcpcd</tt>, as shown in the previous section.<br />
<br />
Here are the key components of the template-based network configuration system:<br />
<br />
;{{c|/etc/init.d/loopback}}: An init script that configures the localhost interface. This script is always enabled and is part of the boot process.<br />
;{{c|/etc/netif.d}}: This is a directory that contains various network configuration templates. Each of these templates is focused on configuring a particular type of network interface, such as a general static IP-based interface, a bridge interface, a bond interface, etc.<br />
;{{c|/etc/init.d/netif.tmpl}}: This is the master init script for the template-based network configuration system. New interfaces are added to your system by creating '''symbolic links''' to this file in {{c|/etc/init.d}}.<br />
<br />
So, if you wanted to use this system to configure {{c|eth0}} with a static IP address, you would create a {{c|net.eth0}} symlink to {{c|netif.tmpl}} as follows:<br />
<br />
{{console|body=<br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth0<br />
}}<br />
<br />
Then, you would create an {{c|/etc/conf.d/net.eth0}} configuration file that would specify which template to use from the {{c|/etc/netif.d}} directory:<br />
<br />
{{file|name=/etc/conf.d/net.eth0|lang=bash|body=<br />
template="interface"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
}}<br />
<br />
To complete our static IP network configuration we would need to:<br />
<br />
{{console|body=# ##i##rc-update add net.eth0 default}}<br />
When configuring your own static network interface, one of {{c|ipaddr}} or {{c|ipaddrs}} is required and should specify the IP address(es) to configure for this interface, in &quot;a.b.c.d/netmask&quot; format. Optional parameters include {{c|gateway}}, which defines a default gateway for your entire network, and if set should specify the gateway's IP address. In addition, {{c|domain}} and {{c|nameservers}} (space-separated if more than one) can be used to specify DNS information for this interface.<br />
<br />
=== Configuration Variables ===<br />
<br />
==== Interface Variables ====<br />
<br />
The <tt>ipaddr</tt> and <tt>ipaddrs</tt> variables are supported by the <tt>interface</tt> and <tt>bridge</tt> templates, and are used to specify a single or multiple IPv4 or IPv6 address(es) for the interface. IP addresses should be specified in 'IP/netmask' format, such as <tt>10.0.0.1/24</tt>. Multiple IP addresses can be specified delimited by whitespace:<br />
<br />
<pre>ipaddrs=&quot;10.0.0.1/24 10.0.0.2/24&quot;</pre><br />
<br />
===== Broadcast Address =====<br />
<br />
By default, a broadcast address will be calculated based on the IP address and network mask. If you need to manually specify a broadcast address, use the following format for your IP address:<br />
<br />
<pre><br />
ipaddrs="10.0.0.1/24;broadcast=10.0.1.255 10.0.0.2/24"<br />
</pre><br />
<br />
===== Not Specifying An Address =====<br />
<br />
Note that in some cases, you may choose to '''not''' specify <tt>ipaddr</tt> or <tt>ipaddrs</tt> for a <tt>bridge</tt> template. That is allowed. If you don't want to specify an IP address for a regular interface, you can choose to use the <tt>interface</tt> template without an IP address specified in the config, or use the <tt>interface-noip</tt> template instead, for the sake of clarity.<br />
<br />
===== Viewing All Configured IP Addresses =====<br />
<br />
Also note that if you specify multiple IPv4 addresses, <tt>ifconfig</tt> will only show the first IP address. To view all IP addresses associated with the interface, use the <tt>ip addr show</tt> command.<br />
<br />
=== General Variables ===<br />
<br />
The following variables are enabled by default for all network scripts, and if specified will trigger a corresponding configuration action:<br />
<br />
;<tt>nameservers</tt>: Set DNS nameservers using OpenResolv. Specify multiple IPv4 or IPv6 nameservers like this: &quot;1.2.3.4 1.2.3.5 1.2.3.6&quot;. Please note that OpenResolv treats <tt>127.0.0.1</tt> specially, and it indicates that you are running a local name resolver like <tt>dnsmasq</tt> or <tt>bind</tt>. OpenResolv will ignore all other name servers specified alongside <tt>127.0.0.1</tt>. See <tt>man resolvconf</tt> and <tt>man resolvconf.conf</tt> for additional setup information.<br />
;<tt>search</tt>: Set DNS search information using OpenResolv.<br />
;<tt>domain</tt>: Set DNS domain using OpenResolv.<br />
;<tt>gateway</tt>: Define a default IPv4 gateway on this interface.<br />
;<tt>gateway6</tt>: Define a default IPv6 gateway on this interface.<br />
;<tt>route</tt>: Specify a semi-colon delimited list of IPv4 routes to apply when this interface is brought up. Will be appended to <tt>ip -4 route add</tt>.<br />
;<tt>route6</tt>: Specify a semi-colon delimited list of IPv6 routes to apply when this interface is brought up. Will be appended to <tt>ip -6 route add</tt>.<br />
;<tt>mtu</tt>: Set Maximum Transmit Unit for the interface<br />
;<tt>mac_replace</tt>: Replace existing MAC address with MAC address specified in this variable.<br />
<br />
==== VLAN Variables ====<br />
<br />
VLAN support is enabled by default for all network configuration scripts. If a network script has a name in the format <tt>net.ethX.Y</tt>, then it is assumed to be a VLAN interface referencing trunk <tt>ethX</tt> and VLAN ID <tt>Y</tt>. If you desire a custom name for your VLAN interface, you can name your interface whatever you'd like and specify the following variables in your interface config:<br />
<br />
;<tt>trunk</tt>: VLAN trunk interface, e.g. &quot;net.eth0&quot;<br />
;<tt>vlan</tt>: VLAN id, e.g. &quot;32&quot;<br />
<br />
==== Bridge / Tap Variables ====<br />
<br />
The following variables for configuring a functional bridge interface with optional tap interfaces:<br />
<br />
;<tt>slaves</tt>: Set slave interfaces of this interface (for bridges, etc.) All slaves will automatically be depended upon, and will also automatically have their <tt>mtu</tt> set to that of the current interface, if an <tt>mtu</tt> is specified for the current interface. This setting is required for the <tt>bond</tt> template and optional for the <tt>bridge</tt> template.<br />
;<tt>stp</tt>: Enables Spanning Tree Protocol on a bridge interface like this &quot;stp=on&quot;<br />
;<tt>forwarding</tt>: Enables forwarding on a bridge interface by calling sysctl; as this interface does not exist when sysctl is called by init, we do it here. If this is disabled, your bridge will not forward traffic back out onto the network. useage: &quot;forwarding=1&quot;<br />
<br />
=== OpenResolv and resolv.conf ===<br />
<br />
OpenResolv will be used to set DNS information provided by the <tt>nameservers</tt>, <tt>domain</tt> and <tt>search</tt> variables when an interface is brought up. The OpenResolv framework will add entries to <tt>/etc/resolv.conf</tt>, and will also handle removing these entries when the interface is brought down. This way, <tt>/etc/resolv.conf</tt> should always contain current information and should not need to be manually edited by the system administrator. <tt>dhcpcd</tt> will use OpenResolv for updating system DNS information as well.<br />
<br />
=== Multiple Network Configurations ===<br />
<br />
For information on how to have multiple, independent network configurations, please see [[Stacked Runlevels]].<br />
<br />
=== Alternate Configs ===<br />
If you need to run the same service with different configuration parameters depending upon runlevel, then you'll be happy to know that you can specify runlevel-specific conf.d files by appending a <tt>.<br />
&lt;runlevel&gt;</tt> suffix. In this particular example, we could imagine a situation where we had two child runlevels named <tt>home</tt> and <tt>work</tt>:<br />
<br />
<pre>/etc/conf.d/net.eth0.home<br />
/etc/conf.d/net.eth0.work</pre>Note that this feature works for all init scripts, not just network configuration scripts.<br />
<br />
=== Interface Renaming ===<br />
<br />
Funtoo network scripts now support interface renaming, so you can create an interface called <tt>lan</tt> if you would like. To do this, simply specify the MAC address of the interface you would like to rename using the <tt>macaddr</tt> variable:<br />
<pre>macaddr=&quot;00:15:17:19:b6:a3&quot;</pre>If this MAC address is part of the <tt>net.lan</tt> configuration file, then when this interface starts, whatever interface currently has the MAC address of 00:15:17:19:b6:a3 (i.e. <tt>eth5</tt>) will be renamed to <tt>lan</tt> prior to the interface being brought up, and will show up in <tt>ifconfig</tt> and <tt>ip</tt> commands as being an interface named <tt>lan</tt>. It is possible to combine this with the <tt>mac_replace</tt> variable to set a new MAC address, if desired.<br />
<br />
=== Basic VLAN Configuration ===<br />
<br />
The standard <tt>interface</tt> template supports VLANs. To use VLAN support, first ensure that your kernel was compiled with VLAN support (the module name is <tt>8021q</tt>) :<br />
<br />
<console><br />
# ##i##grep CONFIG_VLAN /usr/src/linux/.config<br />
CONFIG_VLAN_8021Q=m<br />
CONFIG_VLAN_8021Q_GVRP=y<br />
</console><br />
<br />
Then, configure the trunk interface using the <tt>interface-noip</tt> template. Assuming <tt>eth1</tt> is trunked, you would create the file <tt>/etc/conf.d/net.eth1</tt> with the following contents:<br />
<br />
<pre>template=&quot;interface-noip&quot;</pre><br />
<br />
Then, create a network interface symlink for the trunk and add it to your default runlevel:<br />
<br />
<console><br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth1<br />
# ##i##rc-update add net.eth1 default<br />
</console><br />
<br />
Now, assuming you wanted to configure a VLAN of 32, you would create a config file named <tt>/etc/conf.d/net.eth1.32</tt> that looks something like this:<br />
<br />
<pre><br />
template=&quot;interface&quot;<br />
ipaddr=&quot;1.2.3.4/24&quot;<br />
gateway=&quot;1.2.3.1&quot;# etc...<br />
</pre><br />
<br />
Then, create a VLAN network interface symlink and add it to your default runlevel:<br />
<br />
<console><br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth1.32<br />
# ##i##rc-update add net.eth1.32 default<br />
</console><br />
<br />
The Funtoo network configuration scripts will automatically recognize the filename <tt>net.eth1.32</tt> as being VLAN 32 of trunk interface <tt>net.eth1</tt>.<br />
<br />
When the VLAN interface is brought up, it will be named <tt>eth1.32</tt>.<br />
<br />
=== Custom VLAN Names ===<br />
<br />
However, sometimes you may want to turn off automatic file-based VLAN naming and give your VLAN interface a custom name, such as <tt>mgmt</tt>. To do this, you would set up the trunk interface in the exact same way as described above, but instead of creating a <tt>net.eth1.32</tt> interface, you would create a <tt>net.mgmt</tt> interface, and specify <tt>vlan</tt> and <tt>trunk</tt> in the <tt>/etc/conf.d/net.mgmt</tt> config file, as follows:<br />
<br />
<pre>template=&quot;interface&quot;<br />
vlan=&quot;32&quot;<br />
trunk=&quot;net.eth1&quot;<br />
ipaddr=&quot;1.2.3.4/24&quot;<br />
gateway=&quot;1.2.3.1&quot;<br />
# etc...</pre><br />
When you specify <tt>trunk</tt> and <tt>vlan</tt> in the interface config file, filename-based auto-detecting of VLAN ID and trunk is disabled. Both <tt>trunk</tt> and <tt>vlan</tt> must be specified -- you can't specify just one.<br />
<br />
Then you would simply create a VLAN network interface symlink for <tt>net.mgmt</tt>:<br />
<br />
<console># ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.mgmt<br />
# ##i##rc-update add net.mgmt default</console><br />
When the VLAN interface is brought up, it will be named <tt>mgmt</tt>.<br />
<br />
=== Bonding Configuration ===<br />
<br />
Bonding allows you to aggregate multiple network interfaces into a single logical network interface, allowing for benefits in throughput as well as resiliency in the case that an individual interface may go down. This example shows how you would create a bonding interface (<tt>mybond</tt>) with a simple static ip setup, containing two slave devices (<tt>eth0</tt> and <tt>eth1</tt>).<br />
<br />
First, ensure that your kernel is configured to support bonding (the module name is <tt>bonding</tt>) :<br />
<br />
<console><br />
$ ##i##grep CONFIG_BONDING /usr/src/linux/.config<br />
CONFIG_BONDING=m<br />
</console><br />
<br />
You'l want to ensure that CONFIG_BONDING is set to "m" or "y". You can find this kernel configuration option tucked under "Device Drivers" -> "Network Device Support" -> "Bonding driver support".<br />
Be sure that ifenslave is emerged (this package included in Funtoo stage3):<br />
<br />
<console><br />
# ##i##emerge ifenslave<br />
</console><br />
Once bonding is enabled in the kernel, you will need to choose at least two devices to bond together. These will be set up as "slave" interfaces with no IP address.<br />
<br />
<console><br />
# ##i##cd /etc/init.d/<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##ln -s netif.tmpl net.eth1<br />
</console><br />
<br />
Then, configure the slave interfaces by creating <tt>/etc/conf.d/net.eth0</tt> and <tt>/etc/conf.d/net.eth1</tt> with the following contents:<br />
<br />
<pre><br />
template="interface-noip"<br />
</pre><br />
<br />
Now, we will create the bond interface and make <tt>net.eth0</tt> and <tt>net.eth1</tt> slaves of this interface. Note that our bond interface can have any name. To demonstrate this, we will give it the name of "mybond" below:<br />
<br />
<console><br />
# ##i##ln -s netif.tmpl net.mybond<br />
# ##i##rc-update add net.mybond default<br />
</console><br />
<br />
Now we can configure "mybond" using its configuration file <tt>/etc/conf.d/net.mybond</tt>, just as we would a regular interface, except that we specify <tt>slaves</tt>:<br />
<br />
<pre><br />
template="bond"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
slaves="net.eth0 net.eth1"<br />
</pre><br />
<br />
In a bonded configuration, it is common to set the MTU to the maximum possible value supported by hardware to maximize throughput. In order to do this, simply set the MTU option in <tt>/etc/conf.d/net.mybond</tt> to the maximum value supported by your hardware. The network scripts will ensure that this MTU setting is applied to all slave interfaces:<br />
<br />
<pre><br />
mtu=9000<br />
</pre><br />
<br />
=== Bridge Configuration ===<br />
<br />
When hosting virtual machines, it can be convenient to use a bridge setup. This example shows how you would create a bridge (br0) with a simple static ip setup, containing two slave devices (eth0, tap0).<br />
<br />
First, ensure that your kernel is configured to support bridging (the module name is <tt>bridge</tt>) :<br />
<br />
<console><br />
$ ##i##grep CONFIG_BRIDGE /usr/src/linux/.config<br />
CONFIG_BRIDGE=m<br />
CONFIG_BRIDGE_IGMP_SNOOPING=y<br />
</console><br />
<br />
Second, make sure you have the required software installed:<br />
<br />
<console><br />
# ##i##emerge -av bridge-utils usermode-utilities<br />
</console><br />
<br />
Then, create the necessary symlinks for the interfaces and add them to your default runlevel :<br />
<br />
<console><br />
# ##i##cd /etc/init.d/<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##ln -s netif.tmpl net.br0<br />
# ##i##ln -s netif.tmpl net.tap0<br />
# ##i##rc-update add net.br0 default<br />
# ##i##rc-update add net.tap0 default<br />
</console><br />
<br />
Then, configure the slave interface <tt>/etc/conf.d/net.eth0</tt> :<br />
<br />
<pre><br />
template="interface-noip"<br />
</pre><br />
<br />
Then, configure the slave interface <tt>/etc/conf.d/net.tap0</tt> - note you only require group OR user, not both :<br />
<br />
<pre><br />
template="tap"<br />
group="kvm" <br />
user="kvm"<br />
mac_replace="10:20:30:40:50:66"<br />
</pre><br />
<br />
... and the bridge interface <tt>/etc/conf.d/net.br0</tt> :<br />
<br />
<pre><br />
template="bridge"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
slaves="net.eth0 net.tap0"<br />
stp="on"<br />
forwarding=1<br />
</pre><br />
<br />
If you are using dhcpcd, you should ensure that it does not attempt to configure <tt>eth0</tt> or <tt>br0</tt> by adding the following to <tt>/etc/dhcpcd.conf</tt> :<br />
<br />
<pre><br />
# don't attempt to pull an ip address for br0 or its slave device<br />
denyinterfaces eth0 br0<br />
</pre><br />
<br />
=== More Complex Network Configuration ===<br />
<br />
If the standard templates don't work for your needs, simply create a new template -- I recommend starting from the <tt>interface</tt> template for most things:<br />
<br />
<console># ##i##cd /etc/netif.d<br />
# ##i##cp interface custom</console><br />
You can now call whatever commands you need to <tt>/etc/netif.d/custom</tt>. The following shell functions can be defined in a network script:<br />
<br />
==== netif_create ====<br />
<br />
In <tt>netif_create</tt>, you should call any commands to create the interface if it does not yet exist.<br />
<br />
==== netif_depend ====<br />
<br />
In <tt>netif_depend</tt>, you can define dependencies, using the functions <tt>need</tt> and <tt>use</tt>.<br />
<br />
==== netif_pre_up ====<br />
<br />
In <tt>netif_pre_up</tt>, you can define network configuration actions to perform prior to bringing the interface up. You can also ensure certain variables are specified by calling <tt>require var1 [var2...]</tt> here.<br />
<br />
==== netif_post_up====<br />
<br />
In <tt>netif_post_up</tt>, you can define network configuration actions to perform after bringing the interface up.<br />
<br />
==== netif_pre_down ====<br />
<br />
In <tt>netif_pre_down</tt>, you can define network configuration actions to perform prior to bringing the interface down.<br />
<br />
==== netif_post_down ====<br />
<br />
In <tt>netif_post_down</tt>, you can define network configuration actions to perform after bringing the interface down.<br />
<br />
==== netif_destroy ====<br />
<br />
In <tt>netif_destroy</tt>, you can call any commands necessary to destroy/delete the interface if it is dynamic in nature (tun/tap, etc.)<br />
<br />
==== How It Works ====<br />
<br />
You do not specify a function for actually bringing up the interface, because the template-based system does this for you. The template-based system also performs all normal actions required to bring an interface down, so you only need to specify atypical actions that must be performed - such as removing child interfaces or destroying a bridge using <tt>brctl</tt>.<br />
<br />
When you create your own network configuration template, the following capabilities are available for use automatically, as long as the appropriate variables are set in the <tt>/etc/conf.d/netif.&lt;ifname&gt;</tt> file, without requiring any explicit steps on your part:<br />
<br />
* DNS configuration using <tt>domain</tt> and <tt>nameservers</tt> config settings. OpenResolv is used automatically.<br />
* VLAN configuration using auto-naming (<tt>net.ethX.Y</tt>) or via custom naming with <tt>trunk</tt> and <tt>vlan</tt> config settings.<br />
* Default IPv4 gateway and route configuration using the <tt>gateway</tt> and <tt>route</tt> settings.<br />
* Default IPv6 gateway and route configuration using the <tt>gateway6</tt> and <tt>route6</tt> settings.<br />
* MTU configuration using the <tt>mtu</tt> setting.<br />
* Auto-depend (and auto-MTU configuration) of slave interfaces specified using <tt>slaves</tt> setting.<br />
* Renaming of existing network interface (specify MAC address using <tt>macaddr</tt> setting).<br />
<br />
To take advantage of this functionality, simply enable the appropriate variables.<br />
<br />
All other necessary network configuration and dependency behavior should be defined using the <tt>netif_</tt>-prefix functions described above.<br />
<br />
== Wireless Configuration ==<br />
<br />
The recommended approach for setting up Wi-Fi under Funtoo Linux is to use NetworkManager. Steps are provided in the [[Funtoo Linux Installation#Wi-Fi|Wi-Fi section of the Funtoo Linux Installation Guide]].<br />
<br />
== Other Network Configurations ==<br />
<br />
If you have a network configuration template that might be useful to others, please post it to the [https://bugs.funtoo.org bugtracker] so we can review it and possibly incorporate it into Funtoo.<br />
<br />
== License ==<br />
<br />
Funtoo Linux networking scripts are released under the following license:<br />
<br />
{{BSD2 Funtoo|src=http://github.com/funtoo/corenetwork}}<br />
<br />
[[Category:HOWTO]]<br />
[[Category:Projects]]<br />
[[Category:Networking]]<br />
[[Category:Install]]<br />
[[Category:Funtoo features]]<br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Funtoo:Networking/Configuration&diff=26548Funtoo:Networking/Configuration2019-01-08T17:24:49Z<p>Oleg: /* Using Funtoo Scripts for DHCP */</p>
<hr />
<div><blockquote>This document explains how to configure your network settings by explaining the network configuration functionality available in Funtoo Linux. Also covered is <tt>dhcpcd 5.x</tt>, Wi-Fi (IEEE 802.11) configuration, and the OpenResolv framework.<br />
</blockquote><br />
<br />
== Introduction ==<br />
__TOC__<br />
Funtoo Linux has its own core network configuration system that differs somewhat from upstream network configuration systems used in [http://www.gentoo.org Gentoo Linux] and [http://roy.marples.name/projects/openrc OpenRC].<br />
<br />
In this document, I will explain the unique additions and changes to the Funtoo network configuration and show you how to use this system to configure your network.<br />
<br />
I'll also explain how to use <code>dhcpcd</code> for managing network interfaces on DHCP-based networks, and will also cover OpenRC stacked runlevel configuration, ''Wi-Fi'' (IEEE 802.11) configuration, and the OpenResolv framework, which is enabled in Funtoo Linux by default.<br />
<br />
== A Gentle Introduction to Funtoo Network Configuration ==<br />
<br />
Before I get into the technical details of configuring your network, it's important to understand that Funtoo Linux has a number of different options available to you for network configuration, with more likely to be added in the future. Each approach is different and has its own strengths and weaknesses, and this is, in my opinion, a good thing.<br />
<br />
=== The Easy (Dynamic) Way ===<br />
<br />
When configuring your network, one option is to skip traditional network configuration and simply rely on DHCP. This is by far the simplest method of configuring your network. If you are on a wired network, no other steps are typically required beyond enabling a DHCP client, and Funtoo Linux includes {{c|dhcpcd 7.x}} by default. <br />
<br />
==== Network Manager, Wicd ====<br />
<br />
If you are going to use a third party package such as [[Network Manager]] or [[Wicd]] to manage your network then you do not need to configure DHCP at all. These packages configure DHCP for you. Simply emerge the package you want to use and start using it.<br />
<br />
==== DHCP-Only Systems ====<br />
<br />
If you are not planning to use a third-party package to manage your network interfaces, it is still extremely easy to set up DHCP networking, especially if you always use DHCP to connect to networks, which is common for desktops or laptops. In this scenario, we can simply enable {{c|dhcpcd}} to run at system startup. It will run in the background and automatically look for DHCP servers on all your network interfaces, and will attempt to lease an IP address from any DHCP servers found. <br />
<br />
If this sounds like what you want to do, then add {{c|dhcpcd}} to your default runlevel as follows:<br />
<br />
<console># ##i##rc-update add dhcpcd default</console><br />
<br />
To enable DHCP immediately, you would follow the previous command with an {{c|openrc}} command, which would start the {{c|dhcpcd}} client you just added:<br />
<br />
{{console|body=<br />
###i## openrc<br />
}}<br />
<br />
If you're on a wired network and have the necessary drivers in your kernel, then this should get you going. For wireless networks, more steps are required to utilize your wireless hardware to associate with an access point, which will be covered later in this document. <br />
<br />
===== Tweaking Dhcpcd =====<br />
<br />
For now, it's important to note that {{c|dhcpcd 7.x}} will manage ''all'' available network interfaces by default. If you want to run a DHCP client on ''all but one'' interface, or some other subset of interfaces, you can add the appropriate {{c|denyinterfaces}} or {{c|allowinterfaces}} [[glob pattern]] to {{c|/etc/dhcpcd.conf}}:<br />
<br />
{{file|name=/etc/dhcpcd.conf|lang=bash|body=<br />
# manage all interfaces but eth0 with dhcpcd<br />
denyinterfaces eth0<br />
}}<br />
<br />
This can also be accomplished by modifying {{c|tc/init.d/dhcpcd}} directly and adding {{c|-Z ''ifglob''}} or {{c|-z ''ifglob''}} (the equivalent command-line parameters) to {{c|command_args}}.<br />
<br />
==== Using Funtoo Scripts for DHCP ====<br />
<br />
You can also use the Funtoo Linux networking scripts to start a DHCP client just on a specific interface. This approach is best if you are planning to also do some advanced bridging, bonding or VLAN configuration on your machine along with DHCP, since you will be using the Funtoo Linux networking scripts for that too. <br />
<br />
To use this variant approach, ''don't'' enable {{c|/etc/init.d/dhcpcd}} directly. Instead, use the Funtoo Linux {{c|dhcpcd}} template which will start dhcpcd on only one interface. Below, you will see the steps to do this. This is very similar to how we set up advanced network interfaces, which will be covered later in this documentation:<br />
<br />
{{console|body=<br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##rc-update add net.eth0 default<br />
# ##i##echo template=dhcpcd > /etc/conf.d/net.eth0<br />
# ##i##openrc<br />
}}<br />
<br />
The last command, {{c|openrc}} causes {{c|net.eth0}} to be started.<br />
<br />
=== Server Network Configuration ===<br />
<br />
For servers and advanced networking scenarios, Funtoo Linux offers its own modular, template-based network configuration system. This system offers a lot of flexibility for configuring network interfaces, essentially serving as a &quot;network interface construction kit.&quot; This system can be used by itself, or even combined with <tt>dhcpcd</tt>, as shown in the previous section.<br />
<br />
Here are the key components of the template-based network configuration system:<br />
<br />
;{{c|/etc/init.d/loopback}}: An init script that configures the localhost interface. This script is always enabled and is part of the boot process.<br />
;{{c|/etc/netif.d}}: This is a directory that contains various network configuration templates. Each of these templates is focused on configuring a particular type of network interface, such as a general static IP-based interface, a bridge interface, a bond interface, etc.<br />
;{{c|/etc/init.d/netif.tmpl}}: This is the master init script for the template-based network configuration system. New interfaces are added to your system by creating '''symbolic links''' to this file in {{c|/etc/init.d}}.<br />
<br />
So, if you wanted to use this system to configure {{c|eth0}} with a static IP address, you would create a {{c|net.eth0}} symlink to {{c|netif.tmpl}} as follows:<br />
<br />
{{console|body=<br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth0<br />
}}<br />
<br />
Then, you would create an {{c|/etc/conf.d/net.eth0}} configuration file that would specify which template to use from the {{c|/etc/netif.d}} directory:<br />
<br />
{{file|name=/etc/conf.d/neteth0|lang=bash|body=<br />
template="interface"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
}}<br />
<br />
To complete our static IP network configuration we would need to:<br />
<br />
<console># ##i##rc-update add net.eth0 default</console><br />
When configuring your own static network interface, one of <tt>ipaddr</tt> or <tt>ipaddrs</tt> is required and should specify the IP address(es) to configure for this interface, in &quot;a.b.c.d/netmask&quot; format. Optional parameters include <tt>gateway</tt>, which defines a default gateway for your entire network, and if set should specify the gateway's IP address. In addition, <tt>domain</tt> and <tt>nameservers</tt> (space-separated if more than one) can be used to specify DNS information for this interface.<br />
<br />
=== Configuration Variables ===<br />
<br />
==== Interface Variables ====<br />
<br />
The <tt>ipaddr</tt> and <tt>ipaddrs</tt> variables are supported by the <tt>interface</tt> and <tt>bridge</tt> templates, and are used to specify a single or multiple IPv4 or IPv6 address(es) for the interface. IP addresses should be specified in 'IP/netmask' format, such as <tt>10.0.0.1/24</tt>. Multiple IP addresses can be specified delimited by whitespace:<br />
<br />
<pre>ipaddrs=&quot;10.0.0.1/24 10.0.0.2/24&quot;</pre><br />
<br />
===== Broadcast Address =====<br />
<br />
By default, a broadcast address will be calculated based on the IP address and network mask. If you need to manually specify a broadcast address, use the following format for your IP address:<br />
<br />
<pre><br />
ipaddrs="10.0.0.1/24;broadcast=10.0.1.255 10.0.0.2/24"<br />
</pre><br />
<br />
===== Not Specifying An Address =====<br />
<br />
Note that in some cases, you may choose to '''not''' specify <tt>ipaddr</tt> or <tt>ipaddrs</tt> for a <tt>bridge</tt> template. That is allowed. If you don't want to specify an IP address for a regular interface, you can choose to use the <tt>interface</tt> template without an IP address specified in the config, or use the <tt>interface-noip</tt> template instead, for the sake of clarity.<br />
<br />
===== Viewing All Configured IP Addresses =====<br />
<br />
Also note that if you specify multiple IPv4 addresses, <tt>ifconfig</tt> will only show the first IP address. To view all IP addresses associated with the interface, use the <tt>ip addr show</tt> command.<br />
<br />
=== General Variables ===<br />
<br />
The following variables are enabled by default for all network scripts, and if specified will trigger a corresponding configuration action:<br />
<br />
;<tt>nameservers</tt>: Set DNS nameservers using OpenResolv. Specify multiple IPv4 or IPv6 nameservers like this: &quot;1.2.3.4 1.2.3.5 1.2.3.6&quot;. Please note that OpenResolv treats <tt>127.0.0.1</tt> specially, and it indicates that you are running a local name resolver like <tt>dnsmasq</tt> or <tt>bind</tt>. OpenResolv will ignore all other name servers specified alongside <tt>127.0.0.1</tt>. See <tt>man resolvconf</tt> and <tt>man resolvconf.conf</tt> for additional setup information.<br />
;<tt>search</tt>: Set DNS search information using OpenResolv.<br />
;<tt>domain</tt>: Set DNS domain using OpenResolv.<br />
;<tt>gateway</tt>: Define a default IPv4 gateway on this interface.<br />
;<tt>gateway6</tt>: Define a default IPv6 gateway on this interface.<br />
;<tt>route</tt>: Specify a semi-colon delimited list of IPv4 routes to apply when this interface is brought up. Will be appended to <tt>ip -4 route add</tt>.<br />
;<tt>route6</tt>: Specify a semi-colon delimited list of IPv6 routes to apply when this interface is brought up. Will be appended to <tt>ip -6 route add</tt>.<br />
;<tt>mtu</tt>: Set Maximum Transmit Unit for the interface<br />
;<tt>mac_replace</tt>: Replace existing MAC address with MAC address specified in this variable.<br />
<br />
==== VLAN Variables ====<br />
<br />
VLAN support is enabled by default for all network configuration scripts. If a network script has a name in the format <tt>net.ethX.Y</tt>, then it is assumed to be a VLAN interface referencing trunk <tt>ethX</tt> and VLAN ID <tt>Y</tt>. If you desire a custom name for your VLAN interface, you can name your interface whatever you'd like and specify the following variables in your interface config:<br />
<br />
;<tt>trunk</tt>: VLAN trunk interface, e.g. &quot;net.eth0&quot;<br />
;<tt>vlan</tt>: VLAN id, e.g. &quot;32&quot;<br />
<br />
==== Bridge / Tap Variables ====<br />
<br />
The following variables for configuring a functional bridge interface with optional tap interfaces:<br />
<br />
;<tt>slaves</tt>: Set slave interfaces of this interface (for bridges, etc.) All slaves will automatically be depended upon, and will also automatically have their <tt>mtu</tt> set to that of the current interface, if an <tt>mtu</tt> is specified for the current interface. This setting is required for the <tt>bond</tt> template and optional for the <tt>bridge</tt> template.<br />
;<tt>stp</tt>: Enables Spanning Tree Protocol on a bridge interface like this &quot;stp=on&quot;<br />
;<tt>forwarding</tt>: Enables forwarding on a bridge interface by calling sysctl; as this interface does not exist when sysctl is called by init, we do it here. If this is disabled, your bridge will not forward traffic back out onto the network. useage: &quot;forwarding=1&quot;<br />
<br />
=== OpenResolv and resolv.conf ===<br />
<br />
OpenResolv will be used to set DNS information provided by the <tt>nameservers</tt>, <tt>domain</tt> and <tt>search</tt> variables when an interface is brought up. The OpenResolv framework will add entries to <tt>/etc/resolv.conf</tt>, and will also handle removing these entries when the interface is brought down. This way, <tt>/etc/resolv.conf</tt> should always contain current information and should not need to be manually edited by the system administrator. <tt>dhcpcd</tt> will use OpenResolv for updating system DNS information as well.<br />
<br />
=== Multiple Network Configurations ===<br />
<br />
For information on how to have multiple, independent network configurations, please see [[Stacked Runlevels]].<br />
<br />
=== Alternate Configs ===<br />
If you need to run the same service with different configuration parameters depending upon runlevel, then you'll be happy to know that you can specify runlevel-specific conf.d files by appending a <tt>.<br />
&lt;runlevel&gt;</tt> suffix. In this particular example, we could imagine a situation where we had two child runlevels named <tt>home</tt> and <tt>work</tt>:<br />
<br />
<pre>/etc/conf.d/net.eth0.home<br />
/etc/conf.d/net.eth0.work</pre>Note that this feature works for all init scripts, not just network configuration scripts.<br />
<br />
=== Interface Renaming ===<br />
<br />
Funtoo network scripts now support interface renaming, so you can create an interface called <tt>lan</tt> if you would like. To do this, simply specify the MAC address of the interface you would like to rename using the <tt>macaddr</tt> variable:<br />
<pre>macaddr=&quot;00:15:17:19:b6:a3&quot;</pre>If this MAC address is part of the <tt>net.lan</tt> configuration file, then when this interface starts, whatever interface currently has the MAC address of 00:15:17:19:b6:a3 (i.e. <tt>eth5</tt>) will be renamed to <tt>lan</tt> prior to the interface being brought up, and will show up in <tt>ifconfig</tt> and <tt>ip</tt> commands as being an interface named <tt>lan</tt>. It is possible to combine this with the <tt>mac_replace</tt> variable to set a new MAC address, if desired.<br />
<br />
=== Basic VLAN Configuration ===<br />
<br />
The standard <tt>interface</tt> template supports VLANs. To use VLAN support, first ensure that your kernel was compiled with VLAN support (the module name is <tt>8021q</tt>) :<br />
<br />
<console><br />
# ##i##grep CONFIG_VLAN /usr/src/linux/.config<br />
CONFIG_VLAN_8021Q=m<br />
CONFIG_VLAN_8021Q_GVRP=y<br />
</console><br />
<br />
Then, configure the trunk interface using the <tt>interface-noip</tt> template. Assuming <tt>eth1</tt> is trunked, you would create the file <tt>/etc/conf.d/net.eth1</tt> with the following contents:<br />
<br />
<pre>template=&quot;interface-noip&quot;</pre><br />
<br />
Then, create a network interface symlink for the trunk and add it to your default runlevel:<br />
<br />
<console><br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth1<br />
# ##i##rc-update add net.eth1 default<br />
</console><br />
<br />
Now, assuming you wanted to configure a VLAN of 32, you would create a config file named <tt>/etc/conf.d/net.eth1.32</tt> that looks something like this:<br />
<br />
<pre><br />
template=&quot;interface&quot;<br />
ipaddr=&quot;1.2.3.4/24&quot;<br />
gateway=&quot;1.2.3.1&quot;# etc...<br />
</pre><br />
<br />
Then, create a VLAN network interface symlink and add it to your default runlevel:<br />
<br />
<console><br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth1.32<br />
# ##i##rc-update add net.eth1.32 default<br />
</console><br />
<br />
The Funtoo network configuration scripts will automatically recognize the filename <tt>net.eth1.32</tt> as being VLAN 32 of trunk interface <tt>net.eth1</tt>.<br />
<br />
When the VLAN interface is brought up, it will be named <tt>eth1.32</tt>.<br />
<br />
=== Custom VLAN Names ===<br />
<br />
However, sometimes you may want to turn off automatic file-based VLAN naming and give your VLAN interface a custom name, such as <tt>mgmt</tt>. To do this, you would set up the trunk interface in the exact same way as described above, but instead of creating a <tt>net.eth1.32</tt> interface, you would create a <tt>net.mgmt</tt> interface, and specify <tt>vlan</tt> and <tt>trunk</tt> in the <tt>/etc/conf.d/net.mgmt</tt> config file, as follows:<br />
<br />
<pre>template=&quot;interface&quot;<br />
vlan=&quot;32&quot;<br />
trunk=&quot;net.eth1&quot;<br />
ipaddr=&quot;1.2.3.4/24&quot;<br />
gateway=&quot;1.2.3.1&quot;<br />
# etc...</pre><br />
When you specify <tt>trunk</tt> and <tt>vlan</tt> in the interface config file, filename-based auto-detecting of VLAN ID and trunk is disabled. Both <tt>trunk</tt> and <tt>vlan</tt> must be specified -- you can't specify just one.<br />
<br />
Then you would simply create a VLAN network interface symlink for <tt>net.mgmt</tt>:<br />
<br />
<console># ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.mgmt<br />
# ##i##rc-update add net.mgmt default</console><br />
When the VLAN interface is brought up, it will be named <tt>mgmt</tt>.<br />
<br />
=== Bonding Configuration ===<br />
<br />
Bonding allows you to aggregate multiple network interfaces into a single logical network interface, allowing for benefits in throughput as well as resiliency in the case that an individual interface may go down. This example shows how you would create a bonding interface (<tt>mybond</tt>) with a simple static ip setup, containing two slave devices (<tt>eth0</tt> and <tt>eth1</tt>).<br />
<br />
First, ensure that your kernel is configured to support bonding (the module name is <tt>bonding</tt>) :<br />
<br />
<console><br />
$ ##i##grep CONFIG_BONDING /usr/src/linux/.config<br />
CONFIG_BONDING=m<br />
</console><br />
<br />
You'l want to ensure that CONFIG_BONDING is set to "m" or "y". You can find this kernel configuration option tucked under "Device Drivers" -> "Network Device Support" -> "Bonding driver support".<br />
Be sure that ifenslave is emerged (this package included in Funtoo stage3):<br />
<br />
<console><br />
# ##i##emerge ifenslave<br />
</console><br />
Once bonding is enabled in the kernel, you will need to choose at least two devices to bond together. These will be set up as "slave" interfaces with no IP address.<br />
<br />
<console><br />
# ##i##cd /etc/init.d/<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##ln -s netif.tmpl net.eth1<br />
</console><br />
<br />
Then, configure the slave interfaces by creating <tt>/etc/conf.d/net.eth0</tt> and <tt>/etc/conf.d/net.eth1</tt> with the following contents:<br />
<br />
<pre><br />
template="interface-noip"<br />
</pre><br />
<br />
Now, we will create the bond interface and make <tt>net.eth0</tt> and <tt>net.eth1</tt> slaves of this interface. Note that our bond interface can have any name. To demonstrate this, we will give it the name of "mybond" below:<br />
<br />
<console><br />
# ##i##ln -s netif.tmpl net.mybond<br />
# ##i##rc-update add net.mybond default<br />
</console><br />
<br />
Now we can configure "mybond" using its configuration file <tt>/etc/conf.d/net.mybond</tt>, just as we would a regular interface, except that we specify <tt>slaves</tt>:<br />
<br />
<pre><br />
template="bond"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
slaves="net.eth0 net.eth1"<br />
</pre><br />
<br />
In a bonded configuration, it is common to set the MTU to the maximum possible value supported by hardware to maximize throughput. In order to do this, simply set the MTU option in <tt>/etc/conf.d/net.mybond</tt> to the maximum value supported by your hardware. The network scripts will ensure that this MTU setting is applied to all slave interfaces:<br />
<br />
<pre><br />
mtu=9000<br />
</pre><br />
<br />
=== Bridge Configuration ===<br />
<br />
When hosting virtual machines, it can be convenient to use a bridge setup. This example shows how you would create a bridge (br0) with a simple static ip setup, containing two slave devices (eth0, tap0).<br />
<br />
First, ensure that your kernel is configured to support bridging (the module name is <tt>bridge</tt>) :<br />
<br />
<console><br />
$ ##i##grep CONFIG_BRIDGE /usr/src/linux/.config<br />
CONFIG_BRIDGE=m<br />
CONFIG_BRIDGE_IGMP_SNOOPING=y<br />
</console><br />
<br />
Second, make sure you have the required software installed:<br />
<br />
<console><br />
# ##i##emerge -av bridge-utils usermode-utilities<br />
</console><br />
<br />
Then, create the necessary symlinks for the interfaces and add them to your default runlevel :<br />
<br />
<console><br />
# ##i##cd /etc/init.d/<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##ln -s netif.tmpl net.br0<br />
# ##i##ln -s netif.tmpl net.tap0<br />
# ##i##rc-update add net.br0 default<br />
# ##i##rc-update add net.tap0 default<br />
</console><br />
<br />
Then, configure the slave interface <tt>/etc/conf.d/net.eth0</tt> :<br />
<br />
<pre><br />
template="interface-noip"<br />
</pre><br />
<br />
Then, configure the slave interface <tt>/etc/conf.d/net.tap0</tt> - note you only require group OR user, not both :<br />
<br />
<pre><br />
template="tap"<br />
group="kvm" <br />
user="kvm"<br />
mac_replace="10:20:30:40:50:66"<br />
</pre><br />
<br />
... and the bridge interface <tt>/etc/conf.d/net.br0</tt> :<br />
<br />
<pre><br />
template="bridge"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
slaves="net.eth0 net.tap0"<br />
stp="on"<br />
forwarding=1<br />
</pre><br />
<br />
If you are using dhcpcd, you should ensure that it does not attempt to configure <tt>eth0</tt> or <tt>br0</tt> by adding the following to <tt>/etc/dhcpcd.conf</tt> :<br />
<br />
<pre><br />
# don't attempt to pull an ip address for br0 or its slave device<br />
denyinterfaces eth0 br0<br />
</pre><br />
<br />
=== More Complex Network Configuration ===<br />
<br />
If the standard templates don't work for your needs, simply create a new template -- I recommend starting from the <tt>interface</tt> template for most things:<br />
<br />
<console># ##i##cd /etc/netif.d<br />
# ##i##cp interface custom</console><br />
You can now call whatever commands you need to <tt>/etc/netif.d/custom</tt>. The following shell functions can be defined in a network script:<br />
<br />
==== netif_create ====<br />
<br />
In <tt>netif_create</tt>, you should call any commands to create the interface if it does not yet exist.<br />
<br />
==== netif_depend ====<br />
<br />
In <tt>netif_depend</tt>, you can define dependencies, using the functions <tt>need</tt> and <tt>use</tt>.<br />
<br />
==== netif_pre_up ====<br />
<br />
In <tt>netif_pre_up</tt>, you can define network configuration actions to perform prior to bringing the interface up. You can also ensure certain variables are specified by calling <tt>require var1 [var2...]</tt> here.<br />
<br />
==== netif_post_up====<br />
<br />
In <tt>netif_post_up</tt>, you can define network configuration actions to perform after bringing the interface up.<br />
<br />
==== netif_pre_down ====<br />
<br />
In <tt>netif_pre_down</tt>, you can define network configuration actions to perform prior to bringing the interface down.<br />
<br />
==== netif_post_down ====<br />
<br />
In <tt>netif_post_down</tt>, you can define network configuration actions to perform after bringing the interface down.<br />
<br />
==== netif_destroy ====<br />
<br />
In <tt>netif_destroy</tt>, you can call any commands necessary to destroy/delete the interface if it is dynamic in nature (tun/tap, etc.)<br />
<br />
==== How It Works ====<br />
<br />
You do not specify a function for actually bringing up the interface, because the template-based system does this for you. The template-based system also performs all normal actions required to bring an interface down, so you only need to specify atypical actions that must be performed - such as removing child interfaces or destroying a bridge using <tt>brctl</tt>.<br />
<br />
When you create your own network configuration template, the following capabilities are available for use automatically, as long as the appropriate variables are set in the <tt>/etc/conf.d/netif.&lt;ifname&gt;</tt> file, without requiring any explicit steps on your part:<br />
<br />
* DNS configuration using <tt>domain</tt> and <tt>nameservers</tt> config settings. OpenResolv is used automatically.<br />
* VLAN configuration using auto-naming (<tt>net.ethX.Y</tt>) or via custom naming with <tt>trunk</tt> and <tt>vlan</tt> config settings.<br />
* Default IPv4 gateway and route configuration using the <tt>gateway</tt> and <tt>route</tt> settings.<br />
* Default IPv6 gateway and route configuration using the <tt>gateway6</tt> and <tt>route6</tt> settings.<br />
* MTU configuration using the <tt>mtu</tt> setting.<br />
* Auto-depend (and auto-MTU configuration) of slave interfaces specified using <tt>slaves</tt> setting.<br />
* Renaming of existing network interface (specify MAC address using <tt>macaddr</tt> setting).<br />
<br />
To take advantage of this functionality, simply enable the appropriate variables.<br />
<br />
All other necessary network configuration and dependency behavior should be defined using the <tt>netif_</tt>-prefix functions described above.<br />
<br />
== Wireless Configuration ==<br />
<br />
The recommended approach for setting up Wi-Fi under Funtoo Linux is to use NetworkManager. Steps are provided in the [[Funtoo Linux Installation#Wi-Fi|Wi-Fi section of the Funtoo Linux Installation Guide]].<br />
<br />
== Other Network Configurations ==<br />
<br />
If you have a network configuration template that might be useful to others, please post it to the [https://bugs.funtoo.org bugtracker] so we can review it and possibly incorporate it into Funtoo.<br />
<br />
== License ==<br />
<br />
Funtoo Linux networking scripts are released under the following license:<br />
<br />
{{BSD2 Funtoo|src=http://github.com/funtoo/corenetwork}}<br />
<br />
[[Category:HOWTO]]<br />
[[Category:Projects]]<br />
[[Category:Networking]]<br />
[[Category:Install]]<br />
[[Category:Funtoo features]]<br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Funtoo:Networking/Configuration&diff=26547Funtoo:Networking/Configuration2019-01-08T17:21:39Z<p>Oleg: /* DHCP-Only Systems */</p>
<hr />
<div><blockquote>This document explains how to configure your network settings by explaining the network configuration functionality available in Funtoo Linux. Also covered is <tt>dhcpcd 5.x</tt>, Wi-Fi (IEEE 802.11) configuration, and the OpenResolv framework.<br />
</blockquote><br />
<br />
== Introduction ==<br />
__TOC__<br />
Funtoo Linux has its own core network configuration system that differs somewhat from upstream network configuration systems used in [http://www.gentoo.org Gentoo Linux] and [http://roy.marples.name/projects/openrc OpenRC].<br />
<br />
In this document, I will explain the unique additions and changes to the Funtoo network configuration and show you how to use this system to configure your network.<br />
<br />
I'll also explain how to use <code>dhcpcd</code> for managing network interfaces on DHCP-based networks, and will also cover OpenRC stacked runlevel configuration, ''Wi-Fi'' (IEEE 802.11) configuration, and the OpenResolv framework, which is enabled in Funtoo Linux by default.<br />
<br />
== A Gentle Introduction to Funtoo Network Configuration ==<br />
<br />
Before I get into the technical details of configuring your network, it's important to understand that Funtoo Linux has a number of different options available to you for network configuration, with more likely to be added in the future. Each approach is different and has its own strengths and weaknesses, and this is, in my opinion, a good thing.<br />
<br />
=== The Easy (Dynamic) Way ===<br />
<br />
When configuring your network, one option is to skip traditional network configuration and simply rely on DHCP. This is by far the simplest method of configuring your network. If you are on a wired network, no other steps are typically required beyond enabling a DHCP client, and Funtoo Linux includes {{c|dhcpcd 7.x}} by default. <br />
<br />
==== Network Manager, Wicd ====<br />
<br />
If you are going to use a third party package such as [[Network Manager]] or [[Wicd]] to manage your network then you do not need to configure DHCP at all. These packages configure DHCP for you. Simply emerge the package you want to use and start using it.<br />
<br />
==== DHCP-Only Systems ====<br />
<br />
If you are not planning to use a third-party package to manage your network interfaces, it is still extremely easy to set up DHCP networking, especially if you always use DHCP to connect to networks, which is common for desktops or laptops. In this scenario, we can simply enable {{c|dhcpcd}} to run at system startup. It will run in the background and automatically look for DHCP servers on all your network interfaces, and will attempt to lease an IP address from any DHCP servers found. <br />
<br />
If this sounds like what you want to do, then add {{c|dhcpcd}} to your default runlevel as follows:<br />
<br />
<console># ##i##rc-update add dhcpcd default</console><br />
<br />
To enable DHCP immediately, you would follow the previous command with an {{c|openrc}} command, which would start the {{c|dhcpcd}} client you just added:<br />
<br />
{{console|body=<br />
###i## openrc<br />
}}<br />
<br />
If you're on a wired network and have the necessary drivers in your kernel, then this should get you going. For wireless networks, more steps are required to utilize your wireless hardware to associate with an access point, which will be covered later in this document. <br />
<br />
===== Tweaking Dhcpcd =====<br />
<br />
For now, it's important to note that {{c|dhcpcd 7.x}} will manage ''all'' available network interfaces by default. If you want to run a DHCP client on ''all but one'' interface, or some other subset of interfaces, you can add the appropriate {{c|denyinterfaces}} or {{c|allowinterfaces}} [[glob pattern]] to {{c|/etc/dhcpcd.conf}}:<br />
<br />
{{file|name=/etc/dhcpcd.conf|lang=bash|body=<br />
# manage all interfaces but eth0 with dhcpcd<br />
denyinterfaces eth0<br />
}}<br />
<br />
This can also be accomplished by modifying {{c|tc/init.d/dhcpcd}} directly and adding {{c|-Z ''ifglob''}} or {{c|-z ''ifglob''}} (the equivalent command-line parameters) to {{c|command_args}}.<br />
<br />
==== Using Funtoo Scripts for DHCP ====<br />
<br />
You can also use the Funtoo Linux networking scripts to start a DHCP client just on a specific interface. This approach is best if you are planning to also do some advanced bridging, bonding or VLAN configuration on your machine along with DHCP, since you will be using the Funtoo Linux networking scripts for that too. <br />
<br />
To use this variant approach, ''don't'' enable {{c|/etc/init.d/dhcpcd}} directly. Instead, use the Funtoo Linux {{c|dhcpcd}} template which will start dhcpcd on only one interface. Below, you will see the steps to do this. This is very similar to how we set up advanced network interfaces, which will be covered later in this documentation:<br />
<br />
{{console|body=<br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##rc-update add net.eth0 default<br />
# ##i##echo template=dhcpcd > /etc/conf.d/net.eth0<br />
# ##i##openrc<br />
}}<br />
<br />
The last command, {{c|openrc}}, causes {{c|net.eth0}} to be started.<br />
<br />
=== Server Network Configuration ===<br />
<br />
For servers and advanced networking scenarios, Funtoo Linux offers its own modular, template-based network configuration system. This system offers a lot of flexibility for configuring network interfaces, essentially serving as a &quot;network interface construction kit.&quot; This system can be used by itself, or even combined with <tt>dhcpcd</tt>, as shown in the previous section.<br />
<br />
Here are the key components of the template-based network configuration system:<br />
<br />
;{{c|/etc/init.d/loopback}}: An init script that configures the localhost interface. This script is always enabled and is part of the boot process.<br />
;{{c|/etc/netif.d}}: This is a directory that contains various network configuration templates. Each of these templates is focused on configuring a particular type of network interface, such as a general static IP-based interface, a bridge interface, a bond interface, etc.<br />
;{{c|/etc/init.d/netif.tmpl}}: This is the master init script for the template-based network configuration system. New interfaces are added to your system by creating '''symbolic links''' to this file in {{c|/etc/init.d}}.<br />
<br />
So, if you wanted to use this system to configure {{c|eth0}} with a static IP address, you would create a {{c|net.eth0}} symlink to {{c|netif.tmpl}} as follows:<br />
<br />
{{console|body=<br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth0<br />
}}<br />
<br />
Then, you would create an {{c|/etc/conf.d/net.eth0}} configuration file that would specify which template to use from the {{c|/etc/netif.d}} directory:<br />
<br />
{{file|name=/etc/conf.d/neteth0|lang=bash|body=<br />
template="interface"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
}}<br />
<br />
To complete our static IP network configuration we would need to:<br />
<br />
<console># ##i##rc-update add net.eth0 default</console><br />
When configuring your own static network interface, one of <tt>ipaddr</tt> or <tt>ipaddrs</tt> is required and should specify the IP address(es) to configure for this interface, in &quot;a.b.c.d/netmask&quot; format. Optional parameters include <tt>gateway</tt>, which defines a default gateway for your entire network, and if set should specify the gateway's IP address. In addition, <tt>domain</tt> and <tt>nameservers</tt> (space-separated if more than one) can be used to specify DNS information for this interface.<br />
<br />
=== Configuration Variables ===<br />
<br />
==== Interface Variables ====<br />
<br />
The <tt>ipaddr</tt> and <tt>ipaddrs</tt> variables are supported by the <tt>interface</tt> and <tt>bridge</tt> templates, and are used to specify a single or multiple IPv4 or IPv6 address(es) for the interface. IP addresses should be specified in 'IP/netmask' format, such as <tt>10.0.0.1/24</tt>. Multiple IP addresses can be specified delimited by whitespace:<br />
<br />
<pre>ipaddrs=&quot;10.0.0.1/24 10.0.0.2/24&quot;</pre><br />
<br />
===== Broadcast Address =====<br />
<br />
By default, a broadcast address will be calculated based on the IP address and network mask. If you need to manually specify a broadcast address, use the following format for your IP address:<br />
<br />
<pre><br />
ipaddrs="10.0.0.1/24;broadcast=10.0.1.255 10.0.0.2/24"<br />
</pre><br />
<br />
===== Not Specifying An Address =====<br />
<br />
Note that in some cases, you may choose to '''not''' specify <tt>ipaddr</tt> or <tt>ipaddrs</tt> for a <tt>bridge</tt> template. That is allowed. If you don't want to specify an IP address for a regular interface, you can choose to use the <tt>interface</tt> template without an IP address specified in the config, or use the <tt>interface-noip</tt> template instead, for the sake of clarity.<br />
<br />
===== Viewing All Configured IP Addresses =====<br />
<br />
Also note that if you specify multiple IPv4 addresses, <tt>ifconfig</tt> will only show the first IP address. To view all IP addresses associated with the interface, use the <tt>ip addr show</tt> command.<br />
<br />
=== General Variables ===<br />
<br />
The following variables are enabled by default for all network scripts, and if specified will trigger a corresponding configuration action:<br />
<br />
;<tt>nameservers</tt>: Set DNS nameservers using OpenResolv. Specify multiple IPv4 or IPv6 nameservers like this: &quot;1.2.3.4 1.2.3.5 1.2.3.6&quot;. Please note that OpenResolv treats <tt>127.0.0.1</tt> specially, and it indicates that you are running a local name resolver like <tt>dnsmasq</tt> or <tt>bind</tt>. OpenResolv will ignore all other name servers specified alongside <tt>127.0.0.1</tt>. See <tt>man resolvconf</tt> and <tt>man resolvconf.conf</tt> for additional setup information.<br />
;<tt>search</tt>: Set DNS search information using OpenResolv.<br />
;<tt>domain</tt>: Set DNS domain using OpenResolv.<br />
;<tt>gateway</tt>: Define a default IPv4 gateway on this interface.<br />
;<tt>gateway6</tt>: Define a default IPv6 gateway on this interface.<br />
;<tt>route</tt>: Specify a semi-colon delimited list of IPv4 routes to apply when this interface is brought up. Will be appended to <tt>ip -4 route add</tt>.<br />
;<tt>route6</tt>: Specify a semi-colon delimited list of IPv6 routes to apply when this interface is brought up. Will be appended to <tt>ip -6 route add</tt>.<br />
;<tt>mtu</tt>: Set Maximum Transmit Unit for the interface<br />
;<tt>mac_replace</tt>: Replace existing MAC address with MAC address specified in this variable.<br />
<br />
==== VLAN Variables ====<br />
<br />
VLAN support is enabled by default for all network configuration scripts. If a network script has a name in the format <tt>net.ethX.Y</tt>, then it is assumed to be a VLAN interface referencing trunk <tt>ethX</tt> and VLAN ID <tt>Y</tt>. If you desire a custom name for your VLAN interface, you can name your interface whatever you'd like and specify the following variables in your interface config:<br />
<br />
;<tt>trunk</tt>: VLAN trunk interface, e.g. &quot;net.eth0&quot;<br />
;<tt>vlan</tt>: VLAN id, e.g. &quot;32&quot;<br />
<br />
==== Bridge / Tap Variables ====<br />
<br />
The following variables for configuring a functional bridge interface with optional tap interfaces:<br />
<br />
;<tt>slaves</tt>: Set slave interfaces of this interface (for bridges, etc.) All slaves will automatically be depended upon, and will also automatically have their <tt>mtu</tt> set to that of the current interface, if an <tt>mtu</tt> is specified for the current interface. This setting is required for the <tt>bond</tt> template and optional for the <tt>bridge</tt> template.<br />
;<tt>stp</tt>: Enables Spanning Tree Protocol on a bridge interface like this &quot;stp=on&quot;<br />
;<tt>forwarding</tt>: Enables forwarding on a bridge interface by calling sysctl; as this interface does not exist when sysctl is called by init, we do it here. If this is disabled, your bridge will not forward traffic back out onto the network. useage: &quot;forwarding=1&quot;<br />
<br />
=== OpenResolv and resolv.conf ===<br />
<br />
OpenResolv will be used to set DNS information provided by the <tt>nameservers</tt>, <tt>domain</tt> and <tt>search</tt> variables when an interface is brought up. The OpenResolv framework will add entries to <tt>/etc/resolv.conf</tt>, and will also handle removing these entries when the interface is brought down. This way, <tt>/etc/resolv.conf</tt> should always contain current information and should not need to be manually edited by the system administrator. <tt>dhcpcd</tt> will use OpenResolv for updating system DNS information as well.<br />
<br />
=== Multiple Network Configurations ===<br />
<br />
For information on how to have multiple, independent network configurations, please see [[Stacked Runlevels]].<br />
<br />
=== Alternate Configs ===<br />
If you need to run the same service with different configuration parameters depending upon runlevel, then you'll be happy to know that you can specify runlevel-specific conf.d files by appending a <tt>.<br />
&lt;runlevel&gt;</tt> suffix. In this particular example, we could imagine a situation where we had two child runlevels named <tt>home</tt> and <tt>work</tt>:<br />
<br />
<pre>/etc/conf.d/net.eth0.home<br />
/etc/conf.d/net.eth0.work</pre>Note that this feature works for all init scripts, not just network configuration scripts.<br />
<br />
=== Interface Renaming ===<br />
<br />
Funtoo network scripts now support interface renaming, so you can create an interface called <tt>lan</tt> if you would like. To do this, simply specify the MAC address of the interface you would like to rename using the <tt>macaddr</tt> variable:<br />
<pre>macaddr=&quot;00:15:17:19:b6:a3&quot;</pre>If this MAC address is part of the <tt>net.lan</tt> configuration file, then when this interface starts, whatever interface currently has the MAC address of 00:15:17:19:b6:a3 (i.e. <tt>eth5</tt>) will be renamed to <tt>lan</tt> prior to the interface being brought up, and will show up in <tt>ifconfig</tt> and <tt>ip</tt> commands as being an interface named <tt>lan</tt>. It is possible to combine this with the <tt>mac_replace</tt> variable to set a new MAC address, if desired.<br />
<br />
=== Basic VLAN Configuration ===<br />
<br />
The standard <tt>interface</tt> template supports VLANs. To use VLAN support, first ensure that your kernel was compiled with VLAN support (the module name is <tt>8021q</tt>) :<br />
<br />
<console><br />
# ##i##grep CONFIG_VLAN /usr/src/linux/.config<br />
CONFIG_VLAN_8021Q=m<br />
CONFIG_VLAN_8021Q_GVRP=y<br />
</console><br />
<br />
Then, configure the trunk interface using the <tt>interface-noip</tt> template. Assuming <tt>eth1</tt> is trunked, you would create the file <tt>/etc/conf.d/net.eth1</tt> with the following contents:<br />
<br />
<pre>template=&quot;interface-noip&quot;</pre><br />
<br />
Then, create a network interface symlink for the trunk and add it to your default runlevel:<br />
<br />
<console><br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth1<br />
# ##i##rc-update add net.eth1 default<br />
</console><br />
<br />
Now, assuming you wanted to configure a VLAN of 32, you would create a config file named <tt>/etc/conf.d/net.eth1.32</tt> that looks something like this:<br />
<br />
<pre><br />
template=&quot;interface&quot;<br />
ipaddr=&quot;1.2.3.4/24&quot;<br />
gateway=&quot;1.2.3.1&quot;# etc...<br />
</pre><br />
<br />
Then, create a VLAN network interface symlink and add it to your default runlevel:<br />
<br />
<console><br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth1.32<br />
# ##i##rc-update add net.eth1.32 default<br />
</console><br />
<br />
The Funtoo network configuration scripts will automatically recognize the filename <tt>net.eth1.32</tt> as being VLAN 32 of trunk interface <tt>net.eth1</tt>.<br />
<br />
When the VLAN interface is brought up, it will be named <tt>eth1.32</tt>.<br />
<br />
=== Custom VLAN Names ===<br />
<br />
However, sometimes you may want to turn off automatic file-based VLAN naming and give your VLAN interface a custom name, such as <tt>mgmt</tt>. To do this, you would set up the trunk interface in the exact same way as described above, but instead of creating a <tt>net.eth1.32</tt> interface, you would create a <tt>net.mgmt</tt> interface, and specify <tt>vlan</tt> and <tt>trunk</tt> in the <tt>/etc/conf.d/net.mgmt</tt> config file, as follows:<br />
<br />
<pre>template=&quot;interface&quot;<br />
vlan=&quot;32&quot;<br />
trunk=&quot;net.eth1&quot;<br />
ipaddr=&quot;1.2.3.4/24&quot;<br />
gateway=&quot;1.2.3.1&quot;<br />
# etc...</pre><br />
When you specify <tt>trunk</tt> and <tt>vlan</tt> in the interface config file, filename-based auto-detecting of VLAN ID and trunk is disabled. Both <tt>trunk</tt> and <tt>vlan</tt> must be specified -- you can't specify just one.<br />
<br />
Then you would simply create a VLAN network interface symlink for <tt>net.mgmt</tt>:<br />
<br />
<console># ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.mgmt<br />
# ##i##rc-update add net.mgmt default</console><br />
When the VLAN interface is brought up, it will be named <tt>mgmt</tt>.<br />
<br />
=== Bonding Configuration ===<br />
<br />
Bonding allows you to aggregate multiple network interfaces into a single logical network interface, allowing for benefits in throughput as well as resiliency in the case that an individual interface may go down. This example shows how you would create a bonding interface (<tt>mybond</tt>) with a simple static ip setup, containing two slave devices (<tt>eth0</tt> and <tt>eth1</tt>).<br />
<br />
First, ensure that your kernel is configured to support bonding (the module name is <tt>bonding</tt>) :<br />
<br />
<console><br />
$ ##i##grep CONFIG_BONDING /usr/src/linux/.config<br />
CONFIG_BONDING=m<br />
</console><br />
<br />
You'l want to ensure that CONFIG_BONDING is set to "m" or "y". You can find this kernel configuration option tucked under "Device Drivers" -> "Network Device Support" -> "Bonding driver support".<br />
Be sure that ifenslave is emerged (this package included in Funtoo stage3):<br />
<br />
<console><br />
# ##i##emerge ifenslave<br />
</console><br />
Once bonding is enabled in the kernel, you will need to choose at least two devices to bond together. These will be set up as "slave" interfaces with no IP address.<br />
<br />
<console><br />
# ##i##cd /etc/init.d/<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##ln -s netif.tmpl net.eth1<br />
</console><br />
<br />
Then, configure the slave interfaces by creating <tt>/etc/conf.d/net.eth0</tt> and <tt>/etc/conf.d/net.eth1</tt> with the following contents:<br />
<br />
<pre><br />
template="interface-noip"<br />
</pre><br />
<br />
Now, we will create the bond interface and make <tt>net.eth0</tt> and <tt>net.eth1</tt> slaves of this interface. Note that our bond interface can have any name. To demonstrate this, we will give it the name of "mybond" below:<br />
<br />
<console><br />
# ##i##ln -s netif.tmpl net.mybond<br />
# ##i##rc-update add net.mybond default<br />
</console><br />
<br />
Now we can configure "mybond" using its configuration file <tt>/etc/conf.d/net.mybond</tt>, just as we would a regular interface, except that we specify <tt>slaves</tt>:<br />
<br />
<pre><br />
template="bond"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
slaves="net.eth0 net.eth1"<br />
</pre><br />
<br />
In a bonded configuration, it is common to set the MTU to the maximum possible value supported by hardware to maximize throughput. In order to do this, simply set the MTU option in <tt>/etc/conf.d/net.mybond</tt> to the maximum value supported by your hardware. The network scripts will ensure that this MTU setting is applied to all slave interfaces:<br />
<br />
<pre><br />
mtu=9000<br />
</pre><br />
<br />
=== Bridge Configuration ===<br />
<br />
When hosting virtual machines, it can be convenient to use a bridge setup. This example shows how you would create a bridge (br0) with a simple static ip setup, containing two slave devices (eth0, tap0).<br />
<br />
First, ensure that your kernel is configured to support bridging (the module name is <tt>bridge</tt>) :<br />
<br />
<console><br />
$ ##i##grep CONFIG_BRIDGE /usr/src/linux/.config<br />
CONFIG_BRIDGE=m<br />
CONFIG_BRIDGE_IGMP_SNOOPING=y<br />
</console><br />
<br />
Second, make sure you have the required software installed:<br />
<br />
<console><br />
# ##i##emerge -av bridge-utils usermode-utilities<br />
</console><br />
<br />
Then, create the necessary symlinks for the interfaces and add them to your default runlevel :<br />
<br />
<console><br />
# ##i##cd /etc/init.d/<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##ln -s netif.tmpl net.br0<br />
# ##i##ln -s netif.tmpl net.tap0<br />
# ##i##rc-update add net.br0 default<br />
# ##i##rc-update add net.tap0 default<br />
</console><br />
<br />
Then, configure the slave interface <tt>/etc/conf.d/net.eth0</tt> :<br />
<br />
<pre><br />
template="interface-noip"<br />
</pre><br />
<br />
Then, configure the slave interface <tt>/etc/conf.d/net.tap0</tt> - note you only require group OR user, not both :<br />
<br />
<pre><br />
template="tap"<br />
group="kvm" <br />
user="kvm"<br />
mac_replace="10:20:30:40:50:66"<br />
</pre><br />
<br />
... and the bridge interface <tt>/etc/conf.d/net.br0</tt> :<br />
<br />
<pre><br />
template="bridge"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
slaves="net.eth0 net.tap0"<br />
stp="on"<br />
forwarding=1<br />
</pre><br />
<br />
If you are using dhcpcd, you should ensure that it does not attempt to configure <tt>eth0</tt> or <tt>br0</tt> by adding the following to <tt>/etc/dhcpcd.conf</tt> :<br />
<br />
<pre><br />
# don't attempt to pull an ip address for br0 or its slave device<br />
denyinterfaces eth0 br0<br />
</pre><br />
<br />
=== More Complex Network Configuration ===<br />
<br />
If the standard templates don't work for your needs, simply create a new template -- I recommend starting from the <tt>interface</tt> template for most things:<br />
<br />
<console># ##i##cd /etc/netif.d<br />
# ##i##cp interface custom</console><br />
You can now call whatever commands you need to <tt>/etc/netif.d/custom</tt>. The following shell functions can be defined in a network script:<br />
<br />
==== netif_create ====<br />
<br />
In <tt>netif_create</tt>, you should call any commands to create the interface if it does not yet exist.<br />
<br />
==== netif_depend ====<br />
<br />
In <tt>netif_depend</tt>, you can define dependencies, using the functions <tt>need</tt> and <tt>use</tt>.<br />
<br />
==== netif_pre_up ====<br />
<br />
In <tt>netif_pre_up</tt>, you can define network configuration actions to perform prior to bringing the interface up. You can also ensure certain variables are specified by calling <tt>require var1 [var2...]</tt> here.<br />
<br />
==== netif_post_up====<br />
<br />
In <tt>netif_post_up</tt>, you can define network configuration actions to perform after bringing the interface up.<br />
<br />
==== netif_pre_down ====<br />
<br />
In <tt>netif_pre_down</tt>, you can define network configuration actions to perform prior to bringing the interface down.<br />
<br />
==== netif_post_down ====<br />
<br />
In <tt>netif_post_down</tt>, you can define network configuration actions to perform after bringing the interface down.<br />
<br />
==== netif_destroy ====<br />
<br />
In <tt>netif_destroy</tt>, you can call any commands necessary to destroy/delete the interface if it is dynamic in nature (tun/tap, etc.)<br />
<br />
==== How It Works ====<br />
<br />
You do not specify a function for actually bringing up the interface, because the template-based system does this for you. The template-based system also performs all normal actions required to bring an interface down, so you only need to specify atypical actions that must be performed - such as removing child interfaces or destroying a bridge using <tt>brctl</tt>.<br />
<br />
When you create your own network configuration template, the following capabilities are available for use automatically, as long as the appropriate variables are set in the <tt>/etc/conf.d/netif.&lt;ifname&gt;</tt> file, without requiring any explicit steps on your part:<br />
<br />
* DNS configuration using <tt>domain</tt> and <tt>nameservers</tt> config settings. OpenResolv is used automatically.<br />
* VLAN configuration using auto-naming (<tt>net.ethX.Y</tt>) or via custom naming with <tt>trunk</tt> and <tt>vlan</tt> config settings.<br />
* Default IPv4 gateway and route configuration using the <tt>gateway</tt> and <tt>route</tt> settings.<br />
* Default IPv6 gateway and route configuration using the <tt>gateway6</tt> and <tt>route6</tt> settings.<br />
* MTU configuration using the <tt>mtu</tt> setting.<br />
* Auto-depend (and auto-MTU configuration) of slave interfaces specified using <tt>slaves</tt> setting.<br />
* Renaming of existing network interface (specify MAC address using <tt>macaddr</tt> setting).<br />
<br />
To take advantage of this functionality, simply enable the appropriate variables.<br />
<br />
All other necessary network configuration and dependency behavior should be defined using the <tt>netif_</tt>-prefix functions described above.<br />
<br />
== Wireless Configuration ==<br />
<br />
The recommended approach for setting up Wi-Fi under Funtoo Linux is to use NetworkManager. Steps are provided in the [[Funtoo Linux Installation#Wi-Fi|Wi-Fi section of the Funtoo Linux Installation Guide]].<br />
<br />
== Other Network Configurations ==<br />
<br />
If you have a network configuration template that might be useful to others, please post it to the [https://bugs.funtoo.org bugtracker] so we can review it and possibly incorporate it into Funtoo.<br />
<br />
== License ==<br />
<br />
Funtoo Linux networking scripts are released under the following license:<br />
<br />
{{BSD2 Funtoo|src=http://github.com/funtoo/corenetwork}}<br />
<br />
[[Category:HOWTO]]<br />
[[Category:Projects]]<br />
[[Category:Networking]]<br />
[[Category:Install]]<br />
[[Category:Funtoo features]]<br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Funtoo:Networking/Configuration&diff=26546Funtoo:Networking/Configuration2019-01-08T17:20:47Z<p>Oleg: /* DHCP-Only Systems */</p>
<hr />
<div><blockquote>This document explains how to configure your network settings by explaining the network configuration functionality available in Funtoo Linux. Also covered is <tt>dhcpcd 5.x</tt>, Wi-Fi (IEEE 802.11) configuration, and the OpenResolv framework.<br />
</blockquote><br />
<br />
== Introduction ==<br />
__TOC__<br />
Funtoo Linux has its own core network configuration system that differs somewhat from upstream network configuration systems used in [http://www.gentoo.org Gentoo Linux] and [http://roy.marples.name/projects/openrc OpenRC].<br />
<br />
In this document, I will explain the unique additions and changes to the Funtoo network configuration and show you how to use this system to configure your network.<br />
<br />
I'll also explain how to use <code>dhcpcd</code> for managing network interfaces on DHCP-based networks, and will also cover OpenRC stacked runlevel configuration, ''Wi-Fi'' (IEEE 802.11) configuration, and the OpenResolv framework, which is enabled in Funtoo Linux by default.<br />
<br />
== A Gentle Introduction to Funtoo Network Configuration ==<br />
<br />
Before I get into the technical details of configuring your network, it's important to understand that Funtoo Linux has a number of different options available to you for network configuration, with more likely to be added in the future. Each approach is different and has its own strengths and weaknesses, and this is, in my opinion, a good thing.<br />
<br />
=== The Easy (Dynamic) Way ===<br />
<br />
When configuring your network, one option is to skip traditional network configuration and simply rely on DHCP. This is by far the simplest method of configuring your network. If you are on a wired network, no other steps are typically required beyond enabling a DHCP client, and Funtoo Linux includes {{c|dhcpcd 7.x}} by default. <br />
<br />
==== Network Manager, Wicd ====<br />
<br />
If you are going to use a third party package such as [[Network Manager]] or [[Wicd]] to manage your network then you do not need to configure DHCP at all. These packages configure DHCP for you. Simply emerge the package you want to use and start using it.<br />
<br />
==== DHCP-Only Systems ====<br />
<br />
If you are not planning to use a third-party package to manage your network interfaces, it is still extremely easy to set up DHCP networking, especially if you always use DHCP to connect to networks, which is common for desktops or laptops. In this scenario, we can simply enable {{c|dhcpcd}} to run at system startup. It will run in the background and automatically look for DHCP servers on all your network interfaces, and will attempt to lease an IP address from any DHCP servers found. <br />
<br />
If this sounds like what you want to do, then add {{c|dhcpcd}} to your default runlevel as follows:<br />
<br />
<console># ##i##rc-update add dhcpcd default</console><br />
<br />
To enable DHCP immediately, you would follow the previous command with an <code>rc</code> command, which would start the <code>dhcpcd</code> client you just added:<br />
<br />
{{console|body=<br />
###i## openrc<br />
}}<br />
<br />
If you're on a wired network and have the necessary drivers in your kernel, then this should get you going. For wireless networks, more steps are required to utilize your wireless hardware to associate with an access point, which will be covered later in this document. <br />
<br />
===== Tweaking Dhcpcd =====<br />
<br />
For now, it's important to note that {{c|dhcpcd 7.x}} will manage ''all'' available network interfaces by default. If you want to run a DHCP client on ''all but one'' interface, or some other subset of interfaces, you can add the appropriate {{c|denyinterfaces}} or {{c|allowinterfaces}} [[glob pattern]] to {{c|/etc/dhcpcd.conf}}:<br />
<br />
{{file|name=/etc/dhcpcd.conf|lang=bash|body=<br />
# manage all interfaces but eth0 with dhcpcd<br />
denyinterfaces eth0<br />
}}<br />
<br />
This can also be accomplished by modifying {{c|tc/init.d/dhcpcd}} directly and adding {{c|-Z ''ifglob''}} or {{c|-z ''ifglob''}} (the equivalent command-line parameters) to {{c|command_args}}.<br />
<br />
==== Using Funtoo Scripts for DHCP ====<br />
<br />
You can also use the Funtoo Linux networking scripts to start a DHCP client just on a specific interface. This approach is best if you are planning to also do some advanced bridging, bonding or VLAN configuration on your machine along with DHCP, since you will be using the Funtoo Linux networking scripts for that too. <br />
<br />
To use this variant approach, ''don't'' enable {{c|/etc/init.d/dhcpcd}} directly. Instead, use the Funtoo Linux {{c|dhcpcd}} template which will start dhcpcd on only one interface. Below, you will see the steps to do this. This is very similar to how we set up advanced network interfaces, which will be covered later in this documentation:<br />
<br />
{{console|body=<br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##rc-update add net.eth0 default<br />
# ##i##echo template=dhcpcd > /etc/conf.d/net.eth0<br />
# ##i##openrc<br />
}}<br />
<br />
The last command, {{c|openrc}}, causes {{c|net.eth0}} to be started.<br />
<br />
=== Server Network Configuration ===<br />
<br />
For servers and advanced networking scenarios, Funtoo Linux offers its own modular, template-based network configuration system. This system offers a lot of flexibility for configuring network interfaces, essentially serving as a &quot;network interface construction kit.&quot; This system can be used by itself, or even combined with <tt>dhcpcd</tt>, as shown in the previous section.<br />
<br />
Here are the key components of the template-based network configuration system:<br />
<br />
;{{c|/etc/init.d/loopback}}: An init script that configures the localhost interface. This script is always enabled and is part of the boot process.<br />
;{{c|/etc/netif.d}}: This is a directory that contains various network configuration templates. Each of these templates is focused on configuring a particular type of network interface, such as a general static IP-based interface, a bridge interface, a bond interface, etc.<br />
;{{c|/etc/init.d/netif.tmpl}}: This is the master init script for the template-based network configuration system. New interfaces are added to your system by creating '''symbolic links''' to this file in {{c|/etc/init.d}}.<br />
<br />
So, if you wanted to use this system to configure {{c|eth0}} with a static IP address, you would create a {{c|net.eth0}} symlink to {{c|netif.tmpl}} as follows:<br />
<br />
{{console|body=<br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth0<br />
}}<br />
<br />
Then, you would create an {{c|/etc/conf.d/net.eth0}} configuration file that would specify which template to use from the {{c|/etc/netif.d}} directory:<br />
<br />
{{file|name=/etc/conf.d/neteth0|lang=bash|body=<br />
template="interface"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
}}<br />
<br />
To complete our static IP network configuration we would need to:<br />
<br />
<console># ##i##rc-update add net.eth0 default</console><br />
When configuring your own static network interface, one of <tt>ipaddr</tt> or <tt>ipaddrs</tt> is required and should specify the IP address(es) to configure for this interface, in &quot;a.b.c.d/netmask&quot; format. Optional parameters include <tt>gateway</tt>, which defines a default gateway for your entire network, and if set should specify the gateway's IP address. In addition, <tt>domain</tt> and <tt>nameservers</tt> (space-separated if more than one) can be used to specify DNS information for this interface.<br />
<br />
=== Configuration Variables ===<br />
<br />
==== Interface Variables ====<br />
<br />
The <tt>ipaddr</tt> and <tt>ipaddrs</tt> variables are supported by the <tt>interface</tt> and <tt>bridge</tt> templates, and are used to specify a single or multiple IPv4 or IPv6 address(es) for the interface. IP addresses should be specified in 'IP/netmask' format, such as <tt>10.0.0.1/24</tt>. Multiple IP addresses can be specified delimited by whitespace:<br />
<br />
<pre>ipaddrs=&quot;10.0.0.1/24 10.0.0.2/24&quot;</pre><br />
<br />
===== Broadcast Address =====<br />
<br />
By default, a broadcast address will be calculated based on the IP address and network mask. If you need to manually specify a broadcast address, use the following format for your IP address:<br />
<br />
<pre><br />
ipaddrs="10.0.0.1/24;broadcast=10.0.1.255 10.0.0.2/24"<br />
</pre><br />
<br />
===== Not Specifying An Address =====<br />
<br />
Note that in some cases, you may choose to '''not''' specify <tt>ipaddr</tt> or <tt>ipaddrs</tt> for a <tt>bridge</tt> template. That is allowed. If you don't want to specify an IP address for a regular interface, you can choose to use the <tt>interface</tt> template without an IP address specified in the config, or use the <tt>interface-noip</tt> template instead, for the sake of clarity.<br />
<br />
===== Viewing All Configured IP Addresses =====<br />
<br />
Also note that if you specify multiple IPv4 addresses, <tt>ifconfig</tt> will only show the first IP address. To view all IP addresses associated with the interface, use the <tt>ip addr show</tt> command.<br />
<br />
=== General Variables ===<br />
<br />
The following variables are enabled by default for all network scripts, and if specified will trigger a corresponding configuration action:<br />
<br />
;<tt>nameservers</tt>: Set DNS nameservers using OpenResolv. Specify multiple IPv4 or IPv6 nameservers like this: &quot;1.2.3.4 1.2.3.5 1.2.3.6&quot;. Please note that OpenResolv treats <tt>127.0.0.1</tt> specially, and it indicates that you are running a local name resolver like <tt>dnsmasq</tt> or <tt>bind</tt>. OpenResolv will ignore all other name servers specified alongside <tt>127.0.0.1</tt>. See <tt>man resolvconf</tt> and <tt>man resolvconf.conf</tt> for additional setup information.<br />
;<tt>search</tt>: Set DNS search information using OpenResolv.<br />
;<tt>domain</tt>: Set DNS domain using OpenResolv.<br />
;<tt>gateway</tt>: Define a default IPv4 gateway on this interface.<br />
;<tt>gateway6</tt>: Define a default IPv6 gateway on this interface.<br />
;<tt>route</tt>: Specify a semi-colon delimited list of IPv4 routes to apply when this interface is brought up. Will be appended to <tt>ip -4 route add</tt>.<br />
;<tt>route6</tt>: Specify a semi-colon delimited list of IPv6 routes to apply when this interface is brought up. Will be appended to <tt>ip -6 route add</tt>.<br />
;<tt>mtu</tt>: Set Maximum Transmit Unit for the interface<br />
;<tt>mac_replace</tt>: Replace existing MAC address with MAC address specified in this variable.<br />
<br />
==== VLAN Variables ====<br />
<br />
VLAN support is enabled by default for all network configuration scripts. If a network script has a name in the format <tt>net.ethX.Y</tt>, then it is assumed to be a VLAN interface referencing trunk <tt>ethX</tt> and VLAN ID <tt>Y</tt>. If you desire a custom name for your VLAN interface, you can name your interface whatever you'd like and specify the following variables in your interface config:<br />
<br />
;<tt>trunk</tt>: VLAN trunk interface, e.g. &quot;net.eth0&quot;<br />
;<tt>vlan</tt>: VLAN id, e.g. &quot;32&quot;<br />
<br />
==== Bridge / Tap Variables ====<br />
<br />
The following variables for configuring a functional bridge interface with optional tap interfaces:<br />
<br />
;<tt>slaves</tt>: Set slave interfaces of this interface (for bridges, etc.) All slaves will automatically be depended upon, and will also automatically have their <tt>mtu</tt> set to that of the current interface, if an <tt>mtu</tt> is specified for the current interface. This setting is required for the <tt>bond</tt> template and optional for the <tt>bridge</tt> template.<br />
;<tt>stp</tt>: Enables Spanning Tree Protocol on a bridge interface like this &quot;stp=on&quot;<br />
;<tt>forwarding</tt>: Enables forwarding on a bridge interface by calling sysctl; as this interface does not exist when sysctl is called by init, we do it here. If this is disabled, your bridge will not forward traffic back out onto the network. useage: &quot;forwarding=1&quot;<br />
<br />
=== OpenResolv and resolv.conf ===<br />
<br />
OpenResolv will be used to set DNS information provided by the <tt>nameservers</tt>, <tt>domain</tt> and <tt>search</tt> variables when an interface is brought up. The OpenResolv framework will add entries to <tt>/etc/resolv.conf</tt>, and will also handle removing these entries when the interface is brought down. This way, <tt>/etc/resolv.conf</tt> should always contain current information and should not need to be manually edited by the system administrator. <tt>dhcpcd</tt> will use OpenResolv for updating system DNS information as well.<br />
<br />
=== Multiple Network Configurations ===<br />
<br />
For information on how to have multiple, independent network configurations, please see [[Stacked Runlevels]].<br />
<br />
=== Alternate Configs ===<br />
If you need to run the same service with different configuration parameters depending upon runlevel, then you'll be happy to know that you can specify runlevel-specific conf.d files by appending a <tt>.<br />
&lt;runlevel&gt;</tt> suffix. In this particular example, we could imagine a situation where we had two child runlevels named <tt>home</tt> and <tt>work</tt>:<br />
<br />
<pre>/etc/conf.d/net.eth0.home<br />
/etc/conf.d/net.eth0.work</pre>Note that this feature works for all init scripts, not just network configuration scripts.<br />
<br />
=== Interface Renaming ===<br />
<br />
Funtoo network scripts now support interface renaming, so you can create an interface called <tt>lan</tt> if you would like. To do this, simply specify the MAC address of the interface you would like to rename using the <tt>macaddr</tt> variable:<br />
<pre>macaddr=&quot;00:15:17:19:b6:a3&quot;</pre>If this MAC address is part of the <tt>net.lan</tt> configuration file, then when this interface starts, whatever interface currently has the MAC address of 00:15:17:19:b6:a3 (i.e. <tt>eth5</tt>) will be renamed to <tt>lan</tt> prior to the interface being brought up, and will show up in <tt>ifconfig</tt> and <tt>ip</tt> commands as being an interface named <tt>lan</tt>. It is possible to combine this with the <tt>mac_replace</tt> variable to set a new MAC address, if desired.<br />
<br />
=== Basic VLAN Configuration ===<br />
<br />
The standard <tt>interface</tt> template supports VLANs. To use VLAN support, first ensure that your kernel was compiled with VLAN support (the module name is <tt>8021q</tt>) :<br />
<br />
<console><br />
# ##i##grep CONFIG_VLAN /usr/src/linux/.config<br />
CONFIG_VLAN_8021Q=m<br />
CONFIG_VLAN_8021Q_GVRP=y<br />
</console><br />
<br />
Then, configure the trunk interface using the <tt>interface-noip</tt> template. Assuming <tt>eth1</tt> is trunked, you would create the file <tt>/etc/conf.d/net.eth1</tt> with the following contents:<br />
<br />
<pre>template=&quot;interface-noip&quot;</pre><br />
<br />
Then, create a network interface symlink for the trunk and add it to your default runlevel:<br />
<br />
<console><br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth1<br />
# ##i##rc-update add net.eth1 default<br />
</console><br />
<br />
Now, assuming you wanted to configure a VLAN of 32, you would create a config file named <tt>/etc/conf.d/net.eth1.32</tt> that looks something like this:<br />
<br />
<pre><br />
template=&quot;interface&quot;<br />
ipaddr=&quot;1.2.3.4/24&quot;<br />
gateway=&quot;1.2.3.1&quot;# etc...<br />
</pre><br />
<br />
Then, create a VLAN network interface symlink and add it to your default runlevel:<br />
<br />
<console><br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth1.32<br />
# ##i##rc-update add net.eth1.32 default<br />
</console><br />
<br />
The Funtoo network configuration scripts will automatically recognize the filename <tt>net.eth1.32</tt> as being VLAN 32 of trunk interface <tt>net.eth1</tt>.<br />
<br />
When the VLAN interface is brought up, it will be named <tt>eth1.32</tt>.<br />
<br />
=== Custom VLAN Names ===<br />
<br />
However, sometimes you may want to turn off automatic file-based VLAN naming and give your VLAN interface a custom name, such as <tt>mgmt</tt>. To do this, you would set up the trunk interface in the exact same way as described above, but instead of creating a <tt>net.eth1.32</tt> interface, you would create a <tt>net.mgmt</tt> interface, and specify <tt>vlan</tt> and <tt>trunk</tt> in the <tt>/etc/conf.d/net.mgmt</tt> config file, as follows:<br />
<br />
<pre>template=&quot;interface&quot;<br />
vlan=&quot;32&quot;<br />
trunk=&quot;net.eth1&quot;<br />
ipaddr=&quot;1.2.3.4/24&quot;<br />
gateway=&quot;1.2.3.1&quot;<br />
# etc...</pre><br />
When you specify <tt>trunk</tt> and <tt>vlan</tt> in the interface config file, filename-based auto-detecting of VLAN ID and trunk is disabled. Both <tt>trunk</tt> and <tt>vlan</tt> must be specified -- you can't specify just one.<br />
<br />
Then you would simply create a VLAN network interface symlink for <tt>net.mgmt</tt>:<br />
<br />
<console># ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.mgmt<br />
# ##i##rc-update add net.mgmt default</console><br />
When the VLAN interface is brought up, it will be named <tt>mgmt</tt>.<br />
<br />
=== Bonding Configuration ===<br />
<br />
Bonding allows you to aggregate multiple network interfaces into a single logical network interface, allowing for benefits in throughput as well as resiliency in the case that an individual interface may go down. This example shows how you would create a bonding interface (<tt>mybond</tt>) with a simple static ip setup, containing two slave devices (<tt>eth0</tt> and <tt>eth1</tt>).<br />
<br />
First, ensure that your kernel is configured to support bonding (the module name is <tt>bonding</tt>) :<br />
<br />
<console><br />
$ ##i##grep CONFIG_BONDING /usr/src/linux/.config<br />
CONFIG_BONDING=m<br />
</console><br />
<br />
You'l want to ensure that CONFIG_BONDING is set to "m" or "y". You can find this kernel configuration option tucked under "Device Drivers" -> "Network Device Support" -> "Bonding driver support".<br />
Be sure that ifenslave is emerged (this package included in Funtoo stage3):<br />
<br />
<console><br />
# ##i##emerge ifenslave<br />
</console><br />
Once bonding is enabled in the kernel, you will need to choose at least two devices to bond together. These will be set up as "slave" interfaces with no IP address.<br />
<br />
<console><br />
# ##i##cd /etc/init.d/<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##ln -s netif.tmpl net.eth1<br />
</console><br />
<br />
Then, configure the slave interfaces by creating <tt>/etc/conf.d/net.eth0</tt> and <tt>/etc/conf.d/net.eth1</tt> with the following contents:<br />
<br />
<pre><br />
template="interface-noip"<br />
</pre><br />
<br />
Now, we will create the bond interface and make <tt>net.eth0</tt> and <tt>net.eth1</tt> slaves of this interface. Note that our bond interface can have any name. To demonstrate this, we will give it the name of "mybond" below:<br />
<br />
<console><br />
# ##i##ln -s netif.tmpl net.mybond<br />
# ##i##rc-update add net.mybond default<br />
</console><br />
<br />
Now we can configure "mybond" using its configuration file <tt>/etc/conf.d/net.mybond</tt>, just as we would a regular interface, except that we specify <tt>slaves</tt>:<br />
<br />
<pre><br />
template="bond"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
slaves="net.eth0 net.eth1"<br />
</pre><br />
<br />
In a bonded configuration, it is common to set the MTU to the maximum possible value supported by hardware to maximize throughput. In order to do this, simply set the MTU option in <tt>/etc/conf.d/net.mybond</tt> to the maximum value supported by your hardware. The network scripts will ensure that this MTU setting is applied to all slave interfaces:<br />
<br />
<pre><br />
mtu=9000<br />
</pre><br />
<br />
=== Bridge Configuration ===<br />
<br />
When hosting virtual machines, it can be convenient to use a bridge setup. This example shows how you would create a bridge (br0) with a simple static ip setup, containing two slave devices (eth0, tap0).<br />
<br />
First, ensure that your kernel is configured to support bridging (the module name is <tt>bridge</tt>) :<br />
<br />
<console><br />
$ ##i##grep CONFIG_BRIDGE /usr/src/linux/.config<br />
CONFIG_BRIDGE=m<br />
CONFIG_BRIDGE_IGMP_SNOOPING=y<br />
</console><br />
<br />
Second, make sure you have the required software installed:<br />
<br />
<console><br />
# ##i##emerge -av bridge-utils usermode-utilities<br />
</console><br />
<br />
Then, create the necessary symlinks for the interfaces and add them to your default runlevel :<br />
<br />
<console><br />
# ##i##cd /etc/init.d/<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##ln -s netif.tmpl net.br0<br />
# ##i##ln -s netif.tmpl net.tap0<br />
# ##i##rc-update add net.br0 default<br />
# ##i##rc-update add net.tap0 default<br />
</console><br />
<br />
Then, configure the slave interface <tt>/etc/conf.d/net.eth0</tt> :<br />
<br />
<pre><br />
template="interface-noip"<br />
</pre><br />
<br />
Then, configure the slave interface <tt>/etc/conf.d/net.tap0</tt> - note you only require group OR user, not both :<br />
<br />
<pre><br />
template="tap"<br />
group="kvm" <br />
user="kvm"<br />
mac_replace="10:20:30:40:50:66"<br />
</pre><br />
<br />
... and the bridge interface <tt>/etc/conf.d/net.br0</tt> :<br />
<br />
<pre><br />
template="bridge"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
slaves="net.eth0 net.tap0"<br />
stp="on"<br />
forwarding=1<br />
</pre><br />
<br />
If you are using dhcpcd, you should ensure that it does not attempt to configure <tt>eth0</tt> or <tt>br0</tt> by adding the following to <tt>/etc/dhcpcd.conf</tt> :<br />
<br />
<pre><br />
# don't attempt to pull an ip address for br0 or its slave device<br />
denyinterfaces eth0 br0<br />
</pre><br />
<br />
=== More Complex Network Configuration ===<br />
<br />
If the standard templates don't work for your needs, simply create a new template -- I recommend starting from the <tt>interface</tt> template for most things:<br />
<br />
<console># ##i##cd /etc/netif.d<br />
# ##i##cp interface custom</console><br />
You can now call whatever commands you need to <tt>/etc/netif.d/custom</tt>. The following shell functions can be defined in a network script:<br />
<br />
==== netif_create ====<br />
<br />
In <tt>netif_create</tt>, you should call any commands to create the interface if it does not yet exist.<br />
<br />
==== netif_depend ====<br />
<br />
In <tt>netif_depend</tt>, you can define dependencies, using the functions <tt>need</tt> and <tt>use</tt>.<br />
<br />
==== netif_pre_up ====<br />
<br />
In <tt>netif_pre_up</tt>, you can define network configuration actions to perform prior to bringing the interface up. You can also ensure certain variables are specified by calling <tt>require var1 [var2...]</tt> here.<br />
<br />
==== netif_post_up====<br />
<br />
In <tt>netif_post_up</tt>, you can define network configuration actions to perform after bringing the interface up.<br />
<br />
==== netif_pre_down ====<br />
<br />
In <tt>netif_pre_down</tt>, you can define network configuration actions to perform prior to bringing the interface down.<br />
<br />
==== netif_post_down ====<br />
<br />
In <tt>netif_post_down</tt>, you can define network configuration actions to perform after bringing the interface down.<br />
<br />
==== netif_destroy ====<br />
<br />
In <tt>netif_destroy</tt>, you can call any commands necessary to destroy/delete the interface if it is dynamic in nature (tun/tap, etc.)<br />
<br />
==== How It Works ====<br />
<br />
You do not specify a function for actually bringing up the interface, because the template-based system does this for you. The template-based system also performs all normal actions required to bring an interface down, so you only need to specify atypical actions that must be performed - such as removing child interfaces or destroying a bridge using <tt>brctl</tt>.<br />
<br />
When you create your own network configuration template, the following capabilities are available for use automatically, as long as the appropriate variables are set in the <tt>/etc/conf.d/netif.&lt;ifname&gt;</tt> file, without requiring any explicit steps on your part:<br />
<br />
* DNS configuration using <tt>domain</tt> and <tt>nameservers</tt> config settings. OpenResolv is used automatically.<br />
* VLAN configuration using auto-naming (<tt>net.ethX.Y</tt>) or via custom naming with <tt>trunk</tt> and <tt>vlan</tt> config settings.<br />
* Default IPv4 gateway and route configuration using the <tt>gateway</tt> and <tt>route</tt> settings.<br />
* Default IPv6 gateway and route configuration using the <tt>gateway6</tt> and <tt>route6</tt> settings.<br />
* MTU configuration using the <tt>mtu</tt> setting.<br />
* Auto-depend (and auto-MTU configuration) of slave interfaces specified using <tt>slaves</tt> setting.<br />
* Renaming of existing network interface (specify MAC address using <tt>macaddr</tt> setting).<br />
<br />
To take advantage of this functionality, simply enable the appropriate variables.<br />
<br />
All other necessary network configuration and dependency behavior should be defined using the <tt>netif_</tt>-prefix functions described above.<br />
<br />
== Wireless Configuration ==<br />
<br />
The recommended approach for setting up Wi-Fi under Funtoo Linux is to use NetworkManager. Steps are provided in the [[Funtoo Linux Installation#Wi-Fi|Wi-Fi section of the Funtoo Linux Installation Guide]].<br />
<br />
== Other Network Configurations ==<br />
<br />
If you have a network configuration template that might be useful to others, please post it to the [https://bugs.funtoo.org bugtracker] so we can review it and possibly incorporate it into Funtoo.<br />
<br />
== License ==<br />
<br />
Funtoo Linux networking scripts are released under the following license:<br />
<br />
{{BSD2 Funtoo|src=http://github.com/funtoo/corenetwork}}<br />
<br />
[[Category:HOWTO]]<br />
[[Category:Projects]]<br />
[[Category:Networking]]<br />
[[Category:Install]]<br />
[[Category:Funtoo features]]<br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Funtoo:Networking/Configuration&diff=26545Funtoo:Networking/Configuration2019-01-08T17:20:27Z<p>Oleg: /* A Gentle Introduction to Funtoo Network Configuration */</p>
<hr />
<div><blockquote>This document explains how to configure your network settings by explaining the network configuration functionality available in Funtoo Linux. Also covered is <tt>dhcpcd 5.x</tt>, Wi-Fi (IEEE 802.11) configuration, and the OpenResolv framework.<br />
</blockquote><br />
<br />
== Introduction ==<br />
__TOC__<br />
Funtoo Linux has its own core network configuration system that differs somewhat from upstream network configuration systems used in [http://www.gentoo.org Gentoo Linux] and [http://roy.marples.name/projects/openrc OpenRC].<br />
<br />
In this document, I will explain the unique additions and changes to the Funtoo network configuration and show you how to use this system to configure your network.<br />
<br />
I'll also explain how to use <code>dhcpcd</code> for managing network interfaces on DHCP-based networks, and will also cover OpenRC stacked runlevel configuration, ''Wi-Fi'' (IEEE 802.11) configuration, and the OpenResolv framework, which is enabled in Funtoo Linux by default.<br />
<br />
== A Gentle Introduction to Funtoo Network Configuration ==<br />
<br />
Before I get into the technical details of configuring your network, it's important to understand that Funtoo Linux has a number of different options available to you for network configuration, with more likely to be added in the future. Each approach is different and has its own strengths and weaknesses, and this is, in my opinion, a good thing.<br />
<br />
=== The Easy (Dynamic) Way ===<br />
<br />
When configuring your network, one option is to skip traditional network configuration and simply rely on DHCP. This is by far the simplest method of configuring your network. If you are on a wired network, no other steps are typically required beyond enabling a DHCP client, and Funtoo Linux includes {{c|dhcpcd 7.x}} by default. <br />
<br />
==== Network Manager, Wicd ====<br />
<br />
If you are going to use a third party package such as [[Network Manager]] or [[Wicd]] to manage your network then you do not need to configure DHCP at all. These packages configure DHCP for you. Simply emerge the package you want to use and start using it.<br />
<br />
==== DHCP-Only Systems ====<br />
<br />
If you are not planning to use a third-party package to manage your network interfaces, it is still extremely easy to set up DHCP networking, especially if you always use DHCP to connect to networks, which is common for desktops or laptops. In this scenario, we can simply enable {{c|dhcpcd}} to run at system startup. It will run in the background and automatically look for DHCP servers on all your network interfaces, and will attempt to lease an IP address from any DHCP servers found. <br />
<br />
If this sounds like what you want to do, then add {{c|dhcpcd}} to your default runlevel as follows:<br />
<br />
<console># ##i##rc-update add dhcpcd default</console><br />
<br />
To enable DHCP immediately, you would follow the previous command with an <code>rc</code> command, which would start the <code>dhcpcd</code> client you just added:<br />
<br />
{{console|body=<br />
###i##openrc<br />
}}<br />
<br />
If you're on a wired network and have the necessary drivers in your kernel, then this should get you going. For wireless networks, more steps are required to utilize your wireless hardware to associate with an access point, which will be covered later in this document. <br />
<br />
===== Tweaking Dhcpcd =====<br />
<br />
For now, it's important to note that {{c|dhcpcd 7.x}} will manage ''all'' available network interfaces by default. If you want to run a DHCP client on ''all but one'' interface, or some other subset of interfaces, you can add the appropriate {{c|denyinterfaces}} or {{c|allowinterfaces}} [[glob pattern]] to {{c|/etc/dhcpcd.conf}}:<br />
<br />
{{file|name=/etc/dhcpcd.conf|lang=bash|body=<br />
# manage all interfaces but eth0 with dhcpcd<br />
denyinterfaces eth0<br />
}}<br />
<br />
This can also be accomplished by modifying {{c|tc/init.d/dhcpcd}} directly and adding {{c|-Z ''ifglob''}} or {{c|-z ''ifglob''}} (the equivalent command-line parameters) to {{c|command_args}}.<br />
<br />
==== Using Funtoo Scripts for DHCP ====<br />
<br />
You can also use the Funtoo Linux networking scripts to start a DHCP client just on a specific interface. This approach is best if you are planning to also do some advanced bridging, bonding or VLAN configuration on your machine along with DHCP, since you will be using the Funtoo Linux networking scripts for that too. <br />
<br />
To use this variant approach, ''don't'' enable {{c|/etc/init.d/dhcpcd}} directly. Instead, use the Funtoo Linux {{c|dhcpcd}} template which will start dhcpcd on only one interface. Below, you will see the steps to do this. This is very similar to how we set up advanced network interfaces, which will be covered later in this documentation:<br />
<br />
{{console|body=<br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##rc-update add net.eth0 default<br />
# ##i##echo template=dhcpcd > /etc/conf.d/net.eth0<br />
# ##i##openrc<br />
}}<br />
<br />
The last command, {{c|openrc}}, causes {{c|net.eth0}} to be started.<br />
<br />
=== Server Network Configuration ===<br />
<br />
For servers and advanced networking scenarios, Funtoo Linux offers its own modular, template-based network configuration system. This system offers a lot of flexibility for configuring network interfaces, essentially serving as a &quot;network interface construction kit.&quot; This system can be used by itself, or even combined with <tt>dhcpcd</tt>, as shown in the previous section.<br />
<br />
Here are the key components of the template-based network configuration system:<br />
<br />
;{{c|/etc/init.d/loopback}}: An init script that configures the localhost interface. This script is always enabled and is part of the boot process.<br />
;{{c|/etc/netif.d}}: This is a directory that contains various network configuration templates. Each of these templates is focused on configuring a particular type of network interface, such as a general static IP-based interface, a bridge interface, a bond interface, etc.<br />
;{{c|/etc/init.d/netif.tmpl}}: This is the master init script for the template-based network configuration system. New interfaces are added to your system by creating '''symbolic links''' to this file in {{c|/etc/init.d}}.<br />
<br />
So, if you wanted to use this system to configure {{c|eth0}} with a static IP address, you would create a {{c|net.eth0}} symlink to {{c|netif.tmpl}} as follows:<br />
<br />
{{console|body=<br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth0<br />
}}<br />
<br />
Then, you would create an {{c|/etc/conf.d/net.eth0}} configuration file that would specify which template to use from the {{c|/etc/netif.d}} directory:<br />
<br />
{{file|name=/etc/conf.d/neteth0|lang=bash|body=<br />
template="interface"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
}}<br />
<br />
To complete our static IP network configuration we would need to:<br />
<br />
<console># ##i##rc-update add net.eth0 default</console><br />
When configuring your own static network interface, one of <tt>ipaddr</tt> or <tt>ipaddrs</tt> is required and should specify the IP address(es) to configure for this interface, in &quot;a.b.c.d/netmask&quot; format. Optional parameters include <tt>gateway</tt>, which defines a default gateway for your entire network, and if set should specify the gateway's IP address. In addition, <tt>domain</tt> and <tt>nameservers</tt> (space-separated if more than one) can be used to specify DNS information for this interface.<br />
<br />
=== Configuration Variables ===<br />
<br />
==== Interface Variables ====<br />
<br />
The <tt>ipaddr</tt> and <tt>ipaddrs</tt> variables are supported by the <tt>interface</tt> and <tt>bridge</tt> templates, and are used to specify a single or multiple IPv4 or IPv6 address(es) for the interface. IP addresses should be specified in 'IP/netmask' format, such as <tt>10.0.0.1/24</tt>. Multiple IP addresses can be specified delimited by whitespace:<br />
<br />
<pre>ipaddrs=&quot;10.0.0.1/24 10.0.0.2/24&quot;</pre><br />
<br />
===== Broadcast Address =====<br />
<br />
By default, a broadcast address will be calculated based on the IP address and network mask. If you need to manually specify a broadcast address, use the following format for your IP address:<br />
<br />
<pre><br />
ipaddrs="10.0.0.1/24;broadcast=10.0.1.255 10.0.0.2/24"<br />
</pre><br />
<br />
===== Not Specifying An Address =====<br />
<br />
Note that in some cases, you may choose to '''not''' specify <tt>ipaddr</tt> or <tt>ipaddrs</tt> for a <tt>bridge</tt> template. That is allowed. If you don't want to specify an IP address for a regular interface, you can choose to use the <tt>interface</tt> template without an IP address specified in the config, or use the <tt>interface-noip</tt> template instead, for the sake of clarity.<br />
<br />
===== Viewing All Configured IP Addresses =====<br />
<br />
Also note that if you specify multiple IPv4 addresses, <tt>ifconfig</tt> will only show the first IP address. To view all IP addresses associated with the interface, use the <tt>ip addr show</tt> command.<br />
<br />
=== General Variables ===<br />
<br />
The following variables are enabled by default for all network scripts, and if specified will trigger a corresponding configuration action:<br />
<br />
;<tt>nameservers</tt>: Set DNS nameservers using OpenResolv. Specify multiple IPv4 or IPv6 nameservers like this: &quot;1.2.3.4 1.2.3.5 1.2.3.6&quot;. Please note that OpenResolv treats <tt>127.0.0.1</tt> specially, and it indicates that you are running a local name resolver like <tt>dnsmasq</tt> or <tt>bind</tt>. OpenResolv will ignore all other name servers specified alongside <tt>127.0.0.1</tt>. See <tt>man resolvconf</tt> and <tt>man resolvconf.conf</tt> for additional setup information.<br />
;<tt>search</tt>: Set DNS search information using OpenResolv.<br />
;<tt>domain</tt>: Set DNS domain using OpenResolv.<br />
;<tt>gateway</tt>: Define a default IPv4 gateway on this interface.<br />
;<tt>gateway6</tt>: Define a default IPv6 gateway on this interface.<br />
;<tt>route</tt>: Specify a semi-colon delimited list of IPv4 routes to apply when this interface is brought up. Will be appended to <tt>ip -4 route add</tt>.<br />
;<tt>route6</tt>: Specify a semi-colon delimited list of IPv6 routes to apply when this interface is brought up. Will be appended to <tt>ip -6 route add</tt>.<br />
;<tt>mtu</tt>: Set Maximum Transmit Unit for the interface<br />
;<tt>mac_replace</tt>: Replace existing MAC address with MAC address specified in this variable.<br />
<br />
==== VLAN Variables ====<br />
<br />
VLAN support is enabled by default for all network configuration scripts. If a network script has a name in the format <tt>net.ethX.Y</tt>, then it is assumed to be a VLAN interface referencing trunk <tt>ethX</tt> and VLAN ID <tt>Y</tt>. If you desire a custom name for your VLAN interface, you can name your interface whatever you'd like and specify the following variables in your interface config:<br />
<br />
;<tt>trunk</tt>: VLAN trunk interface, e.g. &quot;net.eth0&quot;<br />
;<tt>vlan</tt>: VLAN id, e.g. &quot;32&quot;<br />
<br />
==== Bridge / Tap Variables ====<br />
<br />
The following variables for configuring a functional bridge interface with optional tap interfaces:<br />
<br />
;<tt>slaves</tt>: Set slave interfaces of this interface (for bridges, etc.) All slaves will automatically be depended upon, and will also automatically have their <tt>mtu</tt> set to that of the current interface, if an <tt>mtu</tt> is specified for the current interface. This setting is required for the <tt>bond</tt> template and optional for the <tt>bridge</tt> template.<br />
;<tt>stp</tt>: Enables Spanning Tree Protocol on a bridge interface like this &quot;stp=on&quot;<br />
;<tt>forwarding</tt>: Enables forwarding on a bridge interface by calling sysctl; as this interface does not exist when sysctl is called by init, we do it here. If this is disabled, your bridge will not forward traffic back out onto the network. useage: &quot;forwarding=1&quot;<br />
<br />
=== OpenResolv and resolv.conf ===<br />
<br />
OpenResolv will be used to set DNS information provided by the <tt>nameservers</tt>, <tt>domain</tt> and <tt>search</tt> variables when an interface is brought up. The OpenResolv framework will add entries to <tt>/etc/resolv.conf</tt>, and will also handle removing these entries when the interface is brought down. This way, <tt>/etc/resolv.conf</tt> should always contain current information and should not need to be manually edited by the system administrator. <tt>dhcpcd</tt> will use OpenResolv for updating system DNS information as well.<br />
<br />
=== Multiple Network Configurations ===<br />
<br />
For information on how to have multiple, independent network configurations, please see [[Stacked Runlevels]].<br />
<br />
=== Alternate Configs ===<br />
If you need to run the same service with different configuration parameters depending upon runlevel, then you'll be happy to know that you can specify runlevel-specific conf.d files by appending a <tt>.<br />
&lt;runlevel&gt;</tt> suffix. In this particular example, we could imagine a situation where we had two child runlevels named <tt>home</tt> and <tt>work</tt>:<br />
<br />
<pre>/etc/conf.d/net.eth0.home<br />
/etc/conf.d/net.eth0.work</pre>Note that this feature works for all init scripts, not just network configuration scripts.<br />
<br />
=== Interface Renaming ===<br />
<br />
Funtoo network scripts now support interface renaming, so you can create an interface called <tt>lan</tt> if you would like. To do this, simply specify the MAC address of the interface you would like to rename using the <tt>macaddr</tt> variable:<br />
<pre>macaddr=&quot;00:15:17:19:b6:a3&quot;</pre>If this MAC address is part of the <tt>net.lan</tt> configuration file, then when this interface starts, whatever interface currently has the MAC address of 00:15:17:19:b6:a3 (i.e. <tt>eth5</tt>) will be renamed to <tt>lan</tt> prior to the interface being brought up, and will show up in <tt>ifconfig</tt> and <tt>ip</tt> commands as being an interface named <tt>lan</tt>. It is possible to combine this with the <tt>mac_replace</tt> variable to set a new MAC address, if desired.<br />
<br />
=== Basic VLAN Configuration ===<br />
<br />
The standard <tt>interface</tt> template supports VLANs. To use VLAN support, first ensure that your kernel was compiled with VLAN support (the module name is <tt>8021q</tt>) :<br />
<br />
<console><br />
# ##i##grep CONFIG_VLAN /usr/src/linux/.config<br />
CONFIG_VLAN_8021Q=m<br />
CONFIG_VLAN_8021Q_GVRP=y<br />
</console><br />
<br />
Then, configure the trunk interface using the <tt>interface-noip</tt> template. Assuming <tt>eth1</tt> is trunked, you would create the file <tt>/etc/conf.d/net.eth1</tt> with the following contents:<br />
<br />
<pre>template=&quot;interface-noip&quot;</pre><br />
<br />
Then, create a network interface symlink for the trunk and add it to your default runlevel:<br />
<br />
<console><br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth1<br />
# ##i##rc-update add net.eth1 default<br />
</console><br />
<br />
Now, assuming you wanted to configure a VLAN of 32, you would create a config file named <tt>/etc/conf.d/net.eth1.32</tt> that looks something like this:<br />
<br />
<pre><br />
template=&quot;interface&quot;<br />
ipaddr=&quot;1.2.3.4/24&quot;<br />
gateway=&quot;1.2.3.1&quot;# etc...<br />
</pre><br />
<br />
Then, create a VLAN network interface symlink and add it to your default runlevel:<br />
<br />
<console><br />
# ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.eth1.32<br />
# ##i##rc-update add net.eth1.32 default<br />
</console><br />
<br />
The Funtoo network configuration scripts will automatically recognize the filename <tt>net.eth1.32</tt> as being VLAN 32 of trunk interface <tt>net.eth1</tt>.<br />
<br />
When the VLAN interface is brought up, it will be named <tt>eth1.32</tt>.<br />
<br />
=== Custom VLAN Names ===<br />
<br />
However, sometimes you may want to turn off automatic file-based VLAN naming and give your VLAN interface a custom name, such as <tt>mgmt</tt>. To do this, you would set up the trunk interface in the exact same way as described above, but instead of creating a <tt>net.eth1.32</tt> interface, you would create a <tt>net.mgmt</tt> interface, and specify <tt>vlan</tt> and <tt>trunk</tt> in the <tt>/etc/conf.d/net.mgmt</tt> config file, as follows:<br />
<br />
<pre>template=&quot;interface&quot;<br />
vlan=&quot;32&quot;<br />
trunk=&quot;net.eth1&quot;<br />
ipaddr=&quot;1.2.3.4/24&quot;<br />
gateway=&quot;1.2.3.1&quot;<br />
# etc...</pre><br />
When you specify <tt>trunk</tt> and <tt>vlan</tt> in the interface config file, filename-based auto-detecting of VLAN ID and trunk is disabled. Both <tt>trunk</tt> and <tt>vlan</tt> must be specified -- you can't specify just one.<br />
<br />
Then you would simply create a VLAN network interface symlink for <tt>net.mgmt</tt>:<br />
<br />
<console># ##i##cd /etc/init.d<br />
# ##i##ln -s netif.tmpl net.mgmt<br />
# ##i##rc-update add net.mgmt default</console><br />
When the VLAN interface is brought up, it will be named <tt>mgmt</tt>.<br />
<br />
=== Bonding Configuration ===<br />
<br />
Bonding allows you to aggregate multiple network interfaces into a single logical network interface, allowing for benefits in throughput as well as resiliency in the case that an individual interface may go down. This example shows how you would create a bonding interface (<tt>mybond</tt>) with a simple static ip setup, containing two slave devices (<tt>eth0</tt> and <tt>eth1</tt>).<br />
<br />
First, ensure that your kernel is configured to support bonding (the module name is <tt>bonding</tt>) :<br />
<br />
<console><br />
$ ##i##grep CONFIG_BONDING /usr/src/linux/.config<br />
CONFIG_BONDING=m<br />
</console><br />
<br />
You'l want to ensure that CONFIG_BONDING is set to "m" or "y". You can find this kernel configuration option tucked under "Device Drivers" -> "Network Device Support" -> "Bonding driver support".<br />
Be sure that ifenslave is emerged (this package included in Funtoo stage3):<br />
<br />
<console><br />
# ##i##emerge ifenslave<br />
</console><br />
Once bonding is enabled in the kernel, you will need to choose at least two devices to bond together. These will be set up as "slave" interfaces with no IP address.<br />
<br />
<console><br />
# ##i##cd /etc/init.d/<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##ln -s netif.tmpl net.eth1<br />
</console><br />
<br />
Then, configure the slave interfaces by creating <tt>/etc/conf.d/net.eth0</tt> and <tt>/etc/conf.d/net.eth1</tt> with the following contents:<br />
<br />
<pre><br />
template="interface-noip"<br />
</pre><br />
<br />
Now, we will create the bond interface and make <tt>net.eth0</tt> and <tt>net.eth1</tt> slaves of this interface. Note that our bond interface can have any name. To demonstrate this, we will give it the name of "mybond" below:<br />
<br />
<console><br />
# ##i##ln -s netif.tmpl net.mybond<br />
# ##i##rc-update add net.mybond default<br />
</console><br />
<br />
Now we can configure "mybond" using its configuration file <tt>/etc/conf.d/net.mybond</tt>, just as we would a regular interface, except that we specify <tt>slaves</tt>:<br />
<br />
<pre><br />
template="bond"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
slaves="net.eth0 net.eth1"<br />
</pre><br />
<br />
In a bonded configuration, it is common to set the MTU to the maximum possible value supported by hardware to maximize throughput. In order to do this, simply set the MTU option in <tt>/etc/conf.d/net.mybond</tt> to the maximum value supported by your hardware. The network scripts will ensure that this MTU setting is applied to all slave interfaces:<br />
<br />
<pre><br />
mtu=9000<br />
</pre><br />
<br />
=== Bridge Configuration ===<br />
<br />
When hosting virtual machines, it can be convenient to use a bridge setup. This example shows how you would create a bridge (br0) with a simple static ip setup, containing two slave devices (eth0, tap0).<br />
<br />
First, ensure that your kernel is configured to support bridging (the module name is <tt>bridge</tt>) :<br />
<br />
<console><br />
$ ##i##grep CONFIG_BRIDGE /usr/src/linux/.config<br />
CONFIG_BRIDGE=m<br />
CONFIG_BRIDGE_IGMP_SNOOPING=y<br />
</console><br />
<br />
Second, make sure you have the required software installed:<br />
<br />
<console><br />
# ##i##emerge -av bridge-utils usermode-utilities<br />
</console><br />
<br />
Then, create the necessary symlinks for the interfaces and add them to your default runlevel :<br />
<br />
<console><br />
# ##i##cd /etc/init.d/<br />
# ##i##ln -s netif.tmpl net.eth0<br />
# ##i##ln -s netif.tmpl net.br0<br />
# ##i##ln -s netif.tmpl net.tap0<br />
# ##i##rc-update add net.br0 default<br />
# ##i##rc-update add net.tap0 default<br />
</console><br />
<br />
Then, configure the slave interface <tt>/etc/conf.d/net.eth0</tt> :<br />
<br />
<pre><br />
template="interface-noip"<br />
</pre><br />
<br />
Then, configure the slave interface <tt>/etc/conf.d/net.tap0</tt> - note you only require group OR user, not both :<br />
<br />
<pre><br />
template="tap"<br />
group="kvm" <br />
user="kvm"<br />
mac_replace="10:20:30:40:50:66"<br />
</pre><br />
<br />
... and the bridge interface <tt>/etc/conf.d/net.br0</tt> :<br />
<br />
<pre><br />
template="bridge"<br />
ipaddr="10.0.1.200/24"<br />
gateway="10.0.1.1"<br />
nameservers="10.0.1.1 10.0.1.2"<br />
domain="funtoo.org"<br />
slaves="net.eth0 net.tap0"<br />
stp="on"<br />
forwarding=1<br />
</pre><br />
<br />
If you are using dhcpcd, you should ensure that it does not attempt to configure <tt>eth0</tt> or <tt>br0</tt> by adding the following to <tt>/etc/dhcpcd.conf</tt> :<br />
<br />
<pre><br />
# don't attempt to pull an ip address for br0 or its slave device<br />
denyinterfaces eth0 br0<br />
</pre><br />
<br />
=== More Complex Network Configuration ===<br />
<br />
If the standard templates don't work for your needs, simply create a new template -- I recommend starting from the <tt>interface</tt> template for most things:<br />
<br />
<console># ##i##cd /etc/netif.d<br />
# ##i##cp interface custom</console><br />
You can now call whatever commands you need to <tt>/etc/netif.d/custom</tt>. The following shell functions can be defined in a network script:<br />
<br />
==== netif_create ====<br />
<br />
In <tt>netif_create</tt>, you should call any commands to create the interface if it does not yet exist.<br />
<br />
==== netif_depend ====<br />
<br />
In <tt>netif_depend</tt>, you can define dependencies, using the functions <tt>need</tt> and <tt>use</tt>.<br />
<br />
==== netif_pre_up ====<br />
<br />
In <tt>netif_pre_up</tt>, you can define network configuration actions to perform prior to bringing the interface up. You can also ensure certain variables are specified by calling <tt>require var1 [var2...]</tt> here.<br />
<br />
==== netif_post_up====<br />
<br />
In <tt>netif_post_up</tt>, you can define network configuration actions to perform after bringing the interface up.<br />
<br />
==== netif_pre_down ====<br />
<br />
In <tt>netif_pre_down</tt>, you can define network configuration actions to perform prior to bringing the interface down.<br />
<br />
==== netif_post_down ====<br />
<br />
In <tt>netif_post_down</tt>, you can define network configuration actions to perform after bringing the interface down.<br />
<br />
==== netif_destroy ====<br />
<br />
In <tt>netif_destroy</tt>, you can call any commands necessary to destroy/delete the interface if it is dynamic in nature (tun/tap, etc.)<br />
<br />
==== How It Works ====<br />
<br />
You do not specify a function for actually bringing up the interface, because the template-based system does this for you. The template-based system also performs all normal actions required to bring an interface down, so you only need to specify atypical actions that must be performed - such as removing child interfaces or destroying a bridge using <tt>brctl</tt>.<br />
<br />
When you create your own network configuration template, the following capabilities are available for use automatically, as long as the appropriate variables are set in the <tt>/etc/conf.d/netif.&lt;ifname&gt;</tt> file, without requiring any explicit steps on your part:<br />
<br />
* DNS configuration using <tt>domain</tt> and <tt>nameservers</tt> config settings. OpenResolv is used automatically.<br />
* VLAN configuration using auto-naming (<tt>net.ethX.Y</tt>) or via custom naming with <tt>trunk</tt> and <tt>vlan</tt> config settings.<br />
* Default IPv4 gateway and route configuration using the <tt>gateway</tt> and <tt>route</tt> settings.<br />
* Default IPv6 gateway and route configuration using the <tt>gateway6</tt> and <tt>route6</tt> settings.<br />
* MTU configuration using the <tt>mtu</tt> setting.<br />
* Auto-depend (and auto-MTU configuration) of slave interfaces specified using <tt>slaves</tt> setting.<br />
* Renaming of existing network interface (specify MAC address using <tt>macaddr</tt> setting).<br />
<br />
To take advantage of this functionality, simply enable the appropriate variables.<br />
<br />
All other necessary network configuration and dependency behavior should be defined using the <tt>netif_</tt>-prefix functions described above.<br />
<br />
== Wireless Configuration ==<br />
<br />
The recommended approach for setting up Wi-Fi under Funtoo Linux is to use NetworkManager. Steps are provided in the [[Funtoo Linux Installation#Wi-Fi|Wi-Fi section of the Funtoo Linux Installation Guide]].<br />
<br />
== Other Network Configurations ==<br />
<br />
If you have a network configuration template that might be useful to others, please post it to the [https://bugs.funtoo.org bugtracker] so we can review it and possibly incorporate it into Funtoo.<br />
<br />
== License ==<br />
<br />
Funtoo Linux networking scripts are released under the following license:<br />
<br />
{{BSD2 Funtoo|src=http://github.com/funtoo/corenetwork}}<br />
<br />
[[Category:HOWTO]]<br />
[[Category:Projects]]<br />
[[Category:Networking]]<br />
[[Category:Install]]<br />
[[Category:Funtoo features]]<br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Release_Notes/1.3-release&diff=26495Release Notes/1.3-release2019-01-06T20:28:24Z<p>Oleg: /* kde-plasma update */</p>
<hr />
<div>1.3-release changelog<br />
== Core parts ==<br />
=== Kits system changes === <br />
Addition of new kits, such as {{c|core-server-kit}} and {{c|core-ui-kit}} and reorganization of package-sets, that forms a list of ebuilds in a kit, accordingly. <br />
meta-repos's kits now truly locked by specific SHA of upstream gentoo portage tree, which will now minimize a cross-kit ebuild dependencies issues that was a problem for older releases. Yes, this means that 1.3-release is now fully frozen and you will not get as many daily updates as everyone used to.<br />
Certain kits are independently maintained such as {{c|xorg-kit}} and {{c|gnome-kit}}. They are under strict release control and form a basis for official desktop applications.<br />
{{c|merge-scripts}} that creating the meta-repo drastically rewritten to achieve the goals of 1.3-release<br />
{{c|app-admin/ego-2.7.2}} got many fixes to problems that found by 1.3-release testing. {{c|boot-update}}, a bootloader configuration tool is part of ego now.<br />
<br />
=== Toolchain package updates ===<br />
* {{c|gcc-7.4.1}} update. Very recent upstream GCC version that also has specific fixes such as crossdev support. (https://bugs.funtoo.org/browse/FL-3787)<br />
Notice, that gcc-7 has no official support for '''-march=skylake''', stage3 builds now changed to use '''-march=broadwell'''. This is fixed in gcc-8 and will be part of 1.4-release (https://bugs.funtoo.org/browse/FL-5816)<br />
* binutils-2.31.1 <br />
* sys-libs/glibc-2.27<br />
* gcc-config-2.0<br />
<br />
=== Other core updates ===<br />
* Default kernel changed from debian-sources to debian-sources-lts, which is tested to be better with container support such as LXD. It is an LTS kernel based on kernel.org 4.9 branch, maintained by Debian. This is the kernel you will find in stage3, when doing your installation of Funtoo Linux<br />
*{{c| sys-apps/openrc}} updated to version 0.40.2<br />
*{{c|dev-lang/perl}} updated to version 5.26<br />
* python-3.6 is still a default version in 1.3-release due to python-3.7 incomplete support in many ebuilds.<br />
<br />
== Deprecation of multilib support==<br />
Funtoo profiles now changed to be pure64 (also known as no-multilib). stage3 builds that building with 1.3-release are pure64 by default. For 32-bit applications, such as {{c|wine}} and {{c|steam}} please, follow:<br />
https://www.funtoo.org/32-bit_Chroot<br />
An alternative way of using 32-bit environment with LXD containers is under development, to be announced (https://bugs.funtoo.org/browse/FL-6098)<br />
<br />
== Desktop parts ==<br />
<br />
=== gnome-3.30 update ===<br />
* various build system fixes in gnome core parts and user experience issues fixes. <br />
* optional support for gnome on wayland (https://bugs.funtoo.org/browse/FL-5954). Please, follow, GNOME First Steps wiki, https://www.funtoo.org/GNOME_First_Steps<br />
=== kde-plasma update === <br />
Kde now updated to version 5.14.3. Important change is that QT core ebuilds, which can be found under {{c|dev-qt/*}} category now belong to special {{c|core-ui-kit}}<br />
{{Note|kde-plasma-5 profile now also inherits the gnome mix-in.}}<br />
<br />
=== xfce update === <br />
Xfce-4.13 updates available. {{c|xfce-kit}} had minor package set changes, for example ebuilds that was previously in other kits, now belong to xfce-kit, as well as other minor issues fixes (https://bugs.funtoo.org/browse/FL-5557)<br />
<br />
== Multimedia fixes ==<br />
{{c|gfxcard-nvidia}} mix-in created for better default USE settings for users with nvidia cards. nvidia-drivers now has uvm is enabled by default (https://bugs.funtoo.org/browse/FL-5974). Number of fixes for video editing software included, such as {{c|media-libs/mlt}}, {{c|media-video/shotcut}} new {{c|media-video/flowblade}} ebuild added.<br />
<br />
==Security fixes == <br />
For reported problems that are required for 1.3-release since a freezing of tree happened.<br />
* net-dns/avahi-0.7-r4 (CVE-2017-6519)<br />
* dev-lang/go-1.10.7, dev-lang/go-1.11.4 (CVE-2018-16873, CVE-2018-16874, CVE-2018-16875)<br />
* dev-libs/libxml2-2.9.8-r1 (CVE-2017-8872, CVE-2018-14404, CVE-2018-14567)<br />
* app-arch/tar-1.30-r1 (CVE-2018-20482)<br />
* mail-client/thunderbird-60.4.0 (https://www.mozilla.org/en-US/security/advisories/mfsa2018-28/)<br />
* www-client/firefox-64.0 (https://www.mozilla.org/en-US/security/advisories/mfsa2018-29/)</div>Oleghttps://www.funtoo.org/index.php?title=Release_Notes/1.3-release&diff=26494Release Notes/1.3-release2019-01-06T20:27:30Z<p>Oleg: /* Toolchain package updates */</p>
<hr />
<div>1.3-release changelog<br />
== Core parts ==<br />
=== Kits system changes === <br />
Addition of new kits, such as {{c|core-server-kit}} and {{c|core-ui-kit}} and reorganization of package-sets, that forms a list of ebuilds in a kit, accordingly. <br />
meta-repos's kits now truly locked by specific SHA of upstream gentoo portage tree, which will now minimize a cross-kit ebuild dependencies issues that was a problem for older releases. Yes, this means that 1.3-release is now fully frozen and you will not get as many daily updates as everyone used to.<br />
Certain kits are independently maintained such as {{c|xorg-kit}} and {{c|gnome-kit}}. They are under strict release control and form a basis for official desktop applications.<br />
{{c|merge-scripts}} that creating the meta-repo drastically rewritten to achieve the goals of 1.3-release<br />
{{c|app-admin/ego-2.7.2}} got many fixes to problems that found by 1.3-release testing. {{c|boot-update}}, a bootloader configuration tool is part of ego now.<br />
<br />
=== Toolchain package updates ===<br />
* {{c|gcc-7.4.1}} update. Very recent upstream GCC version that also has specific fixes such as crossdev support. (https://bugs.funtoo.org/browse/FL-3787)<br />
Notice, that gcc-7 has no official support for '''-march=skylake''', stage3 builds now changed to use '''-march=broadwell'''. This is fixed in gcc-8 and will be part of 1.4-release (https://bugs.funtoo.org/browse/FL-5816)<br />
* binutils-2.31.1 <br />
* sys-libs/glibc-2.27<br />
* gcc-config-2.0<br />
<br />
=== Other core updates ===<br />
* Default kernel changed from debian-sources to debian-sources-lts, which is tested to be better with container support such as LXD. It is an LTS kernel based on kernel.org 4.9 branch, maintained by Debian. This is the kernel you will find in stage3, when doing your installation of Funtoo Linux<br />
*{{c| sys-apps/openrc}} updated to version 0.40.2<br />
*{{c|dev-lang/perl}} updated to version 5.26<br />
* python-3.6 is still a default version in 1.3-release due to python-3.7 incomplete support in many ebuilds.<br />
<br />
== Deprecation of multilib support==<br />
Funtoo profiles now changed to be pure64 (also known as no-multilib). stage3 builds that building with 1.3-release are pure64 by default. For 32-bit applications, such as {{c|wine}} and {{c|steam}} please, follow:<br />
https://www.funtoo.org/32-bit_Chroot<br />
An alternative way of using 32-bit environment with LXD containers is under development, to be announced (https://bugs.funtoo.org/browse/FL-6098)<br />
<br />
== Desktop parts ==<br />
<br />
=== gnome-3.30 update ===<br />
* various build system fixes in gnome core parts and user experience issues fixes. <br />
* optional support for gnome on wayland (https://bugs.funtoo.org/browse/FL-5954). Please, follow, GNOME First Steps wiki, https://www.funtoo.org/GNOME_First_Steps<br />
=== kde-plasma update === <br />
Kde now updated to version 5.14.3. Importnt change is that QT core ebuilds, which can be found under {{c|dev-qt/*}} category now belong to special {{c|core-ui-kit}}<br />
{{Note|kde-plasma-5 profile now also inherits the gnome mix-in.}}<br />
<br />
=== xfce update === <br />
Xfce-4.13 updates available. {{c|xfce-kit}} had minor package set changes, for example ebuilds that was previously in other kits, now belong to xfce-kit, as well as other minor issues fixes (https://bugs.funtoo.org/browse/FL-5557)<br />
<br />
== Multimedia fixes ==<br />
{{c|gfxcard-nvidia}} mix-in created for better default USE settings for users with nvidia cards. nvidia-drivers now has uvm is enabled by default (https://bugs.funtoo.org/browse/FL-5974). Number of fixes for video editing software included, such as {{c|media-libs/mlt}}, {{c|media-video/shotcut}} new {{c|media-video/flowblade}} ebuild added.<br />
<br />
==Security fixes == <br />
For reported problems that are required for 1.3-release since a freezing of tree happened.<br />
* net-dns/avahi-0.7-r4 (CVE-2017-6519)<br />
* dev-lang/go-1.10.7, dev-lang/go-1.11.4 (CVE-2018-16873, CVE-2018-16874, CVE-2018-16875)<br />
* dev-libs/libxml2-2.9.8-r1 (CVE-2017-8872, CVE-2018-14404, CVE-2018-14567)<br />
* app-arch/tar-1.30-r1 (CVE-2018-20482)<br />
* mail-client/thunderbird-60.4.0 (https://www.mozilla.org/en-US/security/advisories/mfsa2018-28/)<br />
* www-client/firefox-64.0 (https://www.mozilla.org/en-US/security/advisories/mfsa2018-29/)</div>Oleghttps://www.funtoo.org/index.php?title=Release_Notes/1.3-release&diff=26493Release Notes/1.3-release2019-01-06T20:21:31Z<p>Oleg: /* Other core updates */</p>
<hr />
<div>1.3-release changelog<br />
== Core parts ==<br />
=== Kits system changes === <br />
Addition of new kits, such as {{c|core-server-kit}} and {{c|core-ui-kit}} and reorganization of package-sets, that forms a list of ebuilds in a kit, accordingly. <br />
meta-repos's kits now truly locked by specific SHA of upstream gentoo portage tree, which will now minimize a cross-kit ebuild dependencies issues that was a problem for older releases. Yes, this means that 1.3-release is now fully frozen and you will not get as many daily updates as everyone used to.<br />
Certain kits are independently maintained such as {{c|xorg-kit}} and {{c|gnome-kit}}. They are under strict release control and form a basis for official desktop applications.<br />
{{c|merge-scripts}} that creating the meta-repo drastically rewritten to achieve the goals of 1.3-release<br />
{{c|app-admin/ego-2.7.2}} got many fixes to problems that found by 1.3-release testing. {{c|boot-update}}, a bootloader configuration tool is part of ego now.<br />
<br />
=== Toolchain package updates ===<br />
* {{c|gcc-7.4.1}} update. Very recent upstream GCC version that also has specific fixes such as crossdev support. (https://bugs.funtoo.org/browse/FL-3787)<br />
Notice, that gcc-7 has no official support for '''-march=skylake''', stage3 builds now changed to use '''-march=broadwell'''. This is fixed in gcc-8 and will be part of 1.4-release (https://bugs.funtoo.org/browse/FL-5816)<br />
* binutils-2.31.1 <br />
.<br />
=== Other core updates ===<br />
* Default kernel changed from debian-sources to debian-sources-lts, which is tested to be better with container support such as LXD. It is an LTS kernel based on kernel.org 4.9 branch, maintained by Debian. This is the kernel you will find in stage3, when doing your installation of Funtoo Linux<br />
*{{c| sys-apps/openrc}} updated to version 0.40.2<br />
*{{c|dev-lang/perl}} updated to version 5.26<br />
* python-3.6 is still a default version in 1.3-release due to python-3.7 incomplete support in many ebuilds.<br />
<br />
== Deprecation of multilib support==<br />
Funtoo profiles now changed to be pure64 (also known as no-multilib). stage3 builds that building with 1.3-release are pure64 by default. For 32-bit applications, such as {{c|wine}} and {{c|steam}} please, follow:<br />
https://www.funtoo.org/32-bit_Chroot<br />
An alternative way of using 32-bit environment with LXD containers is under development, to be announced (https://bugs.funtoo.org/browse/FL-6098)<br />
<br />
== Desktop parts ==<br />
<br />
=== gnome-3.30 update ===<br />
* various build system fixes in gnome core parts and user experience issues fixes. <br />
* optional support for gnome on wayland (https://bugs.funtoo.org/browse/FL-5954). Please, follow, GNOME First Steps wiki, https://www.funtoo.org/GNOME_First_Steps<br />
=== kde-plasma update === <br />
Kde now updated to version 5.14.3. Importnt change is that QT core ebuilds, which can be found under {{c|dev-qt/*}} category now belong to special {{c|core-ui-kit}}<br />
{{Note|kde-plasma-5 profile now also inherits the gnome mix-in.}}<br />
<br />
=== xfce update === <br />
Xfce-4.13 updates available. {{c|xfce-kit}} had minor package set changes, for example ebuilds that was previously in other kits, now belong to xfce-kit, as well as other minor issues fixes (https://bugs.funtoo.org/browse/FL-5557)<br />
<br />
== Multimedia fixes ==<br />
{{c|gfxcard-nvidia}} mix-in created for better default USE settings for users with nvidia cards. nvidia-drivers now has uvm is enabled by default (https://bugs.funtoo.org/browse/FL-5974). Number of fixes for video editing software included, such as {{c|media-libs/mlt}}, {{c|media-video/shotcut}} new {{c|media-video/flowblade}} ebuild added.<br />
<br />
==Security fixes == <br />
For reported problems that are required for 1.3-release since a freezing of tree happened.<br />
* net-dns/avahi-0.7-r4 (CVE-2017-6519)<br />
* dev-lang/go-1.10.7, dev-lang/go-1.11.4 (CVE-2018-16873, CVE-2018-16874, CVE-2018-16875)<br />
* dev-libs/libxml2-2.9.8-r1 (CVE-2017-8872, CVE-2018-14404, CVE-2018-14567)<br />
* app-arch/tar-1.30-r1 (CVE-2018-20482)<br />
* mail-client/thunderbird-60.4.0 (https://www.mozilla.org/en-US/security/advisories/mfsa2018-28/)<br />
* www-client/firefox-64.0 (https://www.mozilla.org/en-US/security/advisories/mfsa2018-29/)</div>Oleghttps://www.funtoo.org/index.php?title=Release_Notes/1.3-release&diff=26492Release Notes/1.3-release2019-01-06T20:19:12Z<p>Oleg: /* Other core updates */</p>
<hr />
<div>1.3-release changelog<br />
== Core parts ==<br />
=== Kits system changes === <br />
Addition of new kits, such as {{c|core-server-kit}} and {{c|core-ui-kit}} and reorganization of package-sets, that forms a list of ebuilds in a kit, accordingly. <br />
meta-repos's kits now truly locked by specific SHA of upstream gentoo portage tree, which will now minimize a cross-kit ebuild dependencies issues that was a problem for older releases. Yes, this means that 1.3-release is now fully frozen and you will not get as many daily updates as everyone used to.<br />
Certain kits are independently maintained such as {{c|xorg-kit}} and {{c|gnome-kit}}. They are under strict release control and form a basis for official desktop applications.<br />
{{c|merge-scripts}} that creating the meta-repo drastically rewritten to achieve the goals of 1.3-release<br />
{{c|app-admin/ego-2.7.2}} got many fixes to problems that found by 1.3-release testing. {{c|boot-update}}, a bootloader configuration tool is part of ego now.<br />
<br />
=== Toolchain package updates ===<br />
* {{c|gcc-7.4.1}} update. Very recent upstream GCC version that also has specific fixes such as crossdev support. (https://bugs.funtoo.org/browse/FL-3787)<br />
Notice, that gcc-7 has no official support for '''-march=skylake''', stage3 builds now changed to use '''-march=broadwell'''. This is fixed in gcc-8 and will be part of 1.4-release (https://bugs.funtoo.org/browse/FL-5816)<br />
* binutils-2.31.1 <br />
.<br />
=== Other core updates ===<br />
* Default kernel changed from debian-sources to debian-sources-lts, which is tested to be better with container support such as LXD. It is an LTS kernel based on kernel.org 4.9 branch, maintained by Debian. This is the kernel you will find in stage3, when doing your installation of Funtoo Linux<br />
*{{c| sys-apps/openrc}} updated to version 0.40.2<br />
<br />
== Deprecation of multilib support==<br />
Funtoo profiles now changed to be pure64 (also known as no-multilib). stage3 builds that building with 1.3-release are pure64 by default. For 32-bit applications, such as {{c|wine}} and {{c|steam}} please, follow:<br />
https://www.funtoo.org/32-bit_Chroot<br />
An alternative way of using 32-bit environment with LXD containers is under development, to be announced (https://bugs.funtoo.org/browse/FL-6098)<br />
<br />
== Desktop parts ==<br />
<br />
=== gnome-3.30 update ===<br />
* various build system fixes in gnome core parts and user experience issues fixes. <br />
* optional support for gnome on wayland (https://bugs.funtoo.org/browse/FL-5954). Please, follow, GNOME First Steps wiki, https://www.funtoo.org/GNOME_First_Steps<br />
=== kde-plasma update === <br />
Kde now updated to version 5.14.3. Importnt change is that QT core ebuilds, which can be found under {{c|dev-qt/*}} category now belong to special {{c|core-ui-kit}}<br />
{{Note|kde-plasma-5 profile now also inherits the gnome mix-in.}}<br />
<br />
=== xfce update === <br />
Xfce-4.13 updates available. {{c|xfce-kit}} had minor package set changes, for example ebuilds that was previously in other kits, now belong to xfce-kit, as well as other minor issues fixes (https://bugs.funtoo.org/browse/FL-5557)<br />
<br />
== Multimedia fixes ==<br />
{{c|gfxcard-nvidia}} mix-in created for better default USE settings for users with nvidia cards. nvidia-drivers now has uvm is enabled by default (https://bugs.funtoo.org/browse/FL-5974). Number of fixes for video editing software included, such as {{c|media-libs/mlt}}, {{c|media-video/shotcut}} new {{c|media-video/flowblade}} ebuild added.<br />
<br />
==Security fixes == <br />
For reported problems that are required for 1.3-release since a freezing of tree happened.<br />
* net-dns/avahi-0.7-r4 (CVE-2017-6519)<br />
* dev-lang/go-1.10.7, dev-lang/go-1.11.4 (CVE-2018-16873, CVE-2018-16874, CVE-2018-16875)<br />
* dev-libs/libxml2-2.9.8-r1 (CVE-2017-8872, CVE-2018-14404, CVE-2018-14567)<br />
* app-arch/tar-1.30-r1 (CVE-2018-20482)<br />
* mail-client/thunderbird-60.4.0 (https://www.mozilla.org/en-US/security/advisories/mfsa2018-28/)<br />
* www-client/firefox-64.0 (https://www.mozilla.org/en-US/security/advisories/mfsa2018-29/)</div>Oleghttps://www.funtoo.org/index.php?title=Release_Notes/1.3-release&diff=26491Release Notes/1.3-release2019-01-06T20:18:46Z<p>Oleg: /* Toolchain package updates */</p>
<hr />
<div>1.3-release changelog<br />
== Core parts ==<br />
=== Kits system changes === <br />
Addition of new kits, such as {{c|core-server-kit}} and {{c|core-ui-kit}} and reorganization of package-sets, that forms a list of ebuilds in a kit, accordingly. <br />
meta-repos's kits now truly locked by specific SHA of upstream gentoo portage tree, which will now minimize a cross-kit ebuild dependencies issues that was a problem for older releases. Yes, this means that 1.3-release is now fully frozen and you will not get as many daily updates as everyone used to.<br />
Certain kits are independently maintained such as {{c|xorg-kit}} and {{c|gnome-kit}}. They are under strict release control and form a basis for official desktop applications.<br />
{{c|merge-scripts}} that creating the meta-repo drastically rewritten to achieve the goals of 1.3-release<br />
{{c|app-admin/ego-2.7.2}} got many fixes to problems that found by 1.3-release testing. {{c|boot-update}}, a bootloader configuration tool is part of ego now.<br />
<br />
=== Toolchain package updates ===<br />
* {{c|gcc-7.4.1}} update. Very recent upstream GCC version that also has specific fixes such as crossdev support. (https://bugs.funtoo.org/browse/FL-3787)<br />
Notice, that gcc-7 has no official support for '''-march=skylake''', stage3 builds now changed to use '''-march=broadwell'''. This is fixed in gcc-8 and will be part of 1.4-release (https://bugs.funtoo.org/browse/FL-5816)<br />
* binutils-2.31.1 <br />
.<br />
=== Other core updates ===<br />
* Default kernel changed from debian-sources to debian-sources-lts, which is tested to be better with container support such as LXD. It is an LTS kernel based on kernel.org 4.9 branch, maintained by Debian. This is the kernel you will find in stage3, when doing your installation of Funtoo Linux<br />
* {{c| sys-apps/openrc}} updated to version 0.40.2<br />
<br />
== Deprecation of multilib support==<br />
Funtoo profiles now changed to be pure64 (also known as no-multilib). stage3 builds that building with 1.3-release are pure64 by default. For 32-bit applications, such as {{c|wine}} and {{c|steam}} please, follow:<br />
https://www.funtoo.org/32-bit_Chroot<br />
An alternative way of using 32-bit environment with LXD containers is under development, to be announced (https://bugs.funtoo.org/browse/FL-6098)<br />
<br />
== Desktop parts ==<br />
<br />
=== gnome-3.30 update ===<br />
* various build system fixes in gnome core parts and user experience issues fixes. <br />
* optional support for gnome on wayland (https://bugs.funtoo.org/browse/FL-5954). Please, follow, GNOME First Steps wiki, https://www.funtoo.org/GNOME_First_Steps<br />
=== kde-plasma update === <br />
Kde now updated to version 5.14.3. Importnt change is that QT core ebuilds, which can be found under {{c|dev-qt/*}} category now belong to special {{c|core-ui-kit}}<br />
{{Note|kde-plasma-5 profile now also inherits the gnome mix-in.}}<br />
<br />
=== xfce update === <br />
Xfce-4.13 updates available. {{c|xfce-kit}} had minor package set changes, for example ebuilds that was previously in other kits, now belong to xfce-kit, as well as other minor issues fixes (https://bugs.funtoo.org/browse/FL-5557)<br />
<br />
== Multimedia fixes ==<br />
{{c|gfxcard-nvidia}} mix-in created for better default USE settings for users with nvidia cards. nvidia-drivers now has uvm is enabled by default (https://bugs.funtoo.org/browse/FL-5974). Number of fixes for video editing software included, such as {{c|media-libs/mlt}}, {{c|media-video/shotcut}} new {{c|media-video/flowblade}} ebuild added.<br />
<br />
==Security fixes == <br />
For reported problems that are required for 1.3-release since a freezing of tree happened.<br />
* net-dns/avahi-0.7-r4 (CVE-2017-6519)<br />
* dev-lang/go-1.10.7, dev-lang/go-1.11.4 (CVE-2018-16873, CVE-2018-16874, CVE-2018-16875)<br />
* dev-libs/libxml2-2.9.8-r1 (CVE-2017-8872, CVE-2018-14404, CVE-2018-14567)<br />
* app-arch/tar-1.30-r1 (CVE-2018-20482)<br />
* mail-client/thunderbird-60.4.0 (https://www.mozilla.org/en-US/security/advisories/mfsa2018-28/)<br />
* www-client/firefox-64.0 (https://www.mozilla.org/en-US/security/advisories/mfsa2018-29/)</div>Oleghttps://www.funtoo.org/index.php?title=Release_Notes/1.3-release&diff=26490Release Notes/1.3-release2019-01-06T20:13:18Z<p>Oleg: /* Deprecation of multilib support */</p>
<hr />
<div>1.3-release changelog<br />
== Core parts ==<br />
=== Kits system changes === <br />
Addition of new kits, such as {{c|core-server-kit}} and {{c|core-ui-kit}} and reorganization of package-sets, that forms a list of ebuilds in a kit, accordingly. <br />
meta-repos's kits now truly locked by specific SHA of upstream gentoo portage tree, which will now minimize a cross-kit ebuild dependencies issues that was a problem for older releases. Yes, this means that 1.3-release is now fully frozen and you will not get as many daily updates as everyone used to.<br />
Certain kits are independently maintained such as {{c|xorg-kit}} and {{c|gnome-kit}}. They are under strict release control and form a basis for official desktop applications.<br />
{{c|merge-scripts}} that creating the meta-repo drastically rewritten to achieve the goals of 1.3-release<br />
{{c|app-admin/ego-2.7.2}} got many fixes to problems that found by 1.3-release testing. {{c|boot-update}}, a bootloader configuration tool is part of ego now.<br />
<br />
=== Toolchain package updates ===<br />
* {{c|gcc-7.4.1}} update. Very recent upstream GCC version that also has specific fixes such as crossdev support. (https://bugs.funtoo.org/browse/FL-3787)<br />
Notice, that gcc-7 has no official support for '''-march=skylake''', stage3 builds now changed to use '''-march=broadwell'''. This is fixed in gcc-8 and will be part of 1.4-release (https://bugs.funtoo.org/browse/FL-5816)<br />
* binutils-2.31.1 <br />
* Default kernel changed from debian-sources to debian-sources-lts, which is tested to be better with container support such as LXD. It is an LTS kernel based on kernel.org 4.9 branch, maintained by Debian. This is the kernel you will find in stage3, when doing your installation of Funtoo Linux.<br />
<br />
== Deprecation of multilib support==<br />
Funtoo profiles now changed to be pure64 (also known as no-multilib). stage3 builds that building with 1.3-release are pure64 by default. For 32-bit applications, such as {{c|wine}} and {{c|steam}} please, follow:<br />
https://www.funtoo.org/32-bit_Chroot<br />
An alternative way of using 32-bit environment with LXD containers is under development, to be announced (https://bugs.funtoo.org/browse/FL-6098)<br />
<br />
== Desktop parts ==<br />
<br />
=== gnome-3.30 update ===<br />
* various build system fixes in gnome core parts and user experience issues fixes. <br />
* optional support for gnome on wayland (https://bugs.funtoo.org/browse/FL-5954). Please, follow, GNOME First Steps wiki, https://www.funtoo.org/GNOME_First_Steps<br />
=== kde-plasma update === <br />
Kde now updated to version 5.14.3. Importnt change is that QT core ebuilds, which can be found under {{c|dev-qt/*}} category now belong to special {{c|core-ui-kit}}<br />
{{Note|kde-plasma-5 profile now also inherits the gnome mix-in.}}<br />
<br />
=== xfce update === <br />
Xfce-4.13 updates available. {{c|xfce-kit}} had minor package set changes, for example ebuilds that was previously in other kits, now belong to xfce-kit, as well as other minor issues fixes (https://bugs.funtoo.org/browse/FL-5557)<br />
<br />
== Multimedia fixes ==<br />
{{c|gfxcard-nvidia}} mix-in created for better default USE settings for users with nvidia cards. nvidia-drivers now has uvm is enabled by default (https://bugs.funtoo.org/browse/FL-5974). Number of fixes for video editing software included, such as {{c|media-libs/mlt}}, {{c|media-video/shotcut}} new {{c|media-video/flowblade}} ebuild added.<br />
<br />
==Security fixes == <br />
For reported problems that are required for 1.3-release since a freezing of tree happened.<br />
* net-dns/avahi-0.7-r4 (CVE-2017-6519)<br />
* dev-lang/go-1.10.7, dev-lang/go-1.11.4 (CVE-2018-16873, CVE-2018-16874, CVE-2018-16875)<br />
* dev-libs/libxml2-2.9.8-r1 (CVE-2017-8872, CVE-2018-14404, CVE-2018-14567)<br />
* app-arch/tar-1.30-r1 (CVE-2018-20482)<br />
* mail-client/thunderbird-60.4.0 (https://www.mozilla.org/en-US/security/advisories/mfsa2018-28/)<br />
* www-client/firefox-64.0 (https://www.mozilla.org/en-US/security/advisories/mfsa2018-29/)</div>Oleghttps://www.funtoo.org/index.php?title=Release_Notes/1.3-release&diff=26489Release Notes/1.3-release2019-01-06T20:12:19Z<p>Oleg: Created page with "1.3-release changelog == Core parts == === Kits system changes === Addition of new kits, such as {{c|core-server-kit}} and {{c|core-ui-kit}} and reorganization of package-set..."</p>
<hr />
<div>1.3-release changelog<br />
== Core parts ==<br />
=== Kits system changes === <br />
Addition of new kits, such as {{c|core-server-kit}} and {{c|core-ui-kit}} and reorganization of package-sets, that forms a list of ebuilds in a kit, accordingly. <br />
meta-repos's kits now truly locked by specific SHA of upstream gentoo portage tree, which will now minimize a cross-kit ebuild dependencies issues that was a problem for older releases. Yes, this means that 1.3-release is now fully frozen and you will not get as many daily updates as everyone used to.<br />
Certain kits are independently maintained such as {{c|xorg-kit}} and {{c|gnome-kit}}. They are under strict release control and form a basis for official desktop applications.<br />
{{c|merge-scripts}} that creating the meta-repo drastically rewritten to achieve the goals of 1.3-release<br />
{{c|app-admin/ego-2.7.2}} got many fixes to problems that found by 1.3-release testing. {{c|boot-update}}, a bootloader configuration tool is part of ego now.<br />
<br />
=== Toolchain package updates ===<br />
* {{c|gcc-7.4.1}} update. Very recent upstream GCC version that also has specific fixes such as crossdev support. (https://bugs.funtoo.org/browse/FL-3787)<br />
Notice, that gcc-7 has no official support for '''-march=skylake''', stage3 builds now changed to use '''-march=broadwell'''. This is fixed in gcc-8 and will be part of 1.4-release (https://bugs.funtoo.org/browse/FL-5816)<br />
* binutils-2.31.1 <br />
* Default kernel changed from debian-sources to debian-sources-lts, which is tested to be better with container support such as LXD. It is an LTS kernel based on kernel.org 4.9 branch, maintained by Debian. This is the kernel you will find in stage3, when doing your installation of Funtoo Linux.<br />
<br />
== Deprecation of multilib support==<br />
Funtoo profiles now changed to be pure64 (also known as no-multilib). stage3 builds that building with 1.3-release are pure64 by default. For 32-bit applications, such as {{c|wine}} and {{c|steam}} please, follow:<br />
https://www.funtoo.org/32-bit_Chroot<br />
An alternative way of using 32-bit environment with LXD containers is under develepment, to be announced (https://bugs.funtoo.org/browse/FL-6098)<br />
<br />
== Desktop parts ==<br />
<br />
=== gnome-3.30 update ===<br />
* various build system fixes in gnome core parts and user experience issues fixes. <br />
* optional support for gnome on wayland (https://bugs.funtoo.org/browse/FL-5954). Please, follow, GNOME First Steps wiki, https://www.funtoo.org/GNOME_First_Steps<br />
=== kde-plasma update === <br />
Kde now updated to version 5.14.3. Importnt change is that QT core ebuilds, which can be found under {{c|dev-qt/*}} category now belong to special {{c|core-ui-kit}}<br />
{{Note|kde-plasma-5 profile now also inherits the gnome mix-in.}}<br />
<br />
=== xfce update === <br />
Xfce-4.13 updates available. {{c|xfce-kit}} had minor package set changes, for example ebuilds that was previously in other kits, now belong to xfce-kit, as well as other minor issues fixes (https://bugs.funtoo.org/browse/FL-5557)<br />
<br />
== Multimedia fixes ==<br />
{{c|gfxcard-nvidia}} mix-in created for better default USE settings for users with nvidia cards. nvidia-drivers now has uvm is enabled by default (https://bugs.funtoo.org/browse/FL-5974). Number of fixes for video editing software included, such as {{c|media-libs/mlt}}, {{c|media-video/shotcut}} new {{c|media-video/flowblade}} ebuild added.<br />
<br />
==Security fixes == <br />
For reported problems that are required for 1.3-release since a freezing of tree happened.<br />
* net-dns/avahi-0.7-r4 (CVE-2017-6519)<br />
* dev-lang/go-1.10.7, dev-lang/go-1.11.4 (CVE-2018-16873, CVE-2018-16874, CVE-2018-16875)<br />
* dev-libs/libxml2-2.9.8-r1 (CVE-2017-8872, CVE-2018-14404, CVE-2018-14567)<br />
* app-arch/tar-1.30-r1 (CVE-2018-20482)<br />
* mail-client/thunderbird-60.4.0 (https://www.mozilla.org/en-US/security/advisories/mfsa2018-28/)<br />
* www-client/firefox-64.0 (https://www.mozilla.org/en-US/security/advisories/mfsa2018-29/)</div>Oleghttps://www.funtoo.org/index.php?title=Upgrade_Instructions/1.3-release&diff=26424Upgrade Instructions/1.3-release2018-12-05T04:42:00Z<p>Oleg: perl updates</p>
<hr />
<div><languages/><br />
<translate><br />
<!--T:1--><br />
{{Important|The goal of these instructions is to provide Funtoo Linux users with a reliable, consistent set of instructions for upgrading Funtoo Linux from 1.2 to 1.3. Please assist in ensuring that these instructions are complete and guide users through any potential complications. Since this is a wiki, make changes to the page that are needed to make these instructions 100% reliable. Thank you!}}<br />
<br />
=== Set Release and Sync ===<br />
<br />
{{Important|Make sure you have {{c|ego 2.6.3}} or later installed!}}<br />
<br />
In {{f|/etc/ego.conf}}, set the release to 1.3:<br />
<br />
{{file|name=/etc/ego.conf|body=<br />
[global]<br />
<br />
release = 1.3<br />
}}<br />
<br />
Then, run {{c|ego sync}}:<br />
<br />
{{console|body=<br />
# ##i##ego sync<br />
}}<br />
<!--T:2--><br />
=== Optionally Remove Xorg-Server ===<br />
<br />
1.3-release contains a completely reworked xorg-server, and as such, it is best to ''remove'' your old xorg-server prior to updating:<br />
<br />
{{console|body=<br />
# ##i##emerge -C xorg-server<br />
}}<br />
<!--T:3--><br />
=== Optionally Retarget Pure64 ===<br />
<br />
If you are running a {{c|pure64}} build of Funtoo Linux, you will need to run the following command to update your {{f|/etc/portage/make.profile/parent}} file to no longer reference the {{c|pure64}} arch profile, as it has been deprecated.<br />
Run the following command:<br />
<br />
{{console|body=<br />
# ##i##epro arch x86-64bit<br />
##y##WARNING: Previous value: x86-64bit -- typically, user should not change this.<br />
<br />
=== Enabled Profiles: ===<br />
<br />
arch: ##c##x86-64bit<br />
build: ##c##current<br />
subarch: ##c##intel64-westmere<br />
flavor: ##c##core<br />
mix-ins: ##c##mediaformat-gfx-common<br />
mix-ins: ##c##mediaformat-gfx-extra<br />
<br />
>>> Set arch to x86-64bit.<br />
Updating profiles at /etc/portage/make.profile/parent...<br />
}}<br />
<!--T:4--><br />
=== Relax Deps and Rebuild ===<br />
<br />
Before upgrading, it is a good idea to perform the following commands to relax any existing 32-bit ABI deps so that installed packages don't block necessary updates. Be sure to back up {{f|/var/db/pkg}}, as included in the instructions below, and specify the {{c|find}} commands below exactly -- best to copy and paste:<br />
<br />
{{console|body=<br />
# ##i##cd /var/db<br />
# ##i##cp -a pkg /var/tmp/pkg.bak<br />
# ##i##cd pkg<br />
# ##i##find -iname RDEPEND -exec sed -i -e 's/\[abi_x86_32(-),abi_x86_64(-)]//g' {} \;<br />
# ##i##find -iname RDEPEND -exec sed -i -e 's/,abi_x86_32(-),abi_x86_64(-)]/]/g' {} \;<br />
}}<br />
<br />
Any critical installed packages should no longer depend on ebuilds providing 32-bit ABIs.<br />
<br />
Now, proceed to perform a system upgrade followed by a world upgrade:<br />
<br />
{{console|body=<br />
# ##i##emerge -u1 gcc<br />
# ##i##emerge -u1 glibc<br />
# ##i##emerge -auDN @system --ignore-world<br />
# ##i##emerge -auDN @world<br />
}}<br />
<br />
{{Important|It appears that {{c|dev-lang/go}} will keep rebuilding against a preserved 32-bit glibc, so to fully remove multilib on a system that has {{c|dev-lang/go}} installed, you will need to perform the following steps: {{c|emerge -C dev-lang/go; emerge dev-lang/go}}.}}<br />
<br />
Now, it should be possible to rebuild any necessary packages to get rid of preserved libraries, paying particular attention to any old versions of glibc:<br />
<br />
{{console|body=<br />
# ##i##emerge -av @preserved-rebuild<br />
}}<br />
<!--T:5--><br />
=== Perl Rebuild ===<br />
With 1.3-release {{c|dev-lang/perl}} updated from 5.24 to 5.26 version. It is necessary to rebuild perl modules installed with such major version update, which can be done with:<br />
{{console|body=<br />
# ##i##perl-cleaner --all<br />
}}<br />
<br />
Now, after update and rebuild you will want to either run {{c|etc-update}} or {{c|dispatch-conf}} to perform changes to the configuration files that may happen with ebuild updates:<br />
{{console|body=<br />
# ##i##etc-update<br />
}}<br />
{{Warning|Upgrading to 1.3 will remove any 32-bit compatibility on your system, as we have deprecated multilib support! Please be aware of this before starting the upgrade process.}}<br />
<!--T:6--><br />
<br />
=== Optionally Update Kernel ===<br />
<br />
If you were not using {{c|debian-sources-lts}} before, you may want to upgrade to this kernel. Do this as follows:<br />
<br />
{{console|body=<br />
# ##i##emerge -av debian-sources-lts<br />
# ##i##ego boot update<br />
}}<br />
<br />
Depending on your {{f|/etc/boot.conf}} settings, you may need to tweak the file in order to have {{c|debian-sources-lts}} selected by default.<br />
<br />
{{Important|Remember to rebuild any necessary kernel modules!}}<br />
This can be achieved with:<br />
{{console|body=<br />
# ##i##emerge -av @module-rebuild --exclude debian-sources-lts<br />
}}<br />
<!--T:7--><br />
=== Reboot ===<br />
<br />
At this point, we recommend rebooting the system to ensure you are running in the new environment:<br />
<br />
{{console|body=<br />
# ##i##reboot<br />
}}<br />
<br />
</translate><br />
<br />
[[Category:Official Documentation]]<br />
[[Category:Upgrade Instructions]]</div>Oleghttps://www.funtoo.org/index.php?title=Upgrade_Instructions/1.3-release&diff=26417Upgrade Instructions/1.3-release2018-12-04T21:06:58Z<p>Oleg: /* Relax Deps and Rebuild */</p>
<hr />
<div><languages/><br />
<translate><br />
<!--T:1--><br />
{{Important|The goal of these instructions is to provide Funtoo Linux users with a reliable, consistent set of instructions for upgrading Funtoo Linux from 1.2 to 1.3. Please assist in ensuring that these instructions are complete and guide users through any potential complications. Since this is a wiki, make changes to the page that are needed to make these instructions 100% reliable. Thank you!}}<br />
<br />
=== Set Release and Sync ===<br />
<br />
{{Important|Make sure you have {{c|ego 2.6.3}} or later installed!}}<br />
<br />
In {{f|/etc/ego.conf}}, set the release to 1.3:<br />
<br />
{{file|name=/etc/ego.conf|body=<br />
[global]<br />
<br />
release = 1.3<br />
}}<br />
<br />
Then, run {{c|ego sync}}:<br />
<br />
{{console|body=<br />
# ##i##ego sync<br />
}}<br />
<!--T:2--><br />
=== Optionally Remove Xorg-Server ===<br />
<br />
1.3-release contains a completely reworked xorg-server, and as such, it is best to ''remove'' your old xorg-server prior to updating:<br />
<br />
{{console|body=<br />
# ##i##emerge -C xorg-server<br />
}}<br />
<!--T:3--><br />
=== Optionally Retarget Pure64 ===<br />
<br />
If you are running a {{c|pure64}} build of Funtoo Linux, you will need to run the following command to update your {{f|/etc/portage/make.profile/parent}} file to no longer reference the {{c|pure64}} arch profile, as it has been deprecated.<br />
Run the following command:<br />
<br />
{{console|body=<br />
# ##i##epro arch x86-64bit<br />
##y##WARNING: Previous value: x86-64bit -- typically, user should not change this.<br />
<br />
=== Enabled Profiles: ===<br />
<br />
arch: ##c##x86-64bit<br />
build: ##c##current<br />
subarch: ##c##intel64-westmere<br />
flavor: ##c##core<br />
mix-ins: ##c##mediaformat-gfx-common<br />
mix-ins: ##c##mediaformat-gfx-extra<br />
<br />
>>> Set arch to x86-64bit.<br />
Updating profiles at /etc/portage/make.profile/parent...<br />
}}<br />
<!--T:4--><br />
=== Relax Deps and Rebuild ===<br />
<br />
Before upgrading, it is a good idea to perform the following commands to relax any existing 32-bit ABI deps so that installed packages don't block necessary updates. Be sure to back up {{f|/var/db/pkg}}, as included in the instructions below, and specify the {{c|find}} commands below exactly -- best to copy and paste:<br />
<br />
{{console|body=<br />
# ##i##cd /var/db<br />
# ##i##cp -a pkg /var/tmp/pkg.bak<br />
# ##i##cd pkg<br />
# ##i##find -iname RDEPEND -exec sed -i -e 's/\[abi_x86_32(-),abi_x86_64(-)]//g' {} \;<br />
# ##i##find -iname RDEPEND -exec sed -i -e 's/,abi_x86_32(-),abi_x86_64(-)]/]/g' {} \;<br />
}}<br />
<br />
Any critical installed packages should no longer depend on ebuilds providing 32-bit ABIs.<br />
<br />
Now, proceed to perform a system upgrade followed by a world upgrade:<br />
<br />
{{console|body=<br />
# ##i##emerge -auDN @system<br />
# ##i##emerge -auDN @world<br />
# ##i##emerge -av @preserved-rebuild<br />
}}<br />
You will want to either run {{c|etc-update}} or {{c|dispatch-conf}} to perform changes to the configuration files that may happen with ebuild updates:<br />
{{console|body=<br />
# ##i##etc-update<br />
}}<br />
{{Warning|Upgrading to 1.3 will remove any 32-bit compatibility on your system, as we have deprecated multilib support! Please be aware of this before starting the upgrade process.}}<br />
<!--T:5--><br />
<br />
=== Optionally Update Kernel ===<br />
<br />
If you were not using {{c|debian-sources-lts}} before, you may want to upgrade to this kernel. Do this as follows:<br />
<br />
{{console|body=<br />
# ##i##emerge -av debian-sources-lts<br />
# ##i##ego boot update<br />
}}<br />
<br />
Depending on your {{f|/etc/boot.conf}} settings, you may need to tweak the file in order to have {{c|debian-sources-lts}} selected by default.<br />
<br />
{{Important|Remember to rebuild any necessary kernel modules!}}<br />
This can be achieved with:<br />
{{console|body=<br />
# ##i##emerge -av @module-rebuild --exclude debian-sources-lts<br />
}}<br />
<!--T:6--><br />
=== Reboot ===<br />
<br />
At this point, we recommend rebooting the system to ensure you are running in the new environment:<br />
<br />
{{console|body=<br />
# ##i##reboot<br />
}}<br />
<br />
</translate><br />
<br />
[[Category:Official Documentation]]<br />
[[Category:Upgrade Instructions]]</div>Oleghttps://www.funtoo.org/index.php?title=Upgrade_Instructions/1.3-release&diff=26416Upgrade Instructions/1.3-release2018-12-04T21:06:32Z<p>Oleg: config file changes</p>
<hr />
<div><languages/><br />
<translate><br />
<!--T:1--><br />
{{Important|The goal of these instructions is to provide Funtoo Linux users with a reliable, consistent set of instructions for upgrading Funtoo Linux from 1.2 to 1.3. Please assist in ensuring that these instructions are complete and guide users through any potential complications. Since this is a wiki, make changes to the page that are needed to make these instructions 100% reliable. Thank you!}}<br />
<br />
=== Set Release and Sync ===<br />
<br />
{{Important|Make sure you have {{c|ego 2.6.3}} or later installed!}}<br />
<br />
In {{f|/etc/ego.conf}}, set the release to 1.3:<br />
<br />
{{file|name=/etc/ego.conf|body=<br />
[global]<br />
<br />
release = 1.3<br />
}}<br />
<br />
Then, run {{c|ego sync}}:<br />
<br />
{{console|body=<br />
# ##i##ego sync<br />
}}<br />
<!--T:2--><br />
=== Optionally Remove Xorg-Server ===<br />
<br />
1.3-release contains a completely reworked xorg-server, and as such, it is best to ''remove'' your old xorg-server prior to updating:<br />
<br />
{{console|body=<br />
# ##i##emerge -C xorg-server<br />
}}<br />
<!--T:3--><br />
=== Optionally Retarget Pure64 ===<br />
<br />
If you are running a {{c|pure64}} build of Funtoo Linux, you will need to run the following command to update your {{f|/etc/portage/make.profile/parent}} file to no longer reference the {{c|pure64}} arch profile, as it has been deprecated.<br />
Run the following command:<br />
<br />
{{console|body=<br />
# ##i##epro arch x86-64bit<br />
##y##WARNING: Previous value: x86-64bit -- typically, user should not change this.<br />
<br />
=== Enabled Profiles: ===<br />
<br />
arch: ##c##x86-64bit<br />
build: ##c##current<br />
subarch: ##c##intel64-westmere<br />
flavor: ##c##core<br />
mix-ins: ##c##mediaformat-gfx-common<br />
mix-ins: ##c##mediaformat-gfx-extra<br />
<br />
>>> Set arch to x86-64bit.<br />
Updating profiles at /etc/portage/make.profile/parent...<br />
}}<br />
<!--T:4--><br />
=== Relax Deps and Rebuild ===<br />
<br />
Before upgrading, it is a good idea to perform the following commands to relax any existing 32-bit ABI deps so that installed packages don't block necessary updates. Be sure to back up {{f|/var/db/pkg}}, as included in the instructions below, and specify the {{c|find}} commands below exactly -- best to copy and paste:<br />
<br />
{{console|body=<br />
# ##i##cd /var/db<br />
# ##i##cp -a pkg /var/tmp/pkg.bak<br />
# ##i##cd pkg<br />
# ##i##find -iname RDEPEND -exec sed -i -e 's/\[abi_x86_32(-),abi_x86_64(-)]//g' {} \;<br />
# ##i##find -iname RDEPEND -exec sed -i -e 's/,abi_x86_32(-),abi_x86_64(-)]/]/g' {} \;<br />
}}<br />
<br />
Any critical installed packages should no longer depend on ebuilds providing 32-bit ABIs.<br />
<br />
Now, proceed to perform a system upgrade followed by a world upgrade:<br />
<br />
{{console|body=<br />
# ##i##emerge -auDN @system<br />
# ##i##emerge -auDN @world<br />
# ##i##emerge -av @preserved-rebuild<br />
}}<br />
You will want to either run {{c|cetc-update}} or {{c|dispatch-conf}} to perform changes to the configuration files that may happen with ebuild updates:<br />
{{console|body=<br />
# ##i##etc-update<br />
}}<br />
{{Warning|Upgrading to 1.3 will remove any 32-bit compatibility on your system, as we have deprecated multilib support! Please be aware of this before starting the upgrade process.}}<br />
<!--T:5--><br />
<br />
=== Optionally Update Kernel ===<br />
<br />
If you were not using {{c|debian-sources-lts}} before, you may want to upgrade to this kernel. Do this as follows:<br />
<br />
{{console|body=<br />
# ##i##emerge -av debian-sources-lts<br />
# ##i##ego boot update<br />
}}<br />
<br />
Depending on your {{f|/etc/boot.conf}} settings, you may need to tweak the file in order to have {{c|debian-sources-lts}} selected by default.<br />
<br />
{{Important|Remember to rebuild any necessary kernel modules!}}<br />
This can be achieved with:<br />
{{console|body=<br />
# ##i##emerge -av @module-rebuild --exclude debian-sources-lts<br />
}}<br />
<!--T:6--><br />
=== Reboot ===<br />
<br />
At this point, we recommend rebooting the system to ensure you are running in the new environment:<br />
<br />
{{console|body=<br />
# ##i##reboot<br />
}}<br />
<br />
</translate><br />
<br />
[[Category:Official Documentation]]<br />
[[Category:Upgrade Instructions]]</div>Oleghttps://www.funtoo.org/index.php?title=Upgrade_Instructions/1.3-release&diff=26415Upgrade Instructions/1.3-release2018-12-04T20:38:37Z<p>Oleg: /* Relax Deps and Rebuild */</p>
<hr />
<div><languages/><br />
<translate><br />
<!--T:1--><br />
{{Important|The goal of these instructions is to provide Funtoo Linux users with a reliable, consistent set of instructions for upgrading Funtoo Linux from 1.2 to 1.3. Please assist in ensuring that these instructions are complete and guide users through any potential complications. Since this is a wiki, make changes to the page that are needed to make these instructions 100% reliable. Thank you!}}<br />
<br />
=== Set Release and Sync ===<br />
<br />
{{Important|Make sure you have {{c|ego 2.6.3}} or later installed!}}<br />
<br />
In {{f|/etc/ego.conf}}, set the release to 1.3:<br />
<br />
{{file|name=/etc/ego.conf|body=<br />
[global]<br />
<br />
release = 1.3<br />
}}<br />
<br />
Then, run {{c|ego sync}}:<br />
<br />
{{console|body=<br />
# ##i##ego sync<br />
}}<br />
<!--T:2--><br />
=== Optionally Remove Xorg-Server ===<br />
<br />
1.3-release contains a completely reworked xorg-server, and as such, it is best to ''remove'' your old xorg-server prior to updating:<br />
<br />
{{console|body=<br />
# ##i##emerge -C xorg-server<br />
}}<br />
<!--T:3--><br />
=== Optionally Retarget Pure64 ===<br />
<br />
If you are running a {{c|pure64}} build of Funtoo Linux, you will need to run the following command to update your {{f|/etc/portage/make.profile/parent}} file to no longer reference the {{c|pure64}} arch profile, as it has been deprecated.<br />
Run the following command:<br />
<br />
{{console|body=<br />
# ##i##epro arch x86-64bit<br />
##y##WARNING: Previous value: x86-64bit -- typically, user should not change this.<br />
<br />
=== Enabled Profiles: ===<br />
<br />
arch: ##c##x86-64bit<br />
build: ##c##current<br />
subarch: ##c##intel64-westmere<br />
flavor: ##c##core<br />
mix-ins: ##c##mediaformat-gfx-common<br />
mix-ins: ##c##mediaformat-gfx-extra<br />
<br />
>>> Set arch to x86-64bit.<br />
Updating profiles at /etc/portage/make.profile/parent...<br />
}}<br />
<!--T:4--><br />
=== Relax Deps and Rebuild ===<br />
<br />
Before upgrading, it is a good idea to perform the following commands to relax any existing 32-bit ABI deps so that installed packages don't block necessary updates. Be sure to back up {{f|/var/db/pkg}}, as included in the instructions below, and specify the {{c|find}} commands below exactly -- best to copy and paste:<br />
<br />
{{console|body=<br />
# ##i##cd /var/db<br />
# ##i##cp -a pkg /var/tmp/pkg.bak<br />
# ##i##cd pkg<br />
# ##i##find -iname RDEPEND -exec sed -i -e 's/\[abi_x86_32(-),abi_x86_64(-)]//g' {} \;<br />
# ##i##find -iname RDEPEND -exec sed -i -e 's/,abi_x86_32(-),abi_x86_64(-)]/]/g' {} \;<br />
}}<br />
<br />
Any critical installed packages should no longer depend on ebuilds providing 32-bit ABIs.<br />
<br />
Now, proceed to perform a system upgrade followed by a world upgrade:<br />
<br />
{{console|body=<br />
# ##i##emerge -auDN @system<br />
# ##i##emerge -auDN @world<br />
# ##i##emerge -av @preserved-rebuild<br />
}}<br />
<br />
{{Warning|Upgrading to 1.3 will remove any 32-bit compatibility on your system, as we have deprecated multilib support! Please be aware of this before starting the upgrade process.}}<br />
<!--T:5--><br />
<br />
=== Optionally Update Kernel ===<br />
<br />
If you were not using {{c|debian-sources-lts}} before, you may want to upgrade to this kernel. Do this as follows:<br />
<br />
{{console|body=<br />
# ##i##emerge -av debian-sources-lts<br />
# ##i##ego boot update<br />
}}<br />
<br />
Depending on your {{f|/etc/boot.conf}} settings, you may need to tweak the file in order to have {{c|debian-sources-lts}} selected by default.<br />
<br />
{{Important|Remember to rebuild any necessary kernel modules!}}<br />
This can be achieved with:<br />
{{console|body=<br />
# ##i##emerge -av @module-rebuild --exclude debian-sources-lts<br />
}}<br />
<!--T:6--><br />
=== Reboot ===<br />
<br />
At this point, we recommend rebooting the system to ensure you are running in the new environment:<br />
<br />
{{console|body=<br />
# ##i##reboot<br />
}}<br />
<br />
</translate><br />
<br />
[[Category:Official Documentation]]<br />
[[Category:Upgrade Instructions]]</div>Oleghttps://www.funtoo.org/index.php?title=Module_rebuild_set&diff=26412Module rebuild set2018-12-04T19:17:41Z<p>Oleg: </p>
<hr />
<div>== Introduction ==<br />
This page will try to describe a special portage's set, <code>@module-rebuild</code>. Documentation of this set is missing or incomplete for regular users. This is requested in https://bugs.funtoo.org/browse/FL-3300<br />
== Set's core ==<br />
{{c|@module-rebuild}} is one of built-in portage package sets and is installed by default on every boxes. Currently, it's very simple set which is consist of following:<br />
<br />
{{console|body=<br />
# Installed packages that own files inside /lib/modules.<br />
[module-rebuild]<br />
class = portage.sets.dbapi.OwnerSet<br />
world-candidate = False<br />
files = /lib/modules<br />
}}<br />
As we can see this set calls special class called {{c|portage.sets.dbapi.OwnerSet}} a special handler for sets, which determines list of package(s) that owns file(s) installed on users box. Currently, this class has only one possible variable -- <code>files</code>, which defines what directories or files to look for package that owns that directory/file. One may guess that {{c|@module-rebuild}} gives a list of ebuilds that installed anything into {{c|/lib/modules}}.<br />
A common case for packages that installing into above mentioned directory are 3-rd-party kernel modules, such as {{c|x11-drivers/nvidia-drivers}} or {{c|net-misc/r8168}} and many more.<br />
<br />
== Use case ==<br />
Let's have a look when and why user should use @module-rebuild. So far, it's main aim is to rebuild the packages after kernel updates. Now that if such kernel module as {{c|11-drivers/nvidia-drivers}} installed against older kernel version and newer kernel update has happened and newer kernel is selected, blindly rebooting will result in nvidia module will not start and consequently your X start will fail too. You need to rebuild {{c|x11-drivers/nvidia-drivers}} and also any other modules installed against newly built and selected kernel. Such bulk rebuilds is what this particular portage set allows to do. Real example, update to debian-sources happened and now running {{c|emerge -1 -av @module-rebuild}}:<br />
{{console|body=<br />
w520 / # emerge -1 -av @module-rebuild<br />
These are the packages that would be merged, in order:<br />
<br />
Calculating dependencies... done!<br />
[ebuild R ] sys-kernel/debian-sources-lts-4.9.130::core-kit USE="binary" 0 KiB<br />
[ebuild U ] sys-power/bbswitch-0.8-r1::nokit [0.8::gentoo] 0 KiB<br />
[ebuild U ] sys-power/acpi_call-1.1.0-r2::nokit [1.1.0-r1::gentoo] 0 KiB<br />
[ebuild R ] x11-drivers/nvidia-drivers-381.22:0/381::core-hw-kit [381.22:0/381::core-hw-kit] USE="X driver gtk3 kms (multilib) tools uvm -acpi -compat -pax_kernel -static-libs -wayland" ABI_X86="(64) -32 (-x32)" 0 KiB<br />
<br />
Total: 4 packages (2 upgrades, 2 reinstalls), Size of downloads: 0 KiB<br />
<br />
Would you like to merge these packages? [Yes/No]<br />
}}<br />
But, we have some interesting result which is that this set trying to rebuild fresh kernel. This is special case on Funtoo Linux because default {{c|sys-kernel/debian-sources-lts}} installed a lot of modules into {{c|/lib/modules}}. This seems unnecessary, so when using default kernel, a more correct way would be:<br />
<br />
{{console|body=<br />
w520 / # emerge -1 -av @module-rebuild --exclude debian-sources-lts<br />
<br />
These are the packages that would be merged, in order:<br />
<br />
Calculating dependencies... done!<br />
[ebuild U ] sys-power/bbswitch-0.8-r1::nokit [0.8::gentoo] 0 KiB<br />
[ebuild U ] sys-power/acpi_call-1.1.0-r2::nokit [1.1.0-r1::gentoo] 0 KiB<br />
[ebuild R ] x11-drivers/nvidia-drivers-381.22:0/381::core-hw-kit [381.22:0/381::core-hw-kit] USE="X driver gtk3 kms (multilib) tools uvm -acpi -compat -pax_kernel -static-libs -wayland" ABI_X86="(64) -32 (-x32)" 0 KiB<br />
<br />
Total: 3 packages (2 upgrades, 1 reinstall), Size of downloads: 0 KiB<br />
<br />
Would you like to merge these packages? [Yes/No]<br />
}}<br />
Hopefully, you now armed with knowledge on how to manage kernel updates and kernel modules rebuilds with portage features.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
[[Category:Portage]]<br />
[[Category:Kernel]]</div>Oleghttps://www.funtoo.org/index.php?title=Module_rebuild_set&diff=26411Module rebuild set2018-12-04T19:15:25Z<p>Oleg: /* Use case */</p>
<hr />
<div>== Introduction ==<br />
This page will try to describe a special portage's set, <code>@module-rebuild</code>. Documentation of this set is missing or incomplete for regular users. This is requested in https://bugs.funtoo.org/browse/FL-3300<br />
== Set's core ==<br />
{{c|@module-rebuild}} is one of built-in portage package sets and is installed by default on every boxes. Currently, it's very simple set which is consist of following:<br />
<br />
{{console|body=<br />
# Installed packages that own files inside /lib/modules.<br />
[module-rebuild]<br />
class = portage.sets.dbapi.OwnerSet<br />
world-candidate = False<br />
files = /lib/modules<br />
}}<br />
As we can see this set calls special class called {{c|portage.sets.dbapi.OwnerSet}} a special handler for sets, which determines list of package(s) that owns file(s) installed on users box. Currently, this class has only one possible variable -- <code>files</code>, which defines what directories or files to look for package that owns that directory/file. One may guess that {{c|@module-rebuild}} gives a list of ebuilds that installed anything into {{c|/lib/modules}}.<br />
A common case for packages that installing into above mentioned directory are 3-rd-party kernel modules, such as {{c|x11-drivers/nvidia-drivers}} or {{c|net-misc/r8168}} and many more.<br />
<br />
== Use case ==<br />
Let's have a look when and why user should use @module-rebuild. So far, it's main aim is to rebuild the packages after kernel updates. Now that if such kernel module as {{c|11-drivers/nvidia-drivers}} installed against older kernel version and newer kernel update has happened and newer kernel is selected, blindly rebooting will result in nvidia module will not start and consequently your X start will fail too. You need to rebuild {{c|x11-drivers/nvidia-drivers}} and also any other modules installed against newly built and selected kernel. Such bulk rebuilds is what this particular portage set allows to do. Real example, update to debian-sources happened and now running {{c|emerge -1 -av @module-rebuild}}:<br />
{{console|body=<br />
w520 / # emerge -1 -av @module-rebuild<br />
These are the packages that would be merged, in order:<br />
<br />
Calculating dependencies... done!<br />
[ebuild R ] sys-kernel/debian-sources-lts-4.9.130::core-kit USE="binary" 0 KiB<br />
[ebuild U ] sys-power/bbswitch-0.8-r1::nokit [0.8::gentoo] 0 KiB<br />
[ebuild U ] sys-power/acpi_call-1.1.0-r2::nokit [1.1.0-r1::gentoo] 0 KiB<br />
[ebuild R ] x11-drivers/nvidia-drivers-381.22:0/381::core-hw-kit [381.22:0/381::core-hw-kit] USE="X driver gtk3 kms (multilib) tools uvm -acpi -compat -pax_kernel -static-libs -wayland" ABI_X86="(64) -32 (-x32)" 0 KiB<br />
<br />
Total: 4 packages (2 upgrades, 2 reinstalls), Size of downloads: 0 KiB<br />
<br />
Would you like to merge these packages? [Yes/No]<br />
}}<br />
But, we have some interesting result which is that this set trying to rebuild fresh kernel. This is special case on Funtoo Linux because default {{c|sys-kernel/debian-sources-lts}} installed a lot of modules into {{c|/lib/modules}}. This seems unnecessary, so when using default kernel, a more correct way would be:<br />
<br />
{{console|body=<br />
w520 / # emerge -1 -av @module-rebuild --exclude debian-sources-lts<br />
<br />
These are the packages that would be merged, in order:<br />
<br />
Calculating dependencies... done!<br />
[ebuild U ] sys-power/bbswitch-0.8-r1::nokit [0.8::gentoo] 0 KiB<br />
[ebuild U ] sys-power/acpi_call-1.1.0-r2::nokit [1.1.0-r1::gentoo] 0 KiB<br />
[ebuild R ] x11-drivers/nvidia-drivers-381.22:0/381::xorg-kit [381.22:0/381::funtoo-overlay] USE="X driver gtk3 kms (multilib) tools uvm -acpi -compat -pax_kernel -static-libs -wayland" ABI_X86="(64) -32 (-x32)" 0 KiB<br />
<br />
Total: 3 packages (2 upgrades, 1 reinstall), Size of downloads: 0 KiB<br />
<br />
Would you like to merge these packages? [Yes/No]<br />
}}<br />
Hopefully, you now armed with knowledge on how to manage kernel updates and kernel modules rebuilds with portage features.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
[[Category:Portage]]<br />
[[Category:Kernel]]</div>Oleghttps://www.funtoo.org/index.php?title=Module_rebuild_set&diff=26410Module rebuild set2018-12-04T19:13:36Z<p>Oleg: </p>
<hr />
<div>== Introduction ==<br />
This page will try to describe a special portage's set, <code>@module-rebuild</code>. Documentation of this set is missing or incomplete for regular users. This is requested in https://bugs.funtoo.org/browse/FL-3300<br />
== Set's core ==<br />
{{c|@module-rebuild}} is one of built-in portage package sets and is installed by default on every boxes. Currently, it's very simple set which is consist of following:<br />
<br />
{{console|body=<br />
# Installed packages that own files inside /lib/modules.<br />
[module-rebuild]<br />
class = portage.sets.dbapi.OwnerSet<br />
world-candidate = False<br />
files = /lib/modules<br />
}}<br />
As we can see this set calls special class called {{c|portage.sets.dbapi.OwnerSet}} a special handler for sets, which determines list of package(s) that owns file(s) installed on users box. Currently, this class has only one possible variable -- <code>files</code>, which defines what directories or files to look for package that owns that directory/file. One may guess that {{c|@module-rebuild}} gives a list of ebuilds that installed anything into {{c|/lib/modules}}.<br />
A common case for packages that installing into above mentioned directory are 3-rd-party kernel modules, such as {{c|x11-drivers/nvidia-drivers}} or {{c|net-misc/r8168}} and many more.<br />
<br />
== Use case ==<br />
Let's have a look when and why user should use @module-rebuild. So far, it's main aim is to rebuild the packages after kernel updates. Now that if such kernel module as {{c|11-drivers/nvidia-drivers}} installed against older kernel version and newer kernel update has happened and newer kernel is selected, blindly rebooting will result in nvidia module will not start and consequently your X start will fail too. You need to rebuild {{c|x11-drivers/nvidia-drivers}} and also any other modules installed against newly built and selected kernel. Such bulk rebuilds is what this particular portage set allows to do. Real example, update to debian-sources happened and now running {{c|emerge -1 -av @module-rebuild}}:<br />
{{console|body=<br />
w520 / # emerge -1 -av @module-rebuild<br />
These are the packages that would be merged, in order:<br />
<br />
Calculating dependencies... done!<br />
[ebuild R #] sys-kernel/debian-sources-4.11.11:4.11.11::core-kit USE="binary" 0 KiB<br />
[ebuild U ] sys-power/bbswitch-0.8-r1::nokit [0.8::gentoo] 0 KiB<br />
[ebuild U ] sys-power/acpi_call-1.1.0-r2::nokit [1.1.0-r1::gentoo] 0 KiB<br />
[ebuild R ] x11-drivers/nvidia-drivers-381.22:0/381::xorg-kit [381.22:0/381::funtoo-overlay] USE="X driver gtk3 kms (multilib) tools uvm -acpi -compat -pax_kernel -static-libs -wayland" ABI_X86="(64) -32 (-x32)" 0 KiB<br />
<br />
Total: 4 packages (2 upgrades, 2 reinstalls), Size of downloads: 0 KiB<br />
<br />
Would you like to merge these packages? [Yes/No]<br />
}}<br />
But, we have some interesting result which is that this set trying to rebuild fresh kernel. This is special case on Funtoo Linux because default {{c|sys-kernel/debian-sources}} installed a lot of modules into {{c|/lib/modules}}. This seems unnecessary, so when using default kernel, a more correct way would be:<br />
<br />
{{console|body=<br />
w520 / # emerge -1 -av @module-rebuild --exclude debian-sources-lts<br />
<br />
These are the packages that would be merged, in order:<br />
<br />
Calculating dependencies... done!<br />
[ebuild U ] sys-power/bbswitch-0.8-r1::nokit [0.8::gentoo] 0 KiB<br />
[ebuild U ] sys-power/acpi_call-1.1.0-r2::nokit [1.1.0-r1::gentoo] 0 KiB<br />
[ebuild R ] x11-drivers/nvidia-drivers-381.22:0/381::xorg-kit [381.22:0/381::funtoo-overlay] USE="X driver gtk3 kms (multilib) tools uvm -acpi -compat -pax_kernel -static-libs -wayland" ABI_X86="(64) -32 (-x32)" 0 KiB<br />
<br />
Total: 3 packages (2 upgrades, 1 reinstall), Size of downloads: 0 KiB<br />
<br />
Would you like to merge these packages? [Yes/No]<br />
}}<br />
Hopefully, you now armed with knowledge on how to manage kernel updates and kernel modules rebuilds with portage features.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
[[Category:Portage]]<br />
[[Category:Kernel]]</div>Oleghttps://www.funtoo.org/index.php?title=Upgrade_Instructions/1.3-release&diff=26408Upgrade Instructions/1.3-release2018-12-04T19:08:08Z<p>Oleg: </p>
<hr />
<div><languages/><br />
<translate><br />
<!--T:1--><br />
{{Important|The goal of these instructions is to provide Funtoo Linux users with a reliable, consistent set of instructions for upgrading Funtoo Linux from 1.2 to 1.3. Please assist in ensuring that these instructions are complete and guide users through any potential complications. Since this is a wiki, make changes to the page that are needed to make these instructions 100% reliable. Thank you!}}<br />
<br />
=== Set Release and Sync ===<br />
<br />
In {{f|/etc/ego.conf}}, set the release to 1.3:<br />
<br />
{{file|name=/etc/ego.conf|body=<br />
[global]<br />
<br />
release = 1.3<br />
}}<br />
<br />
Then, run {{c|ego sync}}:<br />
<br />
{{console|body=<br />
# ##i##ego sync<br />
}}<br />
<!--T:2--><br />
=== Optionally Remove Xorg-Server ===<br />
<br />
1.3-release contains a completely reworked xorg-server, and as such, it is best to ''remove'' your old xorg-server prior to updating:<br />
<br />
{{console|body=<br />
# ##i##emerge -C xorg-server<br />
}}<br />
<!--T:3--><br />
=== Optionally Retarget Pure64 ===<br />
<br />
If you are running a {{c|pure64}} build of Funtoo Linux, you will need to run the following command to update your {{f|/etc/portage/make.profile/parent}} file to no longer reference the {{c|pure64}} arch profile, as it has been deprecated.<br />
Run the following command:<br />
<br />
{{console|body=<br />
# ##i##epro arch x86-64bit<br />
##y##WARNING: Previous value: x86-64bit -- typically, user should not change this.<br />
<br />
=== Enabled Profiles: ===<br />
<br />
arch: ##c##x86-64bit<br />
build: ##c##current<br />
subarch: ##c##intel64-westmere<br />
flavor: ##c##core<br />
mix-ins: ##c##mediaformat-gfx-common<br />
mix-ins: ##c##mediaformat-gfx-extra<br />
<br />
>>> Set arch to x86-64bit.<br />
Updating profiles at /etc/portage/make.profile/parent...<br />
}}<br />
<!--T:4--><br />
=== Update World ===<br />
<br />
Now, proceed to perform a system upgrade followed by a world upgrade:<br />
<br />
{{console|body=<br />
# ##i##emerge -auDN @system<br />
# ##i##emerge -auDN @world<br />
}}<br />
<br />
{{Warning|Upgrading to 1.3 will remove any 32-bit compatibility on your system, as we have deprecated multilib support! Please be aware of this before starting the upgrade process.}}<br />
<!--T:5--><br />
=== Optionally Update Kernel ===<br />
<br />
If you were not using {{c|debian-sources-lts}} before, you may want to upgrade to this kernel. Do this as follows:<br />
<br />
{{console|body=<br />
# ##i##emerge -av debian-sources-lts<br />
# ##i##ego boot update<br />
}}<br />
<br />
Depending on your {{f|/etc/boot.conf}} settings, you may need to tweak the file in order to have {{c|debian-sources-lts}} selected by default.<br />
<br />
{{Important|Remember to rebuild any necessary kernel modules!}}<br />
This can be achieved with:<br />
{{console|body=<br />
# ##i##emerge -av @module-rebuild --exclude debian-sources-lts<br />
}}<br />
<!--T:6--><br />
=== Reboot ===<br />
<br />
At this point, we recommend rebooting the system to ensure you are running in the new environment:<br />
<br />
{{console|body=<br />
# ##i##reboot<br />
}}<br />
<br />
</translate><br />
<br />
[[Category:Official Documentation]]<br />
[[Category:Upgrade Instructions]]</div>Oleghttps://www.funtoo.org/index.php?title=Upgrade_Instructions/1.3-release&diff=26407Upgrade Instructions/1.3-release2018-12-04T19:06:45Z<p>Oleg: </p>
<hr />
<div><languages/><br />
<translate><br />
<!--T:1--><br />
{{Important|The goal of these instructions is to provide Funtoo Linux users with a reliable, consistent set of instructions for upgrading Funtoo Linux from 1.2 to 1.3. Please assist in ensuring that these instructions are complete and guide users through any potential complications. Since this is a wiki, make changes to the page that are needed to make these instructions 100% reliable. Thank you!}}<br />
<br />
=== Set Release and Sync ===<br />
<br />
In {{f|/etc/ego.conf}}, set the release to 1.3:<br />
<br />
{{file|name=/etc/ego.conf|body=<br />
[global]<br />
<br />
release = 1.3<br />
}}<br />
<br />
Then, run {{c|ego sync}}:<br />
<br />
{{console|body=<br />
# ##i##ego sync<br />
}}<br />
<!--T:2--><br />
=== Optionally Remove Xorg-Server ===<br />
<br />
1.3-release contains a completely reworked xorg-server, and as such, it is best to ''remove'' your old xorg-server prior to updating:<br />
<br />
{{console|body=<br />
# ##i##emerge -C xorg-server<br />
}}<br />
<!--T:3--><br />
=== Optionally Retarget Pure64 ===<br />
<br />
If you are running a {{c|pure64}} build of Funtoo Linux, you will need to run the following command to update your {{f|/etc/portage/make.profile/parent}} file to no longer reference the {{c|pure64}} arch profile, as it has been deprecated.<br />
Run the following command:<br />
<br />
{{console|body=<br />
# ##i##epro arch x86-64bit<br />
##y##WARNING: Previous value: x86-64bit -- typically, user should not change this.<br />
<br />
=== Enabled Profiles: ===<br />
<br />
arch: ##c##x86-64bit<br />
build: ##c##current<br />
subarch: ##c##intel64-westmere<br />
flavor: ##c##core<br />
mix-ins: ##c##mediaformat-gfx-common<br />
mix-ins: ##c##mediaformat-gfx-extra<br />
<br />
>>> Set arch to x86-64bit.<br />
Updating profiles at /etc/portage/make.profile/parent...<br />
}}<br />
<!--T:4--><br />
=== Update World ===<br />
<br />
Now, proceed to perform a system upgrade followed by a world upgrade:<br />
<br />
{{console|body=<br />
# ##i##emerge -auDN @system<br />
# ##i##emerge -auDN @world<br />
}}<br />
<br />
{{Warning|Upgrading to 1.3 will remove any 32-bit compatibility on your system, as we have deprecated multilib support! Please be aware of this before starting the upgrade process.}}<br />
<!--T:5--><br />
=== Optionally Update Kernel ===<br />
<br />
If you were not using {{c|debian-sources-lts}} before, you may want to upgrade to this kernel. Do this as follows:<br />
<br />
{{console|body=<br />
# ##i##emerge -av debian-sources-lts<br />
# ##i##ego boot update<br />
}}<br />
<br />
Depending on your {{f|/etc/boot.conf}} settings, you may need to tweak the file in order to have {{c|debian-sources-lts}} selected by default.<br />
<br />
{{Important|Remember to rebuild any necessary kernel modules!}}<br />
This can be achieved with:<br />
{{console|body=<br />
# ##i##emerge -av @module-rebuild<br />
}}<br />
<!--T:6--><br />
=== Reboot ===<br />
<br />
At this point, we recommend rebooting the system to ensure you are running in the new environment:<br />
<br />
{{console|body=<br />
# ##i##reboot<br />
}}<br />
<br />
</translate><br />
<br />
[[Category:Official Documentation]]<br />
[[Category:Upgrade Instructions]]</div>Oleghttps://www.funtoo.org/index.php?title=Upgrade_Instructions/1.2-release&diff=25533Upgrade Instructions/1.2-release2018-11-15T19:03:43Z<p>Oleg: </p>
<hr />
<div><languages/><br />
<translate><br />
<!--T:1--><br />
{{Important|The goal of these instructions is to provide Funtoo Linux users with a reliable, consistent set of instructions for upgrading Funtoo Linux from 1.0 to 1.2. Please assist in ensuring that these instructions are complete and guide users through any potential complications. Since this is a wiki, make changes to the page that are needed to make these instructions 100% reliable. Thank you!}}<br />
<br />
<!--T:2--><br />
These instructions will guide you through the process of upgrading your system from Funtoo Linux 1.0 to 1.2. First, please make sure that you have created a backup of your system. If you choose to proceed without<br />
a backup, then you are assuming the risk of a broken system and dealing with fixing it or re-installing. While these install steps are fairly robust, we will<br />
be removing what appear to be unused packages on the system, and while every precaution is taken to avoid breaking packages, in odd cases this could result in some packages being removed that you actually need. Typically<br />
this will not happen, but since the possibility exists, it is best to be prepared for this possibility, particularly on critical systems.<br />
<br />
<!--T:3--><br />
Now, edit your {{f|/var/lib/portage/world}} file. Look for catpkgs (ie. "category/packagename") that you no longer use or need on your system and remove them. Also consider packages you may have installed<br />
with {{c|--oneshot}} that are not in the world file but should be, and add them. Portage will use {{f|/var/lib/portage/world}} as the master list of packages that should be on<br />
your system. We will now look into cleaning up any unnecessary packages that are not in the world set. We want to remove these packages for a couple of reasons. First,<br />
they will not get upgraded with a {{c|@world}} update. Second, because they are not included in {{c|@world}}, they could be outdated and have old and problematic dependencies that could hamper our upgrade, since portage will not want to "break" dependencies for these orphaned packages. Third, when we do an {{c|emerge @preserved-rebuild}}, we may end up rebuilding packages that we don't need. So removing unnecessary packages is a good idea for quite a few reasons.<br />
<br />
<!--T:4--><br />
{{Note|You may be wondering -- what are these packages on my system that are not part of the world set? They could be a number of things. First, they could be build dependencies for certain packages. They could possibly be old slots of packages you already use -- for example, an old version of PHP. The could also be packages that were dependencies of certain packages, but are no longer needed by those packages -- possibly due to changes in {{c|USE}} flags. Often, virtuals are part of this group of packages, and it is generally safe for virtuals to be removed. They will get re-emerged in the future if referenced by an ebuild.}}<br />
<br />
<!--T:5--><br />
Run the following command and carefully review its output. Do not say "y" at this point:<br />
<br />
<!--T:6--><br />
{{console|body=<br />
# ##i##emerge -p --depclean --ignore-soname-deps=n | less<br />
}}<br />
<br />
<!--T:7--><br />
{{Note|The {{c|1=--ignore-soname-deps=n}} option will prevent packages that provide necessary libraries from being removed, even if they appear to be "orphaned." This is an additional safety measure when cleaning dependencies from your system.}}<br />
<br />
<!--T:8--><br />
Now, review the list of packages that are going to be removed. See anything in this list that you know you need? This would indicate that you need to add the cat/pkg<br />
to {{f|/var/lib/portage/world}} before proceeding. Once the list looks OK, type:<br />
<br />
<!--T:9--><br />
{{console|body=<br />
# ##i##emerge -a --depclean --ignore-soname-deps=n<br />
}}<br />
<br />
<!--T:10--><br />
...And type "y" [enter] to remove old packages.<br />
<br />
<!--T:11--><br />
Now, you should have a still-functioning system, but with all "extra" packages removed. Now it is time to upgrade the packages that remain. <br />
<br />
<!--T:12--><br />
Now you will want to run {{c|ego sync}} and upgrade to the latest ego-2.4.x series available. <br />
{{console|body=<br />
# ##i## ego sync<br />
# ##i## emerge -C boot-update<br />
# ##i## emerge -v1 ego<br />
}}<br />
<br />
<!--T:13--><br />
If you have difficulty satisfying deps for it for whatever reason, the following should work:<br />
{{console|body=<br />
# ##i## emerge -C boot-update<br />
# ##i## emerge -v1 --nodeps ego<br />
}}<br />
<br />
<!--T:14--><br />
If you still cannot merge using emerge, the following should work:<br />
{{console|body=<br />
# ##i## cd /var/git/meta-repo/kits/core-kit/app-admin/ego<br />
# ##i## ebuild ego-2.6.3.ebuild merge<br />
}}<br />
Once the new ego is merged, edit your /etc/ego.conf to look like this:<br />
<br />
<!--T:15--><br />
{{file|name=/etc/ego.conf|body=<br />
[global]<br />
<br />
<!--T:16--><br />
release = 1.2<br />
}}<br />
<br />
<!--T:17--><br />
Now, run the following steps as root.<br />
<br />
<!--T:18--><br />
{{console|body=<br />
# ##i##ego sync<br />
}}<br />
<br />
<!--T:19--><br />
This will activate the new 1.2 kits. Now, time to start upgrading:<br />
<br />
<!--T:20--><br />
{{console|body=<br />
# ##i##emerge -u1 gcc<br />
}}<br />
<br />
<!--T:21--><br />
This will upgrade gcc.<br />
<br />
<!--T:22--><br />
This is an ideal time to review the subarch setting of your Funtoo Linux installation. Funtoo Linux 1.2 now has optimizations available for 5th and 6th-generation Intel<br />
Core processors, as well as Ryzen processors. View your current subarch, as well as available subarches, by typing the following command:<br />
<br />
<!--T:23--><br />
{{console|body=<br />
# ##i##ego profile list subarch<br />
<br />
=== ##g##subarch##!g##: === <!--T:24--><br />
<br />
<!--T:25--><br />
amd64-bulldozer, amd64-excavator, amd64-jaguar, amd64-k10<br />
amd64-k8, amd64-k8+sse3, amd64-piledriver, amd64-steamroller<br />
amd64-zen, atom_64, btver1_64, core-avx-i, core2_64, corei7<br />
generic_64, intel64-broadwell, intel64-haswell, intel64-ivybridge<br />
intel64-nehalem, intel64-sandybridge, intel64-silvermont, intel64-skylake<br />
##c##intel64-westmere##!c##*, native_64, nocona, opteron_64, xen-pentium4+sse3_64<br />
<br />
<!--T:26--><br />
#<br />
}}<br />
<br />
<!--T:27--><br />
If desired and supported by your CPU, you can now update your subarch to a more optimized subarch for your hardware. The new subarch profiles available are:<br />
<br />
<!--T:28--><br />
* {{c|intel64-skylake}} - Optimized for Intel Core 6th Generation Processors (see [[intel64-skylake]] for more info.)<br />
* {{c|intel64-broadwell}} - Optimized for Intel Core 5th Generation Processors (see [[intel64-broadwell]] for more info.)<br />
* {{c|amd64-zen}} - Optimized for AMD Ryzen Processors.<br />
<br />
<!--T:29--><br />
Use the {{c|lscpu}} command to view information about your CPU and do a web search for its name to determine what generation CPU it is. Then, the system's subarch<br />
can be changed as follows:<br />
<br />
{{console|body=<br />
# ##i##ego profile subarch intel64-skylake<br />
=== ##g##Enabled Profiles##!g##: === <!--T:30--><br />
<br />
<!--T:31--><br />
##b##arch##!b##: ##c##x86-64bit<br />
##b##build##!b##: ##c##current<br />
##b##subarch##!b##: ##c##intel64-skylake<br />
##b##flavor##!b##: ##c##core<br />
<br />
<!--T:32--><br />
>>> Set subarch to intel64-skylake.<br />
##b##Updating profiles at /etc/portage/make.profile/parent...<br />
<br />
<!--T:33--><br />
#<br />
}}<br />
<br />
<!--T:34--><br />
Now that we have ensured that we have an optimal subarch set for your system, it's time to begin the process of rebuilding critical packages with the new compiler. We will start with glibc. Enter the following command:<br />
<br />
<!--T:35--><br />
{{console|body=<br />
# ##i##emerge -u1 glibc libnsl libtirpc rpcsvc-proto<br />
}}<br />
<br />
<!--T:36--><br />
Glibc and its dependencies/related packages will now be upgraded.<br />
<br />
<!--T:37--><br />
Since moving to 1.2 also includes moving to python-3.6, perform the following steps:<br />
<br />
<!--T:38--><br />
{{console|body=<br />
# ##i##emerge -u1 =dev-lang/python-3.6*<br />
# ##i##emerge -C =dev-lang/python-3.4*<br />
}}<br />
<br />
<!--T:39--><br />
This will ensure that we have python-3.6 ready and installed, and the older python-3.4 removed. Removing python-3.4 is important to ensure that python modules upgrade properly.<br />
<br />
<!--T:40--><br />
Some packages rely on a current ruby being available on the system, and having stale versions on your system can cause problems. To remove these from your system, run:<br />
<br />
<!--T:41--><br />
{{console|body=<br />
# ##i##emerge -C \<=dev-lang/ruby-2.3.0<br />
}}<br />
<br />
<!--T:42--><br />
For upgrading to 1.2, you<br />
have to rebuild all packages, which will ensure that your system is fully optimized with the new gcc.<br />
<br />
<!--T:43--><br />
{{Note|Include the {{c|1=--jobs=3}} (or higher number) option as a parameter to the {{c|emerge}} command if you have sufficient RAM and CPU cores to build several packages in parallel.}}<br />
<br />
<!--T:44--><br />
Here is what you need to run:<br />
<br />
<!--T:45--><br />
{{console|body=<br />
# ##i##emerge --emptytree -a @world<br />
}}<br />
<br />
<!--T:46--><br />
This will fully rebuild all packages on your system. It will take a lot of time, but will ensure everything is freshly rebuilt. Once completed successfully, this will result in an up-to-date system, with potentially better-optimized binaries, benefiting from more recent gcc improvements.<br />
<br />
<!--T:47--><br />
{{Note|If the preceding command fails, you should run 'emerge -uDN1 --keep-going @world' to ensure that all dependencies are fully resolved and rebuilt, then run the 'emerge --emptytree -a @world' command again to ensure that every package is rebuilt with the new gcc compiler.}}<br />
<br />
<!--T:48--><br />
Finally, you will want to either run {{c|etc-update}} or {{c|dispatch-conf}}:<br />
<br />
<!--T:49--><br />
{{console|body=<br />
# ##i##etc-update<br />
}}<br />
<br />
<!--T:50--><br />
If your kernel has been upgraded, make the necessary changes to {{f|/etc/boot.conf}} to make the new kernel the default, and then re-run {{c|ego boot update}}:<br />
<br />
<!--T:51--><br />
{{console|body=<br />
# ##i##ego boot update<br />
}}<br />
<br />
<!--T:52--><br />
Now is a good time to perform a pre-check of any packages that have been installed that may require configuration file updates. One notable example is php-fpm -- you may need to perform the following steps if your system uses php-fpm:<br />
<br />
<!--T:53--><br />
{{console|body=<br />
# ##i##cp /etc/php/fpm-php-7.1/php* /etc/php/fpm-php-7.3/php*<br />
}}<br />
<br />
<!--T:54--><br />
This will ensure that the settings you use for the previously-installed version of php-fpm will be applied to the current version.<br />
<br />
<!--T:55--><br />
{{Note|If you find any other packages that need similar manual steps post-upgrade, please document them here for the benefit of others! Thanks.}}<br />
<br />
<!--T:56--><br />
At this point, the migration to 1.2 should be complete. At a convenient time, reboot your system, and perform a thorough check of all services to ensure they have started<br />
correctly:<br />
<br />
<!--T:57--><br />
{{console|body=<br />
# ##i##reboot<br />
}}<br />
<br />
<!--T:58--><br />
After reboot:<br />
<br />
<!--T:59--><br />
{{console|body=<br />
# ##i##rc-status<br />
}}<br />
<br />
<!--T:60--><br />
Now, perform a final check of any production services to ensure that they are operating properly, by loading web pages, sending test emails, etc.<br />
<br />
<!--T:61--><br />
At this point, you are now upgraded to Funtoo Linux 1.2! Please report any bugs to https://bugs.funtoo.org and let us know of any issues you experience, either as part of the upgrade, related to dependencies, or related to functionality on your upgraded system.<br />
<br />
<!--T:62--><br />
[[Category:Official Documentation]]<br />
</translate></div>Oleghttps://www.funtoo.org/index.php?title=LXD/LXD_Setup&diff=25528LXD/LXD Setup2018-11-15T05:19:11Z<p>Oleg: /* First setup of LXD/Initialisation */</p>
<hr />
<div>== PART III - LXD Setup ==<br />
{{#layout:doc}}<br />
__TOC__<br />
<br />
=== First setup of LXD/Initialisation ===<br />
<br />
Before using LXD for the first time as a user, you should initialize your LXD environment. As stated earlier btrfs (or zfs) is recommended as your storage filesystem.<br />
<br />
{{console|body=<br />
###i## rc-service lxd start<br />
* Starting lxd server ...<br />
###i## lxd init<br />
Do you want to configure a new storage pool (yes/no) [default=yes]? yes<br />
Name of the new storage pool [default=default]: default<br />
Name of the storage backend to use (dir, btrfs, lvm) [default=dir]: btrfs<br />
Create a new BTRFS pool (yes/no) [default=yes]? yes<br />
Would you like to use an existing block device (yes/no) [default=no]? no<br />
Would you like to create a new subvolume for the BTRFS storage pool (yes/no) [default=yes]: yes<br />
Would you like LXD to be available over the network (yes/no) [default=no]? no<br />
Would you like stale cached images to be updated automatically (yes/no) [default=yes]? no<br />
Would you like to create a new network bridge (yes/no) [default=yes]? yes<br />
What should the new bridge be called [default=lxdbr0]? lxdbr0<br />
What IPv4 address should be used (CIDR subnet notation, “auto” or “none”) [default=auto]? auto<br />
What IPv6 address should be used (CIDR subnet notation, “auto” or “none”) [default=auto]? auto<br />
LXD has been successfully configured.<br />
}}<br />
<br />
What this does is it creates btrfs subvolumes like this:<br />
{{console|body=<br />
$##i## btrfs sub list .<br />
ID 260 gen 1047 top level 5 path rootfs<br />
ID 280 gen 1046 top level 260 path var/lib/lxd/storage-pools/default<br />
ID 281 gen 1043 top level 280 path var/lib/lxd/storage-pools/default/containers<br />
ID 282 gen 1044 top level 280 path var/lib/lxd/storage-pools/default/snapshots<br />
ID 283 gen 1045 top level 280 path var/lib/lxd/storage-pools/default/images<br />
ID 284 gen 1046 top level 280 path var/lib/lxd/storage-pools/default/custom<br />
}}<br />
<br />
It also creates new network interface for you:<br />
{{console|body=<br />
$##i## ip a list dev lxdbr0<br />
8: lxdbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000<br />
link/ether d2:9b:70:f2:8f:6f brd ff:ff:ff:ff:ff:ff<br />
inet 10.250.237.1/24 scope global lxdbr0<br />
valid_lft forever preferred_lft forever<br />
inet 169.254.59.23/16 brd 169.254.255.255 scope global lxdbr0<br />
valid_lft forever preferred_lft forever<br />
inet6 fd42:efd8:662e:3184::1/64 scope global<br />
valid_lft forever preferred_lft forever<br />
inet6 fe80::caf5:b7ed:445e:b112/64 scope link<br />
valid_lft forever preferred_lft forever<br />
}}<br />
<br />
And last but not least it also generates iptables rules for you:<br />
{{console|body=<br />
$##i## iptables -L<br />
Chain INPUT (policy ACCEPT)<br />
target prot opt source destination<br />
ACCEPT tcp -- anywhere anywhere tcp dpt:domain /* generated for LXD network lxdbr0 */<br />
ACCEPT udp -- anywhere anywhere udp dpt:domain /* generated for LXD network lxdbr0 */<br />
ACCEPT udp -- anywhere anywhere udp dpt:bootps /* generated for LXD network lxdbr0 */<br />
<br />
Chain FORWARD (policy ACCEPT)<br />
target prot opt source destination<br />
ACCEPT all -- anywhere anywhere /* generated for LXD network lxdbr0 */<br />
ACCEPT all -- anywhere anywhere /* generated for LXD network lxdbr0 */<br />
<br />
Chain OUTPUT (policy ACCEPT)<br />
target prot opt source destination<br />
ACCEPT tcp -- anywhere anywhere tcp spt:domain /* generated for LXD network lxdbr0 */<br />
ACCEPT udp -- anywhere anywhere udp spt:domain /* generated for LXD network lxdbr0 */<br />
ACCEPT udp -- anywhere anywhere udp spt:bootps /* generated for LXD network lxdbr0 */<br />
<br />
$##i## iptables -L -t nat<br />
Chain PREROUTING (policy ACCEPT)<br />
target prot opt source destination<br />
<br />
Chain INPUT (policy ACCEPT)<br />
target prot opt source destination<br />
<br />
Chain OUTPUT (policy ACCEPT)<br />
target prot opt source destination<br />
<br />
Chain POSTROUTING (policy ACCEPT)<br />
target prot opt source destination<br />
MASQUERADE all -- 10.250.237.0/24 !10.250.237.0/24 /* generated for LXD network lxdbr0 */<br />
<br />
$##i## iptables -L -t mangle<br />
Chain PREROUTING (policy ACCEPT)<br />
target prot opt source destination<br />
<br />
Chain INPUT (policy ACCEPT)<br />
target prot opt source destination<br />
<br />
Chain FORWARD (policy ACCEPT)<br />
target prot opt source destination<br />
<br />
Chain OUTPUT (policy ACCEPT)<br />
target prot opt source destination<br />
<br />
Chain POSTROUTING (policy ACCEPT)<br />
target prot opt source destination<br />
CHECKSUM udp -- anywhere anywhere udp dpt:bootpc /* generated for LXD network lxdbr0 */ CHECKSUM fill<br />
}}<br />
<br />
Some other things done by the initialization and starting of the LXD daemon are: <br />
* dnsmasq listening on lxdbr0<br />
* ...<br />
<br />
=== Finishing up the setup of LXD ===<br />
<br />
{{note|Some good instruction for a production server can be found [https://github.com/lxc/lxd/blob/master/doc/production-setup.md here].}}<br />
<br />
There are still some things that you need to do manually. We need to setup subuid and subgid values for our containers to use. And for using non-systemd containers we will also need app-admin/cgmanager so emerge and start it now.<br />
<br />
{{console|body=<br />
###i## rc-update add lxd default<br />
###i## rc-update add lxcfs default<br />
###i## touch /etc/subuid<br />
###i## touch /etc/subgid<br />
###i## usermod --add-subuids 100000-165535 root<br />
###i## usermod --add-subgids 100000-165535 root<br />
###i## /etc/init.d/lxd restart<br />
###i## openrc<br />
}}<br />
<br />
LXD restart is needed to inform the daemon of the uid/gid changes.</div>Oleghttps://www.funtoo.org/index.php?title=Funtoo:Metro&diff=25481Funtoo:Metro2018-10-30T14:51:11Z<p>Oleg: /* Setting up ego */</p>
<hr />
<div><languages/><br />
<translate><br />
<!--T:1--><br />
{{#layout:doc}}{{#widget:AddThis}}Metro is the build system for Funtoo Linux and [[Gentoo Linux]] stages. It automates the bootstrapping process.<br />
<br />
<!--T:2--><br />
This tutorial will take you through installing, setting up and running Metro.<br />
<br />
<!--T:3--><br />
Other [[:Category:Metro|Metro Documentation]] is available.<br />
<br />
= Preface = <!--T:4--> <br />
<br />
{{Note|If you are unfamiliar with how metro works it is recommended that you continue reading this section so you can become familiar with it. If you are already familiar with it, you may wish to use the new [[Metro AutoSetup|autosetup]] script which uses a curses based menu and allows for quickly setting up and running builds base on your choices without requiring any manual steps. Please see the [[Metro AutoSetup]] page for more details}}<br />
<br />
== How Metro Works == <!--T:5--> <br />
<br />
<!--T:6--><br />
Metro is the Funtoo Linux automated build system, and is used to build Funtoo Linux stage tarballs.<br />
<br />
<!--T:7--><br />
Metro cannot create a stage tarball out of thin air. To build a new stage tarball, Metro must use an existing, older stage tarball called a "seed" stage. This seed stage typically is used as the ''build environment'' for creating the stage we want.<br />
<br />
<!--T:8--><br />
Metro can use two kinds of seed stages. Traditionally, Metro has used a stage3 as a seed stage. This stage3 is then used to build a new stage1, which in turn is used to build a new stage2, and then a new stage3. This is generally the most reliable way to build [[Gentoo Linux]] or Funtoo Linux, so it's the recommended approach.<br />
<br />
== Seeds and Build Isolation == <!--T:9--><br />
<br />
<!--T:10--><br />
Another important concept to mention here is something called ''build isolation''. Because Metro creates an isolated build environment, and the build environment is explicitly defined using existing, tangible entities -- a seed stage and a portage snapshot -- you will get consistent, repeatable results. In other words, the same seed stage, portage snapshot and build instructions will generate an essentially identical result, even if you perform the build a month later on someone else's workstation.<br />
<br />
== Local Build == <!--T:11--> <br />
<br />
<!--T:12--><br />
Say you wanted to build a new <tt>pentium4</tt> stage3 tarball. The recommended method of doing this would be to grab an existing <tt>pentium4</tt> stage3 tarball to use as your seed stage. Metro will be told to use this existing <tt>pentium4</tt> stage3 to build a new stage1 for the same <tt>pentium4</tt>. For this process, the generic <tt>pentium4</tt> stage3 would provide the ''build environment'' for creating our new stage1. Then, the new stage1 would serve as the build environment for creating the new <tt>pentium4</tt> stage2. And the new <tt>pentium4</tt> stage2 would serve as the build environment for creating the new <tt>pentium4</tt> stage3.<br />
<br />
<!--T:13--><br />
In the Metro terminology this is called a '''local build''', which means a stage3 of a given architecture is used to seed a brand new build of the same architecture. Incidentally this will be the first exercise we are going to perform in this tutorial.<br />
<br />
<!--T:14--><br />
A week later, you may want to build a brand new <tt>pentium4</tt> stage3 tarball. Rather than starting from the original <tt>pentium4</tt> stage3 again, you'd probably configure Metro to use the most-recently-built <tt>pentium4</tt> stage3 as the seed. Metro has built-in functionality to make this easy, allowing it to easily find and track the most recent stage3 seed available.<br />
<br />
== Remote Build == <!--T:15--> <br />
<br />
<!--T:16--><br />
Metro can also perform '''remote build''', where a stage3 of a different, but binary compatible, architecture is used as a seed to build a different architecture stage3. Consequentiality the second exercise we are going to perform in this tutorial will be to build a <tt>core2 32bit</tt> stage3 tarball from the <tt>pentium4</tt> stage3 tarball we have just built.<br />
<br />
<!--T:17--><br />
TODO: add caveats about what archs can be seeded and what can be not (maybe a table?)<br />
<br />
== Tailored Build == <!--T:18--> <br />
<br />
<!--T:19--><br />
Last, it's also worthy noting that both in <tt>local</tt> and <tt>remote builds</tt>, Metro can be configured to add and/or remove individual packages to the final tarball.<br />
Let's say you can't live without <tt>app-misc/screen</tt>, at the end of this tutorial, we will show how to have your tailored stage3 to include it.<br />
<br />
== Installing Metro == <!--T:20--><br />
<br />
<!--T:21--><br />
'''The recommended and supported method''' is to use the Git repository of Metro. <br />
<br />
<!--T:22--><br />
Ensure that {{Package|dev-vcs/git}}, {{Package|dev-python/requests}} and {{Package|dev-python/boto}} (optional; required for EC2 support) are installed on your system. <br />
<br />
<!--T:23--><br />
{{console|body=<br />
# ##i##emerge dev-vcs/git dev-python/requests dev-python/boto <br />
}}<br />
<br />
<!--T:24--><br />
Next, clone the master git repository as follows:<br />
<br />
<!--T:25--><br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone git://github.com/funtoo/metro.git<br />
# ##i##cp /root/metro/metro.conf ~/.metro<br />
}}<br />
<br />
<!--T:26--><br />
You will now have a directory called {{c|/root/metro}} that contains all the Metro source code.<br />
<br />
<!--T:27--><br />
=== Setting up ego===<br />
Now, we will set the {{c|ego}}, administration tool of Funtoo/Linux. The way it is used with metro is independent from {{c|app-admin/ego}} installed on your box. Setup is easy as follows:<br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone https://github.com/funtoo/ego.git<br />
}}<br />
This way you will have {{c|/root/ego}} directory with {{c|ego}} binary that is then used by metro.<br />
<br />
<!--T:28--><br />
Metro is now installed. It's time to customize it for your local system.<br />
<br />
= Configuring Metro = <!--T:28--><br />
<br />
<!--T:29--><br />
{{Note|Metro is not currently able to build Gentoo stages. See {{Bug|FL-901}}.}}<br />
<br />
<!--T:30--><br />
[[User:Drobbins|Daniel Robbins]] maintains Metro, so it comes pre-configured to successfully build Funtoo Linux releases. Before reading further, you might want to customize some basic settings like the number of concurrent jobs to fit your hardware's capabilities or the directory to use for produced stage archives. This is accomplished by editing {{c|~/.metro}} which is the Metro's master configuration file.<br />
<br />
<!--T:31--><br />
Please note that {{c|path/install}} must point to where metro was installed. Point {{c|path/distfiles}} to where your distfiles reside. Also set {{c|path/mirror/owner}} and {{c|path/mirror/group}} to the owner and group of all the files that will be written to the build repository directory, which by default (as per the configuration file) is at {{c|/home/mirror/funtoo}}. The cache directory normally resides inside the temp directory -- this can be modified as desired. The cache directory can end up holding many cached .tbz2 packages, and eat up a lot of storage. You may want to place the temp directory on faster storage, for faster compile times, and place the cache directory on slower, but more plentiful storage.<br />
<br />
<!--T:32--><br />
{{file|name=.metro|desc=Metro configuration|body=<br />
# Main metro configuration file - these settings need to be tailored to your install:<br />
<br />
<!--T:33--><br />
[section path]<br />
install: /root/metro<br />
tmp: /var/tmp/metro<br />
cache: $[path/tmp]/cache<br />
distfiles: /var/src/distfiles<br />
work: $[path/tmp]/work/$[target/build]/$[target/name]<br />
<br />
<!--T:34--><br />
[section path/mirror]<br />
<br />
<!--T:35--><br />
: /home/mirror/funtoo<br />
owner: root<br />
group: repomgr<br />
dirmode: 775<br />
<br />
<!--T:36--><br />
[section portage]<br />
<br />
<!--T:37--><br />
MAKEOPTS: auto <br />
<br />
<!--T:38--><br />
[section emerge]<br />
<br />
<!--T:39--><br />
options: --jobs=4 --load-average=4 --keep-going=n<br />
<br />
<!--T:40--><br />
# This line should not be modified:<br />
[collect $[path/install]/etc/master.conf]<br />
}}<br />
<br />
== Arch and Subarch == <!--T:41--><br />
<br />
<!--T:42--><br />
In the following example we are creating a pentium4 stage 3 compiled for x86-32bit binary compatibility. Pentium4 is a subarch of the x86-32bit architecture. Once you have metro installed you may find a full list of each subarch in your {{f|/var/git/meta-repo/kits/core-kit/profiles/funtoo-1.0/linux-gnu/arch/x86-32bit/subarch}} directory:<br />
Example:<br />
{{console|body=<br />
# ##i##ls /var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-32bit/subarch/<br />
amd64-k8+sse3_32 athlon-4 athlon-xp core2_32 i486 k6-2 pentium pentium2 pentiumpro<br />
amd64-k8_32 athlon-mp atom_32 generic_32 i686 k6-3 pentium-m pentium3 prescott<br />
athlon athlon-tbird btver1 geode k6 native_32 pentium-mmx pentium4 xen-pentium4+sse3<br />
}}<br />
<br />
64-bit PC profiles can be found in the {{f|/var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-64bit/subarch/}} directory:<br />
{{console|body=<br />
# ##i##ls /var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-64bit/subarch/<br />
amd64-bulldozer amd64-k8+sse3 btver1_64 generic_64 intel64-nehalem native_64<br />
amd64-jaguar amd64-piledriver core-avx-i intel64-broadwell intel64-sandybridge nocona<br />
amd64-k10 amd64-steamroller core2_64 intel64-haswell intel64-silvermont opteron_64<br />
amd64-k8 atom_64 corei7 intel64-ivybridge intel64-westmere xen-pentium4+sse3_64<br />
}}<br />
<br />
= First stages build (local build) = <!--T:43--><br />
<br />
<!--T:44--><br />
To get this all started, we need to bootstrap the process by downloading an initial seed stage3 to use for building and place it in its proper location in {{f|/home/mirror/funtoo}}, so that Metro can find it. We will also need to create some special &quot;control&quot; files in {{f|/home/mirror/funtoo}}, which will allow Metro to understand how it is supposed to proceed.<br />
<br />
== Step 1: Set up pentium4 repository (local build) == <!--T:45--><br />
<br />
<!--T:46--><br />
Assuming we're following the basic steps outlined in the previous section, and building {{f|funtoo-current}} build for the {{f|pentium4}}, using a generic {{f|pentium4}} stage3 as a seed stage, then here the first set of steps we'd perform:<br />
<br />
<!--T:47--><br />
{{console|body=<br />
# ##i##install -d /home/mirror/funtoo/funtoo-current/x86-32bit/pentium4<br />
# ##i##install -d /home/mirror/funtoo/funtoo-current/snapshots<br />
# ##i##cd /home/mirror/funtoo/funtoo-current/x86-32bit/pentium4<br />
# ##i##install -d 2017-10-01<br />
# ##i##cd 2017-10-01<br />
# ##i##wget -c https://build.funtoo.org/funtoo-current/x86-32bit/pentium4/2017-10-01/stage3-pentium4-funtoo-current-2017-10-01.tar.xz<br />
# ##i##cd ..<br />
# ##i##install -d .control/version<br />
# ##i##echo "2017-10-01" > .control/version/stage3<br />
# ##i##install -d .control/strategy<br />
# ##i##echo local > .control/strategy/build<br />
# ##i##echo stage3 > .control/strategy/seed<br />
}}<br />
<br />
<!--T:48--><br />
OK, let's review the steps above. First, we create the directory {{f|/home/mirror/funtoo/funtoo-current/x86-32bit/pentium4}}, which is where Metro will expect to find {{f|funtoo-current}} pentium4 builds -- it is configured to look here by default. Then we create a specially-named directory to house our seed x86 stage3. Again, by default, Metro expects the directory to be named this way. We enter this directory, and download our seed x86 stage3 from funtoo.org. Note that the {{f|2017-10-01}} version stamp matches. Make sure that your directory name matches the stage3 name too. Everything has been set up to match Metro's default filesystem layout.<br />
<br />
<!--T:49--><br />
Next, we go back to the {{f|/home/mirror/metro/funtoo-current/x86-32bit/pentium4}} directory, and inside it, we create a {{f|.control}} directory. This directory and its subdirectories contain special files that Metro references to determine certain aspects of its behavior. The {{f|.control/version/stage3}} file is used by Metro to track the most recently-built stage3 for this particular build and subarch. Metro will automatically update this file with a new version stamp after it successfully builds a new stage3. But because Metro didn't actually ''build'' this stage3, we need to set up the {{f|.control/version/stage3}} file manually. This will allow Metro to find our downloaded stage3 when we set up our pentium4 build to use it as a seed. Also note that Metro will create a similar {{f|.control/version/stage1}} file after it successfully builds an pentium4 funtoo-current stage1.<br />
<br />
<!--T:50--><br />
We also set up {{f|.control/strategy/build}} and {{f|.control/strategy/seed}} files with values of {{f|local}} and {{f|stage3}} respectively. These files define the building strategy Metro will use when we build pentium4 funtoo-current stages. With a build strategy of {{f|local}}, Metro will source its seed stage from funtoo-current pentium4, the current directory. And with a seed strategy of {{f|stage3}}, Metro will use a stage3 as a seed, and use this seed to build a new stage1, stage2 and stage3.<br />
<br />
== Step 2: Building the pentium4 stages == <!--T:51--><br />
<br />
<!--T:52--><br />
Incidentally, if all you wanted to do at this point was to build a new pentium4 funtoo-current stage1/2/3 (plus openvz and vserver templates). You would begin the process by typing:<br />
<br />
<!--T:53--><br />
{{console|body=<br />
# ##i##cd /root/metro<br />
# ##i##scripts/ezbuild.sh funtoo-current x86-32bit pentium4<br />
}}<br />
<br />
<!--T:54--><br />
If you have a slow machine, it could take several hours to be completed because several "heavy" components like gcc or glibc have to be recompiled in each stage. Once a stage has been successfully completed, it is placed in the {{f|"${METRO_MIRROR}/funtoo-current/x32-bit/pentium4/YYYY-MM-DD"}} subdirectory, where {{f|YYYY-MM-DD}} is today's date at the time the {{f|ezbuild.sh}} script was started or the date you put on the ezscript.sh command line.<br />
<br />
= Building for another binary compatible architecture (remote build) = <!--T:55--><br />
<br />
<!--T:56--><br />
As written above, Metro is able to perform '''remote build''' building different architecture stage3 from a binary compatible seeding stage3 (e.g. using a pentium4 stage3 to seed a <tt>Intel Core2 32bits</tt> stage3). <br />
<br />
<!--T:57--><br />
In the Metro terminology this is called a '''remote build''' (a stage 3 of a different, but binary compatible, architecture is used as a seed). <br />
What's not compatible? You can't use a <tt>Sparc</tt> architecture to generate an <tt>x86</tt> or <tt>ARM</tt> based stage and vice-versa. If you use a 32bit stage then you don't want to seed a 64bit build from it. Be sure that you are using a stage from the same architecture that you are trying to seed. Check [http://ftp.osuosl.org/pub/funtoo/funtoo-current/ Funtoo-current FTP Mirror] for a stage that is from the same Architecture that you will be building. <br />
<br />
<!--T:58--><br />
{{Note|Often, one build (ie. funtoo-current) can be used as a seed for another build such as funtoo-stable. However, hardened builds require hardened stages as seeds in order for the build to complete successfully.}}<br />
<br />
== Step 1: Set up Core_2 32bit repository == <!--T:59--><br />
<br />
<!--T:60--><br />
In this example, we're going to use this pentium4 funtoo-current stage3 to seed a new Core_2 32bit funtoo-current build. To get that done, we need to set up the pentium4 build directory as follows:<br />
<br />
<!--T:61--><br />
{{console|body=<br />
# ##i## cd /home/mirror/funtoo/funtoo-current/x86-32bit<br />
# ##i##install -d core2_32<br />
# ##i##cd core2_32<br />
# ##i##install -d .control/strategy<br />
# ##i##echo remote > .control/strategy/build<br />
# ##i##echo stage3 > .control/strategy/seed<br />
# ##i##install -d .control/remote<br />
# ##i##echo funtoo-current > .control/remote/build<br />
# ##i##echo x86-32bit > .control/remote/arch_desc<br />
# ##i##echo pentium4 > .control/remote/subarch<br />
}}<br />
<br />
<!--T:62--><br />
The steps we follow are similar to those we performed for a ''local build'' to set up our pentium4 directory for local build. However, note the differences. We didn't download a stage, because we are going to use the pentium4 stage to build a new Core_2 32bit stage. We also didn't create the <tt>.control/version/stage{1,3}</tt> files because Metro will create them for us after it successfully builds a new stage1 and stage3. We are still using a <tt>stage3</tt> seed strategy, but we've set the build strategy to <tt>remote</tt>, which means that we're going to use a seed stage that's not from this particular subdirectory. Where are we going to get it from? The <tt>.control/remote</tt> directory contains this information, and lets Metro know that it should look for its seed stage3 in the <tt>/home/mirror/funtoo/funtoo-current/x86-32bit/pentium4</tt> directory. Which one will it grab? You guessed it -- the most recently built ''stage3'' (since our seed strategy was set to <tt>stage3</tt>) that has the version stamp of <tt>2010-12-24</tt>, as recorded in <tt>/home/mirror/funtoo-current/x86-32bit/pentium4/.control/version/stage</tt>. Now you can see how all those control files come together to direct Metro to do the right thing.<br />
<br />
<!--T:63--><br />
{{Note|<code>arch_desc</code> should be set to one of: <code>x86-32bit</code>, <code>x86-64bit</code> or <code>pure64</code> for PC-compatible systems. You must use a 32-bit build as a seed for other 32-bit builds, and a 64-bit build as a seed for other 64-bit builds.}}<br />
<br />
== Step 2: Building the Core_2 32bit stages == <!--T:64--><br />
<br />
<!--T:65--><br />
Now, you could start building your new Core_2 32bit stage1/2/3 (plus openvz and vserver templates) by typing the following:<br />
<br />
<!--T:66--><br />
{{console|body=<br />
# ##i##/root/metro/scripts/ezbuild.sh funtoo-current x86-32bit core2_32<br />
}}<br />
<br />
<!--T:67--><br />
In that case, the produced stages are placed in the <tt>/home/mirror/funtoo/funtoo-current/x32-bit/core2_32/YYYY-MM-DD</tt> subdirectory.<br />
<br />
== Step 3: The Next Build == <!--T:68--><br />
<br />
<!--T:69--><br />
At this point, you now have a new Core_2 32bit stage3, built using a "remote" pentium4 stage3. Once the first remote build completes successfully, metro will automatically change {{c|.control/strategy/build}} to be {{c|local}} instead of {{c|remote}}, so it will use the most recently-built Core_2 32bit stage3 as a seed for any new Core_2 32bit builds from now on.<br />
<br />
= Build your own tailored stage3 = <!--T:70--><br />
<br />
<!--T:71--><br />
Metro can be easily configured for building custom stage3 by including additional packages. You can find following directory {{c|/etc/builds/packages}} in your copy of metro repository and a corresponding {{c|arch}} configuration files inside:<br />
{{file|name=/etc/builds/packages/x86-64bit.conf|body=<br />
[section emerge]<br />
<br />
packages: [<br />
sys-kernel/debian-sources<br />
]<br />
}}<br />
Notice a {{c|debian-sources}} ebuild is added for all 64-bit stages. Modify the file to include (or exclude in case Funtoo add additional) packages of your choice. They will be included in your custom stage3 portage's world file.<br />
<br />
= Building Gentoo stages = <!--T:98--><br />
<br />
<!--T:99--><br />
Metro can also build Gentoo stages. After switching to Funtoo profile, see http://www.funtoo.org/Funtoo_Profiles metro require additional steps for this. We have an open bug for this -- it is simply due to the fact that we focus on ensuring Funtoo Linux builds and building Gentoo is a lower priority. Historical note: Funtoo Linux originally started as a fork of Gentoo Linux so that metro could reliably build Gentoo stages.<br />
http://www.funtoo.org/Funtoo_Profiles<br />
<br />
= Advanced Features = <!--T:100--><br />
<br />
<!--T:101--><br />
Metro also includes a number of advanced features that can be used to automate builds and set up distributed build servers. These features require you to {{c|emerge sqlalchemy}}, as SQLite is used as a dependency and also {{c|emerge dev-python/lxml}} as this is needed for index file generation.<br />
<br />
== Repository Management == <!--T:102--><br />
<br />
<!--T:103--><br />
Metro includes a script in the {{c|scripts}} directory called {{c|buildrepo}}. Buildrepo serves as the heart of Metro's advanced repository management features.<br />
<br />
=== Initial Setup === <!--T:104--><br />
<br />
<!--T:105--><br />
To use {{c|buildrepo}}, you will first need to create a {{f|.buildbot}} configuration file. Here is the file I use on my AMD Jaguar build server:<br />
<br />
<!--T:106--><br />
{{file|name=/root/.buildbot|lang=python|body=<br />
builds = (<br />
"funtoo-current",<br />
"funtoo-current-hardened",<br />
)<br />
<br />
<!--T:107--><br />
arches = (<br />
"x86-64bit",<br />
"pure64"<br />
)<br />
<br />
<!--T:108--><br />
subarches = (<br />
"amd64-jaguar",<br />
"amd64-jaguar-pure64",<br />
)<br />
<br />
<!--T:109--><br />
def map_build(build, subarch, full, full_date):<br />
# arguments refer to last build...<br />
if full == True:<br />
buildtype = ( "freshen", )<br />
else:<br />
buildtype = ("full", )<br />
# return value can be a string like "full+openvz" or a sequence type like [ "freshen", "openvz" ]<br />
return buildtype<br />
}}<br />
<br />
<!--T:110--><br />
This file is actually a python source file that defines the tuples {{c|builds}}, {{c|arches}} and {{c|subarches}}. These variables tell {{c|buildrepo}} which builds, arches and subarches it should manage. A {{c|map_build()}} function is also defined which {{c|buildbot}} uses to determine what kind of build to perform. The arguments passed to the function are based on the last successful build. The function can read these arguments and return a string to define the type of the next build. In the above example, the {{c|map_build()}} function will cause the next build after a freshen build to be a full build, and the next build after a full build to be a freshen build, so that the build will alternate between full and freshen.<br />
<br />
== Automated Builds == <!--T:111--><br />
<br />
<!--T:112--><br />
Once the {{c|.buildbot}} file has been created, the {{c|buildrepo}} and {{c|buildbot.sh}} tools are ready to use. Here's how they work. These tools are designed to keep your repository ({{c|path/mirror}} in {{f|/root/.metro}} up-to-date by inspecting your repository and looking for stages that are out-of-date. <br />
<br />
<!--T:113--><br />
To list the next build that will be performed, do this -- this is from my ARM build server:<br />
<br />
<!--T:114--><br />
{{console|body=<br />
# ##i##./buildrepo nextbuild<br />
build=funtoo-current<br />
arch_desc=arm-32bit<br />
subarch=armv7a_hardfp<br />
fulldate=2015-02-08<br />
nextdate=2015-02-20<br />
failcount=0<br />
target=full<br />
extras=''<br />
}}<br />
<br />
<!--T:115--><br />
If no output is displayed, then all your builds are up-to-date.<br />
<br />
<!--T:116--><br />
To actually run the next build, run {{c|buildbot.sh}}:<br />
<br />
<!--T:117--><br />
{{console|body=<br />
# ##i##./buildbot.sh<br />
}}<br />
<br />
<!--T:118--><br />
If you're thinking that {{c|buildbot.sh}} would be a good candidate for a cron job, you've got the right idea!<br />
<br />
=== List Builds === <!--T:119--><br />
<br />
<!--T:120--><br />
To get a quick look at our repository, let's run the {{c|buildrepo fails}} command:<br />
<br />
<!--T:121--><br />
{{console|body=<br />
# ##i##./buildrepo fails<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current/x86-64bit/amd64-jaguar<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current/pure64/amd64-jaguar-pure64<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current-hardened/x86-64bit/amd64-jaguar<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current-hardened/pure64/amd64-jaguar-pure64<br />
}}<br />
<br />
<!--T:122--><br />
On my AMD Jaguar build server, on Feb 20, 2015, this lists all the builds that {{c|buildrepo}} has been configured to manage. The first number on each line is a '''failcount''', which is the number of consecutive times that the build has failed. A zero value indicates that everything's okay. The failcount is an important feature of the advanced repository management features. Here are a number of behaviors that are implemented based on failcount:<br />
<br />
<!--T:123--><br />
* If {{c|buildbot.sh}} tries to build a stage and the build fails, the failcount is incremented.<br />
* If the build succeeds for a particular build, the failcount is reset to zero.<br />
* Builds with the lowest failcount are prioritized by {{buildrepo}} to build next, to steer towards builds that are more likely to complete successfully.<br />
* Once the failcount reaches 3 for a particular build, it is removed from the build rotation.<br />
<br />
=== Resetting Failcount === <!--T:124--><br />
<br />
<!--T:125--><br />
If a build has issues, the failcount for a build will reach 3, at which point it will be pulled out of build rotation. To clear failcount, so that these builds are attempted again -- possibly fixed by new updates to the Portage tree -- use {{c|buildrepo zap}}:<br />
<br />
<!--T:126--><br />
{{console|body=<br />
# /root/metro/scripts/buildrepo zap<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv7a_hardfp/.control/.failcount...<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv6j_hardfp/.control/.failcount...<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv5te/.control/.failcount...<br />
}}<br />
<br />
== Repository Maintenance == <!--T:127--><br />
<br />
<!--T:128--><br />
A couple of repository maintenance tools are provided:<br />
<br />
<!--T:129--><br />
* {{c|buildrepo digestgen}} will generate hash files for the archives in your repository, and clean up stale hashes. <br />
* {{c|buildrepo index.xml}} will create an index.xml file at the root of your repository, listing all builds available.<br />
* {{c|buildrepo clean}} will output a shell script that will remove old stages. No more than the three most recent stage builds for each build/arch/subarch are kept.<br />
<br />
== Distributed Repositories == <!--T:130--><br />
<br />
<!--T:131--><br />
In many situation, you will have a number of build servers, and each will build a subset of your master repository, and then upload builds to the master repository. This is an area of Metro that is being actively developed. For now, automated upload functionality is not enabled, but is expected to be implemented in the relatively near future. However, it is possible to have your master repository differentiate between subarches that are built locally, and thus should be part of that system's {{c|buildbot}} build rotation, and those that are stored locally and built remotely. These builds should be cleaned when {{c|buildrepo clean}} is run, but should not enter the local build rotation. To set this up, modify {{f|/root/.buildbot}} and use the {{c|subarches}} and {{c|all_subarches}} variables:<br />
<br />
<!--T:132--><br />
{{file|name=/root/.buildbot|desc=Excerpt of .buildbot config for master repository|body=<br />
# subarches we are building locally:<br />
<br />
<!--T:133--><br />
subarches = ( <br />
"pentium4",<br />
"athlon-xp",<br />
"corei7",<br />
"corei7-pure64",<br />
"generic_32", <br />
"i686", <br />
"amd64-k8",<br />
"amd64-k8-pure64",<br />
"core2_64",<br />
"core2_64-pure64",<br />
"generic_64",<br />
"generic_64-pure64",<br />
) <br />
<br />
# Things we need to clean, even if we may not be building:<br />
<br />
all_subarches = subarches + (<br />
"atom_32",<br />
"atom_64",<br />
"atom_64-pure64",<br />
"amd64-k10",<br />
"amd64-k10-pure64",<br />
"amd64-bulldozer",<br />
"amd64-bulldozer-pure64",<br />
"amd64-steamroller",<br />
"amd64-steamroller-pure64",<br />
"amd64-piledriver",<br />
"amd64-piledriver-pure64",<br />
"amd64-jaguar",<br />
"amd64-jaguar-pure64",<br />
"intel64-haswell",<br />
"intel64-haswell-pure64",<br />
"intel64-ivybridge-pure64",<br />
"intel64-ivybridge",<br />
"armv7a_hardfp",<br />
"armv6j_hardfp",<br />
"armv5te"<br />
) <br />
}}<br />
== Using binary cache ==<br />
Metro has built-in feature which allows to use binary packages cache rather then building same list of packages from sources. For example, core packages, such as @system are updated at slower pace and it makes sense to enable binary cache to make stage building blazing fast. However, the real disadvantage with using binary cache could be a core package update that due to internal ABI changes require rebuilding of numerous packages from sources. Good example is {{c|sys-libs/ncurses-5}} to {{c|sys-libs/ncurses-6}} major update. This is the case when you would need to disable binary cache and use regular ebuild installation from sources. To enable binary cache, in your metro git repository copy, edit the {{c|common.conf}} <br />
{{file|name=/etc/builds/common.conf|desc=Excerpt of default common.conf|body=<br />
[section metro]<br />
<br />
options:<br />
options/stage:<br />
target: gentoo<br />
}}<br />
and set {{c|cache/package}}<br />
{{file|name=/etc/builds/common.conf|desc=Excerpt of common.conf with binary cache enabled|body=<br />
[section metro]<br />
<br />
options:<br />
options/stage: cache/package<br />
target: gentoo<br />
}}<br />
<br />
During stage build metro will save package cache in {{c|/var/tmp/metro/cache/package-cache}}. With any next builds this binary package cache will be used.<br />
<!--T:134--><br />
[[Category:HOWTO]]<br />
[[Category:Metro]]<br />
__TOC__<br />
</translate><br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Funtoo:Metatools/Advanced_Usage&diff=25321Funtoo:Metatools/Advanced Usage2018-10-15T15:34:47Z<p>Oleg: /* Using New Kits with Metro */</p>
<hr />
<div>This page documents how to set up a local development environment that will allow full local testing of any of your customizations, bug fixes or improvements to Funtoo Linux. This includes generating your own meta-repo and kits with your custom changes, as well as getting metro set up for unit tests. We recommend that all Funtoo Linux developers who will be submitting ebuild improvements to use set up this environment for local testing of changes. This setup also allows more involved changes to be tested and validated locally, so that your contributions to Funtoo are as production-ready as possible.<br />
<br />
== Overview ==<br />
<br />
For our local development setup, we will be using [http://gitolite.com gitolite]. Gitolite will make things quite a bit easier by managing git repositories for us. ''Think of gitolite as your own private GitHub that has no Web user interface'' (we modify its settings by pushing to its special {{c|gitolite-admin}} repo) and you'll have a pretty good idea of what gitolite does. We will be using the following systems in these examples:<br />
<br />
* {{c|repohost}} - this system will be running gitolite under the {{c|repos}} user account and will house git repositories for meta-repo and kits so that they are stored at a handy central location.<br />
* {{c|ryzen}} - in these examples, this will be the primary development workstation, which will be used for editing cloned git code as well as generating custom kits. Once generated, the custom meta-repo and kits are pushed up to {{c|repohost}}.<br />
<br />
{{Note|When you follow this guide, it is certainly possible to have {{c|repohost}} and {{c|ryzen}} be the same computer.}}<br />
<br />
{{Important|This document assumes you have basic knowledge of {{c|ssh-keygen}} and how to generate public/private SSH key pairs. If you don't know how to to this, see [[Funtoo Containers#Generating SSH Keys|Generating SSH Keys]] for quick steps or [[OpenSSH Key Management, Part 1]] for a more detailed introduction. For this article, you'll probably want to generate a private keys without a passphrase, which is more convenient but a much greater security risk if the private key gets compromised, or one with a passphrase but using [[keychain]] to manage ssh-agent for you.}}<br />
<br />
== Gitolite ==<br />
<br />
=== Installation ===<br />
<br />
To set up gitolite on your LAN, first choose a system that will be used to house your meta-repo and kits git repositories. You can do this on the same system you will be using for testing (and even development), or you can set it up on a dedicated system. It's actually fine to set this up anywhere on the Internet, as git will use ssh to access this repository, but for the purposes of this article, we're assuming you're setting it up somewhere on your LAN. We will refer to this system as '''repohost'''. <br />
<br />
On this system, perform the following steps as root:<br />
<br />
{{console|body=<br />
# ##i##useradd -m repos<br />
}}<br />
<br />
The {{c|repos}} user will be a dedicated user account on the system that will have gitolite enabled and will house our git repositories. Now, we are going to {{c|su}} to this new user on '''repohost''' and perform gitolite configuration:<br />
<br />
{{console|body=<br />
# ##i##su repos<br />
$ ##i##git clone https://github.com/sitaramc/gitolite<br />
$ ##i##install -d ~/bin<br />
}}<br />
<br />
Now, as the {{c|repos}} user, add the following to the end of your {{c|~/.bashrc}} file:<br />
<br />
{{file|name=~/.bashrc|body=<br />
export PATH=$HOME/bin:$PATH<br />
}}<br />
<br />
What we're doing is setting up a {{c|bin}} directory where the {{c|gitolite}} command will be installed, which will be in your path, so that you can use it more easily. With this done, perform the following steps:<br />
<br />
{{console|body=<br />
$ ##i##source ~/.bashrc<br />
$ ##i##gitolite/install -ln<br />
}}<br />
<br />
Now, your {{c|repos}} account is almost ready to be used for hosting repositories. The way gitolite works is that it is going to basically take over ssh access to the account, so that when you connect via ssh with git, it will perform its own authentication. For this to work, you will need to enable your own "master key" to access gitolite. To do this, you'll want to decide from which account you'll want to administer gitolite itself. I prefer to use my "drobbins" account on my development workstation '''ryzen''', so I will copy my ssh public key from {{c|~/.ssh/id_rsa.pub}} to {{c|/var/tmp/ryzen-drobbins.pub}} on the gitolite system, and then perform the following steps to "prime" gitolite with this admin public key:<br />
<br />
{{console|body=<br />
$ ##i##gitolite setup -pk /var/tmp/ryzen-drobbins.pub<br />
}}<br />
<br />
Gitolite will now be initialized to recognize the {{c|drobbins}} remote account as an administrator, which will allow this remote account to clone from {{c|repos@repohost:gitolite-admin}} and push any changes to this special git repository which contains the master configuration for gitolite. This is important because we will be performing the rest of gitolite setup over ssh, using this account.<br />
<br />
{{Note|OK, gitolite is installed on '''repohost'''! From this point forward, we will be using the {{c|drobbins}} (or equivalent) account on your development workstation to configure gitolite remotely.}}<br />
<br />
=== gitolite-admin Clone ===<br />
<br />
Now that gitolite is ready on '''repohost''', we can do everything else remotely. I am going to use the {{c|drobbins}} account on my development workstation '''ryzen''', and you will use whatever account is associated with the public key you loaded into gitolite. I like storing my development repos in {{c|/var/src}} on '''ryzen''', so I'll go ahead and clone the gitolite-admin repo to that location so it can live along all my other git repos. Feel free to put this git repo wherever you like to store git repos that you develop on:<br />
<br />
{{console|body=<br />
$ ##i##cd /var/src<br />
$ ##i##git clone repos@repohost:gitolite-admin<br />
$ ##i##cd gitolite-admin<br />
}}<br />
<br />
We are now ready to configure gitolite. We'll do this by modifying {{c|conf/gitolite.conf}} in the git repo and adding new ssh public keys to {{c|keydir/}} as needed. You will see that the initial public key you used to "prime" gitolite already exists in {{c|keydir/}}. Once we change the configuration, and potentially add new public ssh keys that we want to grant access to gitolite-managed repositories, we'll perform a {{c|git commit}} and {{c|git push}}, and if gitolite doesn't complain about our changes, they'll take effect immediately. We'll go through our initial configuration steps below.<br />
<br />
=== gitolite Configuration ===<br />
<br />
Since I will be generating meta-repo and kits on '''ryzen''', this system will need to have permissions to create repositories in gitolite. I would typically do this on '''ryzen''' as follows. First, as root:<br />
<br />
{{console|body=<br />
# ##i##cp /root/.ssh/id_rsa.pub /var/tmp<br />
}}<br />
<br />
Then, as my regular {{c|drobbins}} user that I typically use to perform development:<br />
<br />
{{console|body=<br />
$ ##i##cd /var/src/gitolite-admin<br />
$ ##i##cp /var/tmp/id_rsa.pub keydir/ryzen-root.pub<br />
$ ##i##git add keydir/*<br />
}}<br />
<br />
{{Note|It's important to change the filename of the public keys you are adding to the {{c|gitolite-admin}} repository. I typically use the format {{c|host-user.pub}}.}}<br />
<br />
Now, we're ready to edit {{c|conf/gitolite.conf}} so that it looks like this:<br />
<br />
{{file|name=gitolite.conf|body=<br />
# Group definitions below, starting with @. This makes it easy to associate multiple ssh keys with a particular person.<br />
<br />
@drobbins = ryzen-drobbins<br />
@repomgr = ryzen-root<br />
<br />
# To enable read-only access to your meta-repo and kits, use this along with<br />
# commented-out line under wildrepo. You will need to add box1-root.pub and<br />
# box2-root.pub to keydir/ as well. This is good for boxes that will be testing<br />
# your meta-repo and kits only but should not be able to modify them.<br />
<br />
#@reporead = box1-root box2-root<br />
<br />
# repositories:<br />
<br />
# SPECIAL ADMIN REPO BELOW -- modify with care! I've switched over to using the @drobbins group instead of <br />
# referencing the individual ryzen-drobbins key directly.<br />
<br />
repo gitolite-admin<br />
RW+ = @drobbins<br />
<br />
# AUTO-CREATED (wild) REPOS: gitolite will auto-create repos under wildrepo/ for us<br />
# upon initial clone of any path within, if the repo doesn't already exist.<br />
<br />
repo wildrepo/..*<br />
C = @repomgr<br />
RW+ = @repomgr @drobbins<br />
# NOTE: to enable read-only access for certain boxes, uncomment this line:<br />
# R = @reporead<br />
}}<br />
<br />
== merge-scripts ==<br />
<br />
=== Overview ===<br />
<br />
Funtoo Linux uses "merge scripts" to create its kits. These scripts work by sourcing ebuilds from various overlays, and combining them using special algorithms to yield the kits you use. A meta-repo is also generated, which points to the specific kits generated that are designed to work together.<br />
<br />
Required for merge scripts is {{c|dev-python/lxml}}.<br />
<br />
=== The Code ===<br />
<br />
You can find the code that does this on GitHub, housed at https://github.com/funtoo/merge-scripts. The script that does all the heavy-lifting is called {{c|merge-all-kits.py}}. Let's clone it from git, on the machine that will be generating new kits and meta-repo. In our example, this will be as the root user on ryzen:<br />
<br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone https://github.com/funtoo/merge-scripts<br />
}}<br />
<br />
{{Note|While it is possible to use your own custom merge script repository, we recommend starting by using our official merge-scripts repository on GitHub, and progress to using your own fork of merge-scripts only when you need to.}}<br />
<br />
=== Configuration ===<br />
<br />
Now that merge-scripts is cloned, we will need to create a {{c|/root/.merge}} configuration file. Use the following file as a starting point:<br />
<br />
{{file|name=/root/.merge|body=<br />
[sources]<br />
<br />
flora = https://github.com/funtoo/flora<br />
kit-fixups = https://github.com/mygithub_user/kit-fixups<br />
gentoo-staging = https://github.com/funtoo/gentoo-staging<br />
<br />
[destinations]<br />
<br />
base_url = repos@repohost:wildrepo/staging<br />
<br />
[branches]<br />
<br />
flora = master<br />
kit-fixups = master<br />
meta-repo = master<br />
<br />
[work]<br />
<br />
source = /var/git/source-trees<br />
destination = /var/git/dest-trees<br />
}}<br />
<br />
==== Sources Section ====<br />
<br />
Let's walk through this configuration file. The {{c|[sources]}} section defines locations of repositories that the merge scripts will use as sources for creating kits and meta-repo. In the above sample config, we are using the official [[Flora]] repository from Funtoo, and the official gentoo-staging repository used by Funtoo, but we are using our own fork of https://github.com/funtoo/kit-fixups, which will allow us to add new ebuilds that will appear in kits, such as bug fixes to existing ebuilds in kits, as well as security fixes. For a core Funtoo Linux developer, this is a good way to start. If you are more interested in contributing third-party ebuilds, then you may instead choose to create your own fork of https://github.com/funtoo/flora, and use our standard kit-fixups repository. Or, you could choose to create forks of both. The recommended best practice is to use our upstream repos when possible, and fork only those repos you want to customize. This way, you'll ensure that you have the most up-to-date versions of ebuilds in those unforked repos.<br />
<br />
==== Branches Section ====<br />
<br />
The {{c|[branches]}} section is used to define the default branches that are used by the merge-scripts. In general, sticking with {{c|master}} is fine, but if you need the flexibility, you can point the merge scripts to a particular feature branch to use instead, for example.<br />
<br />
==== Work Section ====<br />
<br />
The {{c|[work]}} section is used to define paths where the merge-scripts will do their work. The {{c|source}} and {{c|destination}} settings above are good defaults, and define where the merge-scripts will clone source repositories and destination (written to) repositories, such as kits and meta-repo. These are kept in two separate hierarchies so they don't get mixed up.<br />
<br />
== Generating New Kits ==<br />
<br />
With this all configured, you are ready to generate new kits. These kits will be generated as root on your development system, and will be stored on '''repohost'''. Here are the steps you'd perform:<br />
<br />
{{console|body=<br />
# ##i##cd /root/merge-scripts<br />
# ##i##bin/merge-all-kits push<br />
}}<br />
<br />
Before starting the script for the first time you should configure your git user.name and user.email variables.<br />
<br />
{{console|body=<br />
# ##i##git config --global user.email "you@example.com"<br />
# ##i##git config --global user.name "Your Name"<br />
}}<br />
<br />
{{c|merge-all-kits}} will proceed to create new kits and meta-repo, and will push them up to repos@repohost:wildrepo/staging/meta-repo, repos@repohost:wildrepo/staging/core-kit, etc. This process can take quite a while but has been optimized to run quickly on multi-core systems.<br />
<br />
== Using New Kits ==<br />
<br />
Now that the new meta-repo and kits are created, here's how you'll use them on an existing Funtoo system, instead of your official Funtoo meta-repo and kits. First, we'll want to modify {{c|/etc/ego.conf}} as follows:<br />
<br />
{{file|name=/etc/ego.conf|body=<br />
[global]<br />
<br />
sync_user = root<br />
sync_base_url = repos@repohost:wildrepo/staging/{repo}<br />
<br />
# Yes, you are supposed to have a literal "{repo}", above. Ego recognizes this special pattern.<br />
<br />
# You can have whatever [kits] section you want, below...<br />
}}<br />
<br />
You will want to make sure that whatever system is connecting to '''repohost''' is permitted by gitolite, and has its {{c|/root/.ssh/id_rsa.pub file}} stored and committed to the gitolite-admin repository's {{c|keydir/}} and is referenced in {{c|conf/gitolite.conf}}. Otherwise, gitolite will not allow this system to connect. Of course, if you are configuring this on the system that's running {{c|merge-all-kits.py}}, it already has RW permissions to these repositories. Otherwise, you will want to make sure that gitolite has the system's keys as part of its {{c|@reporead}} group.<br />
<br />
{{Note|We change the {{c|sync_user}} to {{c|root}} to force {{c|ego sync}} to use {{c|/root/.ssh/id_rsa}} for authentication. By default, {{c|ego sync}} will attempt to use the {{c|portage}} user which does not have a private key installed, and thus will not be able to authenticate with gitolite. Rather than mess with the {{c|portage}} user and give it a proper home directory and ssh key pair, it's easier just to not drop perms to {{c|portage}} in the first place.}}<br />
<br />
Now, let's move the current meta-repo out of the way -- you can also simply delete the existing meta-repo. And then we'll re-run {{c|ego sync}}:<br />
<br />
{{console|body=<br />
# ##i##cd /var/git<br />
# ##i##mv meta-repo meta-repo.official<br />
# ##i##ego sync<br />
}}<br />
<br />
Ego will now sync your custom repository. If you type {{c|emerge -auDN @world}}, ego will now be using your custom kits, rather than the official Funtoo ones. This means that you can perform a variety of things you couldn't before. You can now add your own custom ebuilds to your fork of {{c|kit-fixups}}, and {{c|merge-all-kits.py}} will automatically incorporate these changes into your own custom kits. This will allow you to locally test any changes before submitting them as pull requests to Funtoo. You will also be able to maintain your own meta-repo and kits with your own local modifications, and have your systems use these meta-repo/kits instead of the official Funtoo ones.<br />
<br />
== Using New Kits with Metro ==<br />
<br />
Rather than using new kits on a local Funtoo Linux system, you may want to use the kits to perform a metro build. This is desirable when you may want to use metro to perform [[Wikipedia:Continuous integration|continuous integration]] by attempting to perform a full stage1, 2 and 3 build, which is what the steps in this example accomplish. To do this, first set up metro on your system:<br />
<br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone https://github.com/funtoo/metro<br />
# ##i##cd metro<br />
# ##i##scripts/autosetup<br />
}}<br />
Now, we will set the {{c|ego}}, administration tool of Funtoo/Linux. The way it is used with metro is independent from {{c|app-admin/ego}} installed on your box. Setup is easy as follows:<br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone https://github.com/funtoo/ego.git<br />
}}<br />
This way you will have {{c|/root/ego}} directory with {{c|ego}} binary that is then used by metro.<br />
<br />
After following the interactive prompts, you'll then want to perform the following steps to install an ego.conf for use by metro:<br />
<br />
{{console|body=<br />
# ##i##install -d /etc/metro/chroot/etc<br />
# ##i##vim /etc/metro/chroot/etc/ego.conf<br />
}}<br />
<br />
To ensure that ego uses your custom local meta-repo, you'll want to ensure that the following lines are in your /etc/metro/chroot/etc/ego.conf:<br />
<br />
{{file|body=<br />
[global]<br />
<br />
sync_user = root<br />
sync_base_url = repos@repohost:wildrepo/staging/{repo}<br />
}}<br />
<br />
Without these ego options, metro will use the official meta-repo instead of your local custom meta-repo. Of course, be sure to specify any custom kit branches to use as well.<br />
<br />
Once your ego.conf is installed, you can perform a metro build with something similar to the following commands:<br />
<br />
{{console|body=<br />
# ##i##cd /root/metro<br />
# ##i##scripts/ezbuild.sh funtoo-current x86-64bit intel64-haswell full<br />
}}<br />
<br />
For a full continuous integration loop, which includes regeneration of new meta-repo contents, the following script can be used:<br />
<br />
{{file|lang=bash|body=<br />
#!/bin/bash<br />
/root/merge-scripts/bin/merge-all-kits push<br />
rm /home/mirror/funtoo/funtoo-current/snapshots/portage-*<br />
/root/metro/scripts/ezbuild.sh funtoo-current x86-64bit intel64-haswell full<br />
}}<br />
<br />
The extra "rm" command is included to clean out any previous daily snapshots, to force metro to regenerate snapshots. Otherwise, any existing daily portage snapshots will be used and new ones will not be generated.<br />
<br />
{{Important|If you are "switching back and forth" between different versions of kits, it may be necessary to wipe metro's binary package cache to avoid build failures caused by emerge trying to install binary packages built for a different environment. A symptom of a package cache needing to be wiped out are builds failing because they depend on a masked slot/subslot combination (and emerge asking you to remove a mask.) In this case, you will want to recursively remove the contents of {{f|/var/tmp/metro/cache/package-cache}}. For periodic updates of a consistent set of kits, this is typically not necessary.}}<br />
<br />
== Changing Repo Definitions ==<br />
<br />
Now that you have generated your first meta-repo, let's look at the key file that defines what overlays and repositories are used to create meta-repo, as well as what kit branches exist, and which ones are prime and which are not. You will want to turn your attention to the {{f|kit-fixups/modules/fixups/foundations.py}} file, which [https://github.com/funtoo/kit-fixups/blob/master/modules/fixups/foundations.py can be viewed here on GitHub]]. Let's take a look at various sections of this file:<br />
<br />
{{file|name=foundations.py|lang=python|desc=Top of foundations.py file|body=<br />
#!/usr/bin/python3<br />
<br />
from enum import Enum<br />
<br />
<br />
class KitStabilityRating(Enum):<br />
PRIME = 0 # Kit is enterprise-quality<br />
NEAR_PRIME = 1 # Kit is approaching enterprise-quality<br />
BETA = 2 # Kit is in beta<br />
ALPHA = 3 # Kit is in alpha<br />
DEV = 4 # Kit is newly created and in active development<br />
CURRENT = 10 # Kit follows Gentoo currrent<br />
DEPRECATED = 11 # Kit is deprecated/retired<br />
<br />
<br />
def KitRatingString(kit_enum):<br />
if kit_enum is KitStabilityRating.PRIME:<br />
return "prime"<br />
elif kit_enum is KitStabilityRating.NEAR_PRIME:<br />
return "near-prime"<br />
elif kit_enum is KitStabilityRating.BETA:<br />
return "beta"<br />
elif kit_enum is KitStabilityRating.ALPHA:<br />
return "alpha"<br />
elif kit_enum is KitStabilityRating.DEV:<br />
return "dev"<br />
elif kit_enum is KitStabilityRating.CURRENT:<br />
return "current"<br />
elif kit_enum is KitStabilityRating.DEPRECATED:<br />
return "deprecated"<br />
}}<br />
<br />
At the top of the file, we define an [[Wikipedia:Enumeration|enumeration]] of the different levels of stability that can exist for a particular kit. Kits marked with a stability rating of {{c|KitStabilityRating.PRIME}} will be tagged in the meta-repo JSON as being a "prime" kit and suitable for production use. <br />
<br />
As you scroll down further, you will see some code like this:<br />
<br />
{{file|name=foundations.py|lang=python|desc=foundations.py kit_groups|body=<br />
class KitFoundation:<br />
<br />
kit_groups = {<br />
'prime': [<br />
{'name': 'core-kit', 'branch': '1.0-prime', 'source': 'gentoo_prime_protected', 'default': True},<br />
{'name': 'core-kit', 'branch': '1.1-prime', 'source': 'gentoo_prime_mk3_protected', 'stability': KitStabilityRating.DEPRECATED},<br />
{'name': 'core-kit', 'branch': '1.2-prime', 'source': 'gentoo_prime_mk4_protected', 'stability': KitStabilityRating.BETA},<br />
{'name': 'core-hw-kit', 'branch': 'master', 'source': 'funtoo_current', 'default': True},<br />
{'name': 'security-kit', 'branch': '1.0-prime', 'source': 'gentoo_prime_protected', 'default': True},<br />
{'name': 'security-kit', 'branch': '1.1-prime', 'source': 'gentoo_prime_mk3_protected', 'stability': KitStabilityRating.DEPRECATED},<br />
{'name': 'security-kit', 'branch': '1.2-prime', 'source': 'gentoo_prime_mk4_protected', 'stability': KitStabilityRating.BETA},<br />
{'name': 'xorg-kit', 'branch': '1.17-prime', 'source': 'funtoo_prime_xorg', 'default': False, 'stability': KitStabilityRating.PRIME},<br />
{'name': 'xorg-kit', 'branch': '1.19-prime', 'source': 'funtoo_mk2_prime', 'default': True, 'stability': KitStabilityRating.PRIME}, # MK2<br />
{'name': 'gnome-kit', 'branch': '3.20-prime', 'source': 'funtoo_prime_gnome', 'default': True},<br />
{'name': 'gnome-kit', 'branch': '3.26-prime', 'source': 'funtoo_mk4_prime', 'default': False, 'stability': KitStabilityRating.DEV},<br />
{'name': 'kde-kit', 'branch': '5.10-prime', 'source': 'funtoo_mk3_prime', 'default': False, 'stability': KitStabilityRating.DEPRECATED},<br />
{'name': 'kde-kit', 'branch': '5.11-prime', 'source': 'funtoo_prime_kde', 'stability': KitStabilityRating.DEPRECATED},<br />
{'name': 'kde-kit', 'branch': '5.12-prime', 'source': 'funtoo_prime_kde_late', 'default': True, 'stability': KitStabilityRating.PRIME},<br />
{'name': 'media-kit', 'branch': '1.0-prime', 'source': 'funtoo_prime_media', 'default': False, 'stability': KitStabilityRating.DEPRECATED},<br />
{'name': 'media-kit', 'branch': '1.1-prime', 'source': 'funtoo_mk3_prime', 'default': True, 'stability': KitStabilityRating.PRIME}, # MK3<br />
{'name': 'media-kit', 'branch': '1.2-prime', 'source': 'funtoo_mk4_prime', 'stability': KitStabilityRating.BETA},<br />
{'name': 'perl-kit', 'branch': '5.24-prime', 'source': 'funtoo_prime_perl', 'default': True},<br />
{'name': 'perl-kit', 'branch': '5.26-prime', 'source': 'funtoo_mk3_prime', 'default': False, 'stability': KitStabilityRating.DEV},<br />
{'name': 'python-modules-kit', 'branch': 'master', 'source': 'funtoo_current', 'default': True, 'stability': KitStabilityRating.PRIME},<br />
}}<br />
<br />
Here, we see the beginning of the definition of the {{c|KitFoundation}} class, which contains the {{c|kit_groups}} class variable. This variable, as you can see, defines a bunch of kits -- their name, the branch, a {{c|stability}} value that defines the stability rating of the kit, and other values. Note that if a kit is marked as {{c|default}}, it is assumed to have a kit stability rating of PRIME. You will also see that there is a {{c|source}} key, which defines the group of repositories and overlays that are used to create this kit.<br />
<br />
Here are three important things to take away from this part of code: First, '''this is the official master definition of what kits exist, their stability rating, and what overlays/repositories are used to generate each kit.''' The next thing to take away from this code is that it is certainly possible to modify these settings for your custom meta-repo so that they are different from the Funtoo defaults.<br />
<br />
Here is the third and most important thing about {{c|kit_groups}} -- the order that they are defined determines the order which the merge scripts try to match packages. The {{f|merge-scripts/package-sets}} files will be processed in order that the kits are defined in {{c|kit_groups}}, so {{c|core-kit}} first, {{c|security-kit}} second, etc. Once a catpkg has been added to a kit, it will not be added to any successive kits.<br />
<br />
=== Going Deeper into Foundations.py ===<br />
<br />
Moving right along, you will see a section of the {{c|KitFoundation}} class that looks like this:<br />
<br />
{{file|name=foundations.py|lang=python|desc=foundations.py python_kit_settings|body=<br />
python_kit_settings = {<br />
# branch / primary python / alternate python / python mask (if any)<br />
'master': {<br />
"primary": "python3_6",<br />
"alternate": "python2_7",<br />
"mask": None<br />
},<br />
'3.4-prime': {<br />
"primary": "python3_4",<br />
"alternate": "python2_7",<br />
"mask": ">=dev-lang/python-3.5"<br />
},<br />
'3.6-prime': {<br />
"primary": "python3_6",<br />
"alternate": "python2_7",<br />
"mask": ">=dev-lang/python-3.7"<br />
},<br />
'3.6.3-prime': {<br />
"primary": "python3_6",<br />
"alternate": "python2_7",<br />
"mask": ">=dev-lang/python-3.7"<br />
}<br />
}<br />
}}<br />
<br />
This section is used to define special settings for python-kit. Specifically, for each branch of python-kit, it defines the primary version of python that we will attempt to use to satisfy python-single USE dependencies, followed by an alternate value to use if the ebuild does not support the first. We can also specify a mask to be injected into each python-kit to mask certain versions of python that should be masked due to lack of support in that kit.<br />
<br />
=== Foundations.py Kit Source Definitons ===<br />
<br />
Next, we have a section that looks like this:<br />
<br />
{{file|name=foundations.py|lang=python|desc=foundations.py kit_source_defs|body=<br />
kit_source_defs = {<br />
"funtoo_current": [<br />
{"repo": "flora"},<br />
{"repo": "faustoo"},<br />
{"repo": "fusion809"},<br />
{"repo": "gentoo-staging"}<br />
],<br />
"funtoo_mk2_prime": [<br />
{"repo": "flora", },<br />
{"repo": "faustoo"},<br />
{"repo": "fusion809", "src_sha1": "489b46557d306e93e6dc58c11e7c1da52abd34b0", 'date': '31 Aug 2017'},<br />
{"repo": "gentoo-staging", "src_sha1": '80d2f3782e7f351855664919d679e94a95793a06', 'date': '31 Aug 2017'},<br />
# add current gentoo-staging to catch any new ebuilds that are not yet in our snapshot above (dev-foo/* match)<br />
{"repo": "gentoo-staging-underlay"},<br />
],<br />
"funtoo_mk3_prime": [<br />
{"repo": "flora", },<br />
{"repo": "faustoo", },<br />
{"repo": "fusion809", "src_sha1": "8733034816d3932486cb593db2dfbfbc7577e28b", 'date': '09 Oct 2017'},<br />
{"repo": "gentoo-staging", "src_sha1": '2de4b388863ab0dbbd291422aa556c9de646f1ff', 'date': '10 Oct 2017'},<br />
{"repo": "gentoo-staging-underlay"},<br />
], ...<br />
}}<br />
<br />
This section contains definitions of various overlay stacks, called "kit sources". Kit sources are a combination of overlays, arranged in a python list ({{c|<nowiki>[ ]</nowiki>}}). A kit source serves as a unified collection of source catpkgs for a particular kit. Each kit can have one kit source. Kit sources ''may'' be shared among kits for consistency purposes, and to avoid duplication and to help organization. Note that this is where we specify branch or SHA1 that is used for each overlay. If no SHA1 is specified, the merge scripts will use the top commit of master as the source for that overlay (ie. the overlay will track upstream.)<br />
<br />
{{Note|A "catpkg" is a category/package combination, like {{c|sys-apps/portage}}, with no version information. When we match catpkgs, we look for a cat/pkg directory in the overlay, and if we find a match, we grab the full contents of the catpkg directory from the overlay, and ignore any other occurrence of the same catpkg in other overlays. When we talk of "package-set" rules below, we are referring to text files that define what catpkgs go into what kits. We can specify catpkgs literally or use patterns and other approaches to select catpkgs destined for a particular kit. The merge scripts uses these package-set rules files to define what goes in each kit. But where they search for each catpkg is defined in {{f|foundations.py}}, above.}}<br />
<br />
Currently, package-set rules (see note above) are applied in<br />
the same order that the overlay appears in the {{c|kit_source_defs list}} -- so for "funtoo_current", package-set rules -- an attempt to match catpkg patterns for a kit -- will<br />
be applied to flora first, then faustoo, then fusion809, etc, with gentoo-staging searched last. In theory, this would allow any "upper" overlay to override catpkgs in later-appearing overlays. However, this is not the case, because the "upper" overlays are generally "locked down" so that they are only ''allowed to provide a particular set of catpkgs'', or are allowed to provide any catpkg as long as it doesn't appear in Gentoo, for example. So in actual practice, gentoo-staging is set up to have the higher priority, even though it is searched for matches later. Also notice that newer kit source definitions now have a "bottom" gentoo-staging-underlay definition that has a special purpose -- to allow us to grab new Gentoo catpkgs that aren't in our snapshotted gentoo-staging repository yet. This works because gentoo-staging-underlay tracks upstream master, while gentoo-staging is locked to a particular point in the past. This way we get brand-new gentoo catpkgs, but existing gentoo catpkgs stay locked to a particular point in time.<br />
<br />
=== Completing the Deep Dive ===<br />
<br />
Our deep dive into the functionality of {{f|foundations.py}} is almost complete -- just one more part to go! You will notice the following code at the end of the file:<br />
<br />
{{file|name=foundations.py|lang=python|desc=foundations.py overlay definitions|body=<br />
@property<br />
def overlays(self):<br />
return {<br />
# use gentoo-staging-2017 dirname to avoid conflicts with ports-2012 generation<br />
"gentoo-staging": {"url": self.config.gentoo_staging, "dirname": "gentoo-staging-2017"},<br />
"gentoo-staging-underlay": {"url": self.config.gentoo_staging, "dirname": "gentoo-staging-2017-underlay"},<br />
"faustoo": {"url": "https://github.com/fmoro/faustoo.git", "eclasses": [<br />
"waf",<br />
"googlecode"<br />
],<br />
# SKIP any catpkgs that also exist in gentoo-staging (like nvidia-drivers). All others will be copied.<br />
"filter": ["gentoo-staging"],<br />
# well, I lied. There are some catpkgs that exist in gentoo-staging that we DO want to copy. These are the<br />
# ones we will copy. We need to specify each one. This list may change over time as faustoo/gentoo gets stale.<br />
"force": [<br />
"dev-java/maven-bin",<br />
"dev-java/sun-java3d-bin",<br />
"dev-php/pecl-mongo",<br />
"dev-php/pecl-mongodb",<br />
"dev-python/mongoengine",<br />
"dev-python/pymongo",<br />
"dev-util/idea-community",<br />
"dev-util/webstorm",<br />
"x11-wm/blackbox"<br />
]<br />
},<br />
"fusion809": {"url": "https://github.com/fusion809/fusion809-overlay.git", "select": [<br />
"app-editors/atom-bin",<br />
"app-editors/notepadqq",<br />
"app-editors/bluefish",<br />
"app-editors/textadept",<br />
"app-editors/scite",<br />
"app-editors/gvim",<br />
"app-editors/vim",<br />
"app-editors/vim-core",<br />
"app-editors/sublime-text"<br />
]<br />
}, # FL-3633, FL-3663, FL-3776<br />
"plex": {"url": "https://github.com/Ghent/funtoo-plex.git", "select": [<br />
"media-tv/plex-media-server",<br />
],<br />
},<br />
# damex's deadbeef (music player like foobar2000) overlay<br />
"deadbeef": {"url": "https://github.com/damex/deadbeef-overlay.git", "copyfiles": {<br />
"profiles/package.mask": "profiles/package.mask/deadbeef.mask"<br />
},<br />
},<br />
# damex's wmfs (window manager from scratch) overlay<br />
"wmfs": {"url": "https://github.com/damex/wmfs-overlay.git", "copyfiles": {<br />
"profiles/package.mask": "profiles/package.mask/wmfs.mask"<br />
},<br />
},<br />
"flora": {"url": self.config.flora, "copyfiles": {<br />
"licenses/renoise-EULA": "licenses/renoise-EULA"<br />
},<br />
},<br />
}<br />
}}<br />
<br />
This section of {{f|foundations.py}} declares the names for the actual overlays themselves, where we can clone them from, and also helps us answer the question of which overlay has priority over another. In the case of the fusion809 overlay, for example, we will only grab a select set of catpkgs, specified in the {{c|select}} list. Whereas in the faustoo overlay, we will force the merge scripts to grab updates the the catpkgs listed in the {{c|force}} list, but ''skip'' any catpkgs in faustoo that also appear in our gentoo-staging snapshot. All these special rules define ''exceptions'' to the normal processing logic of the merge scripts, which will look for catpkg matches in the first overlay listed in the kit source definition, before proceeding to the second, etc.<br />
<br />
=== Wrapping Up Foundations.py ===<br />
<br />
We certainly have a lot of levers we can use to control what gets included in a particular kit. Using {{f|foundations.py}}, we can add or remove overlays we want to use, and we can also alter the order they are scanned by the merge scripts. We can control what snapshots are used, and vary this on a per-kit basis. And in the overlay definitions, we can also create exceptions to the normal processing order, to effectively prioritize gentoo catpkgs over overlay catpkgs, even if the overlays are processed first. Or we can limit the scope of what catpkgs we scan a particular overlay for, or force certain catpkgs to be copied from a particular overlay if we ''know'' we want those particular ones.<br />
<br />
All that is left to cover are the package-set rules, which define which catpkgs go into each kit, and kit-fixups, which allow us to have Funtoo overrides for any catpkgs we want. We'll cover those topics next.<br />
<br />
== Package Sets ==<br />
<br />
'''Package sets''' define which catpkgs go in which kits. The package set files are located at {{c|kit-fixups/package-sets}}. They can either consist of a single file named {{c|(kit)-packages}} or a directory with the same name. In the case of a directory, all of the files inside the directory are concatenated and used as a package set.<br />
<br />
=== Package Set Format ===<br />
<br />
Package sets are a text-based format that consist of one catpkg entry per line. Let's look at the various types of package sets entries:<br />
<br />
{{TableStart}}<br />
<tr><th>Entry type</th><th>Example</th><th>Explanation</th></tr><br />
<tr><td>literal</td><td>{{c|sys-apps/portage}}</td><td>Specify a single catpkg by exact name.</td></tr><br />
<tr><td>literal with move</td><td>{{c|sys-apps/oldpkg -> sys-foo/newpkg}}<td>Specify a catpkg by exact name, but copy it over as a new name.</td></tr><br />
<tr><td>category wildcard</td><td>{{c|sys-apps/*}}</td><td>Specify all packages that appear in a particular category.</td></tr><br />
<tr><td>category wildcard with exceptions</td><td>{{c|sys-apps/* -sys-apps/foo -sys-apps/bar}}</td><td>Specify all packages that appear in a particular category, with some exceptions.</td></tr><br />
<tr><td>regex</td><td>{{c|sys-.*/foo.*}}</td><td>Specify all catpkgs that match a particular regex.</td></tr><br />
<tr><td>dependencies in category</td><td>{{c|@depsincat@:x11-base/xorg-x11:media-fonts}}</td><td>In this example, anything in the {{c|media-fonts}} category that has dependencies upon {{c|x11-base/xorg-x11}}.</td></tr><br />
<tr><td>maintainer</td><td>{{c|@maintainer@:dev-lang:ml@gentoo.org}}</td><td>In this example, all {{c|dev-lang}} packages that have a maintainer of {{c|ml@gentoo.org}}.</td></tr><br />
<tr><td>has eclass</td><td>{{c|@has_eclass@:kde5}}</td><td>In this example, all catpkgs that use an eclass of {{c|kde5.eclass}}.</td></tr><br />
<tr><td>category has eclass</td><td>{{c|@cat_has_eclass@:x11-apps:xorg-2}}</td><td>In this example, all catpkgs in the {{c|x11-apps}} category that use an eclass of {{c|xorg-2.eclass}}.</td></tr><br />
{{TableEnd}}<br />
<br />
{{Important|It is possible to blacklist certain catpkgs for inclusion in particular kit by creating a "skip" list at {{f|kit-fixups/package-sets/mykitnamehere-packages-skip}}. This file can contain individual catpkgs, one per line, that should not be included in the package-set, even if they match any rules above. Note that you must use literal<br />
catpkgs in the skip list -- no patterns or other special matches are supported.}}<br />
<br />
==== Inline Package Moves ====<br />
<br />
{{Note|The "literal with move" option is a new feature, described below.}}<br />
<br />
The "literal with move" package set syntax described above is one way to tell the merge scripts to copy a catpkg from a source location but give it a new catpkg name, and they are specified directly in the package-set files.<br />
<br />
If the old package name is found, it will be copied over as the new name. If the old name is not found, but the new name is found, the new name will be copied over as the new name. So either the old name or the new name, if found, will be copied over.<br />
<br />
===== Move-Maps =====<br />
<br />
You can also specify package moves by creating a file called {{f|kit-fixups/move-maps/kitname}} containing the same "literal with move" syntax. Global move maps can be placed in {{f|kit-fixups/move-maps/global}}. Also note that {{f|kit-fixups/move-maps/nokit}} or {{f|kit-fixups/move-maps/global}} is the only way to perform funtoo package moves for nokit. I recommend using the "global" method since it will still automatically work if someone else moves your package into a kit.<br />
<br />
This move-maps functionality works similarly to package-moves that appear within a kit package-set, except that these moves do not automatically add either specified catpkg to the kit. So there must be something in the package-set that matches the old package name. After the old package is matched, additional logic looks at the move-maps and see if the match is in a move-map that tells us to copy it over as the "new" name. The old package will be copied over as the new name.<br />
<br />
If you need to just rename a single package in a kit, it's fine to use the inline method. For more capability, {{f|move-maps/}} are your friend. They're more powerful because a move-map specified with {{f|move-maps/}} will also apply to any special wildcard package-set matches via {{c|@regex@}}, etc, whereas the inline method is limited by design. Also, the out-of-band method allows you to perform package moves on catpkgs in nokit.<br />
<br />
{{Important|Package moves give you the ability to rename catpkgs as they appear in kits. We still need to add functionality to provide this data to Portage so that it can update any package database entries for packages installed under the old name. This part is not done yet, so this is considered a testing-only feature for the time being and should only be used on local kit-fixups overlays for testing, not in our official kit-fixups repo yet.}}<br />
<br />
=== Package Sets -- Putting It All Together ===<br />
<br />
Here are some important facts about package sets:<br />
<br />
# Package set matches are executed in a particular order, and this order is defined by the order of kits in {{c|kit_groups}} in {{c|foundations.py}}.<br />
# Once a catpkg is matched during processing of a kit, that catpkg is assigned to that kit, and cannot appear in another kit.<br />
# If a catpkg is included in a particular branch of a kit, then that catpkg will appear in all branches of that kit, assuming it is available.<br />
# Each kit and branch defined in {{c|kit_groups}} specifies a "source" -- an entry in {{c|kit_source_defs}} which in turn defines a stack of repositories/overlays and associated SHA1 commits to use as sources for catpkgs.<br />
<br />
So, here's how the package set processing would begin. If we look at {{c|kit_groups}} in {{c|foundations.py}}, we see that core-kit 1.0-prime is listed first. So we will look for catpkg matches for core-kit 1.0-prime using the {{c|kit-fixups/package-sets/core-kit-packages}} package set directory. We will apply these match rules against the {{c|gentoo_prime_protected}} kit source definition, and we will look for matches in each repository in the order listed in the {{c|kit_source_defs}} entry. The first match we find will be used as the source catpkg. But remember that we have specific rules in place, defined in the {{c|overlays}} property, that effectively gives gentoo-staging priority for most of the catpkgs.<br />
<br />
After this is done, we will then process core-kit 1.2-prime, since 1.1-prime is deprecated and will be skipped, and then continue to core-hw-kit, and continue to work down the {{c|kit_groups}} list. This process will build up a set of catpkgs that will appear in each kit.<br />
<br />
== Kit Fixups ==<br />
<br />
And finally, we have saved a very key part of the kit generation process for last. The {{c|kit-fixups}} repository is so named because it contains ''fixups'', which are forked Funtoo catpkgs that are used to override catpkgs that appear in the upstream overlays and repositories. They have a special structure. We will look at the structure of the core-kit fixup directory, although others will follow the same model:<br />
<br />
{{TableStart}}<br />
<tr><th>CatPkg Path</th><th>Description</th></tr><br />
<tr><td>{{f|kit-fixups/core-kit/global/sys-apps/portage}}</td><td>Due to the {{c|global}} directory, this catpkg will always be used when a package set specifies a match for {{c|sys-apps/portage}}, for '''all''' branches of core-kit.</td></tr><br />
<tr><td>{{f|kit-fixups/core-kit/curated/sys-apps/portage}}</td><td>Due to the {{c|curated}} directory, this catpkg will always be used when a package set specifies a match for {{c|sys-apps/portage}}, for '''all''' branches of core-kit '''except a master branch.'''</td></tr><br />
<tr><td>{{f|kit-fixups/core-kit/1.2-prime/sys-apps/portage}}</td><td>Due to the {{c|1.2-prime}} directory, this catpkg will always be used when a package set specifies a match for {{c|sys-apps/portage}}, for the '''1.2-prime''' branch of core-kit only.</td></tr><br />
{{TableEnd}}<br />
<br />
Remember that kit-fixups is designed so that ''a fixup will always override any upstream packages.'' This makes it easy to keep track of Funtoo-maintained core packages. And also note that the flora repository should be used for "bonus" packages while kit-fixups should focus more on forks of critical system packages and bug fixes for Funtoo. This way, we can keep contributed ebuilds separate from core operating system ebuilds and associated bug fixes for upstream issues.<br />
<br />
== Developer Q&A ==<br />
<br />
This section contains various tasks that a developer may need to perform, and what steps should be taken to perform each of these steps.<br />
<br />
; I want to move a catpkg sys-apps/blah from core-kit to foobar-kit.: To do this, first we'll note that ''core-kit comes before foobar-kit in {{c|kit_groups}}. This means that core-kit's package set rules will run first. So we will want to make sure that sys-apps/blah does '''not''' match any rules in the core-kit package-set. This can be done by possibly removing a package-set rule, or using a wildcard with exclusion like {{c|sys-apps/* -sys-apps/blah}}. ''If this doesn't work, a file can be created called {{f|core-kit-packages-skip}} which contains exclusions, and sys-apps/blah can be added to a line in this file.'' Then, you will want to make sure that sys-apps/blah ''does'' match a package set rule for foobar-kit.<br />
<br />
; I want to move a catpkg sys-apps/blah from foobar-kit to core-kit.: To do this, first we'll note that ''foobar-kit comes after core-kit in {{c|kit-groups}}, so core-kit's package set rules will run first. We can thus simply add something that will match 'sys-apps/blah' to core-kit's package-set rules. Once sys-apps/blah is included in core-kit, it will not be available for inclusion in foobar-kit, even if it has an identical rule, or a rule like 'sys-apps/*'. However, note that it is good practice to clean up any rules in foobar-kit that you know are no longer matching any catpkgs.<br />
<br />
{{Note|The above two approaches can be used to move catpkgs between kits transparently to the end-user. In the next ego sync, the catpkg will atomically move from one kit to another and no re-emerging will be required, even if the user had emerged the package from the 'old' kit location.}}<br />
<br />
; I want to contribute a cool package to Funtoo.: To do this, you will want to open a pull request against https://github.com/funtoo/flora. Flora is used for all 'bonus' community-contributed ebuilds.<br />
<br />
; I want to fix a bug in a particular ebuild.: To do this, first find out where the ebuild is coming from. A good way to do this is to type {{c|ls -d /var/git/meta-repo/kits/*/sys-apps/foobar}}, which will show you what kit it is in. Running {{c|emerge -s sys-apps/foobar}} will also display this information. For research purposes, it is often useful to find where the original catpkg was sourced from. You can consult https://ports.funtoo.org/packages.xml which contains a list of all catpkgs and their source repository. After doing some initial research and seeing what's wrong, you might have a fix for the ebuild. Generally, the best way to fix the ebuild is to clone https://github.com/funtoo/kit-fixups and create an appropriate fixup for the ebuild if none exists, and simply improve our fixup if one exists already. Then you can create a GitHub pull request, or open a bug on bugs.funtoo.org, or both. Remember that fixup catpkgs will totally replace all upstream ebuilds, so you may need to include multiple versions of the ebuild, even ones that don't need a fix, if they are still needed for certain packages.<br />
<br />
{{Note|If you want to fix a bug in an ebuild and you find that the ebuild comes from flora, you will want to fork flora and submit a pull request against flora instead.}}<br />
<br />
; I want to make a particular branch of a kit the default kit.: To do this, you will modify {{c|kit_groups}} and set the kit you want to be default to have {{c|'default' : True}} or {{c|'stability' : KitStabilityRating.PRIME}}, or both. Only one kit branch can be set as default.<br />
<br />
; I don't want to generate a particular branch of a kit.: To prevent a branch of a kit from being generated, set its {{c|stability}} to {{c|KitStabilityRating.DEPRECATED}} in {{c|kit_groups}}.<br />
<br />
; I want to generate a new kit branch that uses much newer ebuilds from Gentoo or from an upstream repo.: First, define a new entry in {{c|kit_source_defs}} that contains the collection of overlays and repos you want to use as sources. Specify the SHA1 commits you want to use for each repo (or don't specify one to use master.) Then, you will want to add a new kit definition to {{c|kit_groups}}, in the "prime" section.<br />
<br />
; I want to include a package in Funtoo, but move it to a new name.: To do this, use the "literal with package move" format in the package set (see section on [[#Package Moves]], above). You can also use files in {{f|kit_fixups/move-maps/kitname}} or {{f|kit_fixups/move-maps/global}} (info in a note below the Package Set syntax section.).<br />
<br />
{{Important|Be sure to stop by {{c|#funtoo-dev}} on irc.freenode.net if you need further assistance! We are here to help.}}<br />
<br />
[[Category:Official Documentation]]<br />
[[Category:Development]]</div>Oleghttps://www.funtoo.org/index.php?title=LXD&diff=24842LXD2018-10-07T17:23:13Z<p>Oleg: /* PART II - LXD Installation */</p>
<hr />
<div>== PART I - Introduction ==<br />
LXD is a container "hypervisor" it should provide user with a new and fresh experience using [[LXC]] technology.<br />
{{#layout:doc}}<br />
__TOC__<br />
<br />
LXD consists of three components:<br />
<br />
* A system-wide daemon (lxd)<br />
* A command line client (lxc)<br />
* An OpenStack Nova plugin (nova-compute-lxd)<br />
<br />
A REST API that is accesible both locally and if enabled, over the network is provided from the lxd daemon.<br />
<br />
The command line tool is designed to be a very simple, yet very powerful tool to manage all your containers. It can handle connections to multiple container hosts and easily give you an overview of all the containers on your network, let you create some more where you want them and even move them around while they're running.<br />
<br />
The OpenStack plugin then allows you to use your lxd hosts as compute nodes, running workloads on containers rather than virtual machines.<br />
<br />
The LXD project was founded and is currently led by Canonical Ltd and Ubuntu with contributions from a range of other companies and individual contributors.<br />
<br />
=== Features ===<br />
<br />
Some of the biggest features of LXD are:<br />
<br />
* Secure by design (unprivileged containers, resource restrictions and much more)<br />
* Scalable (from containers on your laptop to thousand of compute nodes)<br />
* Intuitive (simple, clear API and crisp command line experience)<br />
* Image based (no more distribution templates, only good, trusted images)<br />
* Live migration<br />
<br />
==== Unprivileged Containers ====<br />
<br />
LXD uses unprivileged containers by default. The difference between an unprivileged container and a privileged one is whether the root user in the container is the “real” root user (uid 0 at the kernel level).<br />
<br />
The way unprivileged containers are created is by taking a set of normal UIDs and GIDs from the host, usually at least 65536 of each (to be POSIX compliant) and mapping those into the container.<br />
<br />
The most common example and what most LXD users will end up with by default is a map of 65536 UIDs and GIDs, with a host base id of 100000. This means that root in the container (uid 0) will be mapped to the host uid 100000 and uid 65535 in the container will be mapped to uid 165535 on the host. UID/GID 65536 and higher in the container aren’t mapped and will return an error if you attempt to use them.<br />
<br />
From a security point of view, that means that anything which is not owned by the users and groups mapped into the container will be inaccessible. Any such resource will show up as being owned by uid/gid “-1” (rendered as 65534 or nobody/nogroup in userspace). It also means that should there be a way to escape the container, even root in the container would find itself with just as much privileges on the host as a nobody user.<br />
<br />
LXD does offer a number of options related to unprivileged configuration:<br />
<br />
* Increasing the size of the default uid/gid map<br />
* Setting up per-container maps<br />
* Punching holes into the map to expose host users and groups<br />
<br />
=== Relationship with LXC ===<br />
LXD isn't a rewrite of LXC, in fact it's building on top of LXC to provide a new, better user experience. Under the hood, LXD uses LXC through liblxc and its Go binding<br />
to create and manage the containers.<br />
<br />
It's basically an alternative to LXC's tools and distribution template system with the added features that come from being controllable over the network.<br />
<br />
=== Licensing ===<br />
LXD is free software and is developed under the Apache 2 license.<br />
<br />
Let's break this tutorial into smaller parts.<br />
<br />
<hr><br />
<hr><br />
<br />
== [[LXD/LXD Installation|PART II - LXD Installation]] ==<br />
<br />
Installing LXD is as simple as emerging the package:<br />
<br />
<console><br />
# ##i## emerge app-emulation/lxd<br />
</console><br />
<br />
Portage will automatically pull in required dependencies.<br />
<br />
== [[LXD/LXD Setup|PART III - LXD Setup]] ==<br />
<br />
== Containers, snapshots and images ==<br />
'''Containers''' in LXD are made of:<br />
* A filesystem (rootfs)<br />
* A list of configuration options, including resource limits, environment, security options and more<br />
* A bunch of devices like disks, character/block unix devices and network interfaces<br />
* A set of profiles the container inherits configuration from (see below)<br />
* Some properties (container architecture, ephemeral or persistent and the name)<br />
* Some runtime state (when using CRIU for checkpoint/restore)<br />
<br />
Container '''snapshots''' as the name states snapshots of the container in time and cannot be modified in any way. It is worth noting that because snapshots can store the container runtime state, which gives us ability of “stateful” snapshots. That is, the ability to rollback the container including its cpu and memory state at the time of the snapshot.<br />
<br />
LXD is '''image''' based, all LXD containers come from an image. Images are typically clean Linux distribution images similar to what you would use for a virtual machine or cloud instance. It is possible to “publish” a container, making an image from it which can then be used by the local or remote LXD hosts.<br />
<br />
=== Our first image ===<br />
Let's get our hands even more dirty and create our first image. We will be using a generic 64 bit Funtoo Linux image. <br />
<br />
{{note|The Funtoo's default build host is building only westmere stage for now.}}<br />
<br />
Grab the image here:<br />
https://build.funtoo.org/funtoo-current/x86-64bit/intel64-westmere/lxd-latest.tar.xz<br />
<br />
Grab also the hash file:<br />
https://build.funtoo.org/funtoo-current/x86-64bit/intel64-westmere/lxd-latest.tar.xz.hash.txt<br />
<br />
{{tip|Check the hash of the downloaded file against the one from server. Proceed if they match. }}<br />
<br />
==== Import the image ====<br />
After we have successfully downloaded the archive we can now finally import it into LXD and start using it as our "seed" image for all our containers.<br />
{{console|body=<br />
###i## lxc image import lxd-latest.tar.xz --alias funtoo<br />
Image imported with fingerprint: 6c2ca3af0222d503656f5a1838885f1b9b6aed2c1994f1d7ef94e2efcb7233c4<br />
###i## lxc image ls<br />
<nowiki>+--------+--------------+--------+------------------------------------+--------+----------+-----------------------------+<br />
| ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCH | SIZE | UPLOAD DATE |<br />
+--------+--------------+--------+------------------------------------+--------+----------+-----------------------------+<br />
| funtoo | 6c2ca3af0222 | no | Funtoo Current Generic Pure 64-bit | x86_64 |227.99MB | Dec 13, 2017 at 11:01pm (UTC) |<br />
+--------+--------------+--------+------------------------------------+--------+----------+-----------------------------+<br />
</nowiki>}}<br />
And there we have our very first Funtoo Linux image imported inside LXD. You can reference the image through the alias or through the fingerprint. Aliases can be added also later.<br />
<br />
Let me show you some basic usage then.<br />
<br />
=== Creating your first container ===<br />
So now we can launch our first container. That is done using this command:<br />
<br />
{{console|body=<br />
###i## lxc launch funtoo fun-1<br />
Creating fun-1<br />
Starting fun-1<br />
###i## lxc ls<br />
<nowiki>+-------+---------+------+-----------------------------------------------+------------+-----------+<br />
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |<br />
+-------+---------+------+-----------------------------------------------+------------+-----------+<br />
| fun-1 | RUNNING | | fd42:156d:4593:a619:216:3eff:fef7:c1c2 (eth0) | PERSISTENT | 0 |<br />
+-------+---------+------+-----------------------------------------------+------------+-----------+<br />
</nowiki>}}<br />
<br />
{{tip|lxc launch is a shortcut for lxc init and lxc start, lxc init creates the container without starting it. }}<br />
<br />
==== Profiles intermezzo ====<br />
LXD has the ability to change quite a few container settings, including resource limitation, control of container startup and a variety of device pass-through options using what is called profiles. Multiple profiles can be applied to a single container, and the last profile overrides the other ones it the resources being configured is the same for multiple profiles. Let me show you how can this be used.<br />
<br />
This is the default profile that gets inherited by all containers.<br />
{{console|body=<br />
###i## lxc profile list<br />
<nowiki>+---------+---------+<br />
| NAME | USED BY |<br />
+---------+---------+<br />
| default | 1 |<br />
+---------+---------+<br />
</nowiki><br />
###i## lxc profile show default<br />
config: {}<br />
description: Default LXD profile<br />
devices:<br />
eth0:<br />
nictype: bridged<br />
parent: lxdbr0<br />
type: nic<br />
root:<br />
path: /<br />
pool: default<br />
type: disk<br />
name: default<br />
used_by:<br />
- /1.0/containers/fun-1<br />
}}<br />
<br />
Now let's edit this profile for our funtoo containers. It will include some useful stuff.<br />
<br />
{{console|body=<br />
###i## lxc profile set default raw.lxc "lxc.mount.entry = none dev/shm tmpfs rw,nosuid,nodev,create=dir"<br />
###i## lxc profile set default environment.LANG "en_US.UTF-8"<br />
###i## lxc profile set default environment.LC_ALL "en_US.UTF-8"<br />
###i## lxc profile set default environment.LC_COLLATE "POSIX"<br />
}}<br />
<br />
Profiles can store any configuration that a container can (key/value or devices) and any number of profiles can be applied to a container. Profiles are applied in the order they are specified so the last profile to specify a specific key wins. In any case, resource-specific configuration always overrides that coming from the profiles.<br />
<br />
The default profile is set for any new container created which doesn't specify a different profiles list.<br />
<br />
{{note|LXD supports simple instance types. Those are represented as a string which can be passed at container creation time. [https://github.com/lxc/lxd/blob/master/doc/containers.md#instance-types containers.md#instance-types]}}<br />
<br />
=== Using our first container ===<br />
After we have done all these customizations we can now start using our container.<br />
The next command will give us shell inside the container.<br />
<br />
{{console|body=<br />
###i## lxc exec fun-1 bash<br />
}}<br />
<br />
Now you should see a different prompt starting with<br />
<br />
{{console|body=<br />
fun-1 ~ #<br />
}}<br />
<br />
If we run top or ps for example we will see only the processes of the container.<br />
<br />
{{console|body=<br />
fun-1 ~ # ps aux<br />
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND<br />
root 1 0.0 0.0 4248 748 ? Ss+ 13:20 0:00 init [3]<br />
root 266 0.0 0.0 30488 472 ? Ss 13:20 0:00 /usr/sbin/sshd<br />
root 312 0.2 0.0 17996 3416 ? Ss 13:29 0:00 bash<br />
root 317 0.0 0.0 19200 2260 ? R+ 13:29 0:00 ps aux<br />
}}<br />
<br />
As you can see only the container's processes are shown. User running the processes is root here. What happens if we search for all sshd processes for example on the host box?<br />
<br />
{{console|body=<br />
###i## <nowiki>ps aux|grep ssh<br />
root 14505 0.0 0.0 30564 1508 ? Ss Sep07 0:00 /usr/sbin/sshd <br />
100000 25863 0.0 0.0 30488 472 ? Ss 15:20 0:00 /usr/sbin/sshd <br />
root 29487 0.0 0.0 8324 828 pts/2 S+ 15:30 0:00 grep --colour=auto sshd</nowiki><br />
###i##<br />
}}<br />
<br />
So as you can see, the sshd process is running under user with uid 100000 on the host machine and has a different PID.<br />
<br />
=== Basic actions with containers ===<br />
==== Listing containers ====<br />
{{console|body=<br />
###i## lxc ls<br />
<nowiki>+-------+---------+-----------------------+------------------------------------------------+------------+-----------+<br />
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |<br />
+-------+---------+-----------------------+------------------------------------------------+------------+-----------+<br />
| fun-1 | RUNNING | 10.214.101.187 (eth0) | fd42:156d:4593:a619:a5ad:edaf:7270:e6c4 (eth0) | PERSISTENT | 0 |<br />
| | | | fd42:156d:4593:a619:216:3eff:fef7:c1c2 (eth0) | | |<br />
+-------+---------+-----------------------+------------------------------------------------+------------+-----------+<br />
</nowiki>}}<br />
lxc ls also accepts arguments as filters. For example lxc ls web will list all containers that have web in their name.<br />
<br />
==== Container details ====<br />
{{console|body=<br />
###i## lxc info c1<br />
Name: c1<br />
Remote: unix://<br />
Architecture: x86_64<br />
Created: 2017/09/08 02:07 UTC<br />
Status: Running<br />
Type: persistent<br />
Profiles: default, prf-funtoo<br />
Pid: 6366<br />
Ips:<br />
eth0: inet 10.214.101.79 vethFG4HXG<br />
eth0: inet6 fd42:156d:4593:a619:8619:546e:43f:2089 vethFG4HXG<br />
eth0: inet6 fd42:156d:4593:a619:216:3eff:fe4a:3d4f vethFG4HXG<br />
eth0: inet6 fe80::216:3eff:fe4a:3d4f vethFG4HXG<br />
lo: inet 127.0.0.1<br />
lo: inet6 ::1<br />
Resources:<br />
Processes: 6<br />
CPU usage:<br />
CPU usage (in seconds): 25<br />
Memory usage:<br />
Memory (current): 69.01MB<br />
Memory (peak): 258.92MB<br />
Network usage:<br />
eth0:<br />
Bytes received: 83.65kB<br />
Bytes sent: 9.44kB<br />
Packets received: 188<br />
Packets sent: 93<br />
lo:<br />
Bytes received: 0B<br />
Bytes sent: 0B<br />
Packets received: 0<br />
Packets sent: 0<br />
}}<br />
<br />
==== Container configuration ====<br />
{{console|body=<br />
###i## lxc config edit c1<br />
### This is a yaml representation of the configuration.<br />
### Any line starting with a '# will be ignored.<br />
###<br />
### A sample configuration looks like:<br />
### name: container1<br />
### profiles:<br />
### - default<br />
### config:<br />
### volatile.eth0.hwaddr: 00:16:3e:e9:f8:7f<br />
### devices:<br />
### homedir:<br />
### path: /extra<br />
### source: /home/user<br />
### type: disk<br />
### ephemeral: false<br />
###<br />
### Note that the name is shown but cannot be changed<br />
<br />
architecture: x86_64<br />
config:<br />
image.architecture: x86_64<br />
image.description: Funtoo Current Generic Pure 64-bit<br />
image.name: funtoo-generic_64-pure64-funtoo-current-2016-12-10<br />
image.os: funtoo<br />
image.release: "1.0"<br />
image.variant: current<br />
volatile.base_image: e279c16d1a801b2bd1698df95e148e0a968846835f4769b24988f2eb3700100f<br />
volatile.eth0.hwaddr: 00:16:3e:4a:3d:4f<br />
volatile.eth0.name: eth0<br />
volatile.idmap.base: "0"<br />
volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'<br />
volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'<br />
volatile.last_state.power: RUNNING<br />
devices: {}<br />
ephemeral: false<br />
profiles:<br />
- default<br />
- prf-funtoo<br />
stateful: false<br />
description: ""<br />
}}<br />
<br />
One can also add environment variables.<br />
{{console|body=<br />
###i## lxc config set <container> environment.LANG en_US.UTF-8<br />
###i## lxc config set <container> environment.LC_COLLATE POSIX<br />
}}<br />
<br />
=== Managing files ===<br />
<br />
=== Snapshots ===<br />
<br />
=== Cloning, copying and moving containers ===<br />
<br />
=== Resource control ===<br />
LXD offers a variety of resource limits. Some of those are tied to the container itself, like memory quotas, CPU limits and I/O priorities. Some are tied to a particular device instead, like I/O bandwidth or disk usage limits.<br />
<br />
As with all LXD configuration, resource limits can be dynamically changed while the container is running. Some may fail to apply, for example if setting a memory value smaller than the current memory usage, but LXD will try anyway and report back on failure.<br />
<br />
All limits can also be inherited through profiles in which case each affected container will be constrained by that limit. That is, if you set limits.memory=256MB in the default profile, every container using the default profile (typically all of them) will have a memory limit of 256MB.<br />
<br />
==== Disk ====<br />
Setting a size limit on the container’s filesystem and have it enforced against the container. Right now LXD only supports disk limits if you’re using the ZFS or btrfs storage backend.<br />
<br />
To set a disk limit (requires btrfs or ZFS):<br />
<br />
{{console|body=<br />
###i## lxc config device set c1 root size 20GB<br />
}}<br />
<br />
==== CPU ====<br />
To just limit a container to any 2 CPUs, do:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.cpu 2<br />
}}<br />
<br />
To pin to specific CPU cores, say the second and fourth:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.cpu 1,3<br />
}}<br />
<br />
More complex pinning ranges like this works too:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.cpu 0-3,7-11<br />
}}<br />
<br />
==== Memory ====<br />
To apply a straightforward memory limit run:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.memory 256MB<br />
}}<br />
<br />
(The supported suffixes are kB, MB, GB, TB, PB and EB)<br />
<br />
To turn swap off for the container (defaults to enabled):<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.memory.swap false<br />
}}<br />
To tell the kernel to swap this container’s memory first:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.memory.swap.priority 0<br />
}}<br />
And finally if you don’t want hard memory limit enforcement:<br />
<br />
{{console|body=<br />
###i## lxc config set c1 limits.memory.enforce soft<br />
}}<br />
<br />
==== Network ====<br />
==== Block I/O ====<br />
<br />
=== Resource limits using profile - Funtoo Containers example ===<br />
So I am going to create 3 profiles to mimic the resource limits for current Funtoo Containers. <br />
<br />
{{TableStart}}<br />
<tr class="danger"><th>Price</th><th>RAM</th><th>CPU Threads</th><th>Disk Space</th><th>Sign Up</th></tr><br />
<tr><td>'''$15/mo'''</td><td>4GB</td><td>6 CPU Threads</td><td>50GB</td><td>[https://funtoo.chargebee.com/hosted_pages/plans/container_small Sign Up! (small)]</td></tr><br />
<tr><td>'''$30/mo'''</td><td>12GB</td><td>12 CPU Threads</td><td>100GB</td><td>[https://funtoo.chargebee.com/hosted_pages/plans/container_medium Sign Up! (medium)]</td></tr><br />
<tr><td>'''$45/mo'''</td><td>48GB</td><td>24 CPU Threads</td><td>200GB</td><td>[https://funtoo.chargebee.com/hosted_pages/plans/container_large Sign Up! (large)]</td></tr><br />
{{TableEnd}}<br />
<br />
I am going to create one profile and copy/edit it for the remaining two options.<br />
<br />
{{console|body=<br />
###i## lxc profile create res-small<br />
###i## lxc profile edit res-small<br />
config:<br />
limits.cpu: "6"<br />
limits.memory: 4GB<br />
description: Small Variant of Funtoo Containers<br />
devices:<br />
root:<br />
path: /<br />
pool: default<br />
size: 50GB<br />
type: disk<br />
name: small<br />
used_by: []<br />
###i## lxc profile copy res-small res-medium<br />
###i## lxc profile copy res-small res-large<br />
###i## lxc profile set res-medium limits.cpu 12<br />
###i## lxc profile set res-medium limits.memory 12GB<br />
###i## lxc profile device set res-medium root size 100GB<br />
###i## lxc profile set res-large limits.cpu 24<br />
###i## lxc profile set res-large limits.memory 48GB<br />
###i## lxc profile device set res-large root size 200GB<br />
}}<br />
Now let's create a container and assign the res-small and funtoo profiles to it.<br />
{{console|body=<br />
###i## lxc init funtoo c-small<br />
###i## lxc profile assign c-small res-small<br />
###i## lxc profile add c-small funtoo<br />
}}<br />
<br />
=== Image manipulations ===<br />
<br />
=== Remote hosts ===<br />
<br />
== Running systemd container on a non-systemd host ==<br />
To use systemd in the container, a recent enough (>=4.6) kernel version with support for cgroup namespaces is needed. Additionally the host needs to have a <code>name=systemd</code> cgroup hierarchy mounted:<br />
{{console|body=<br />
###i## mkdir -p /sys/fs/cgroup/systemd<br />
###i## mount -t cgroup -o none,name=systemd systemd /sys/fs/cgroup/systemd <br />
}}<br />
{{note|Doing so does not require running <code>systemd</code> on the host, it only allows to run systemd correctly inside the container(s) .}}<br />
<br />
If you want to get <code>systemd</code> hierarchy mounted automatically on system startup, using <code>/etc/fstab</code> will not work, but the <br />
{{Package|dev-libs/libcgroup}} can be used for this. First you needed to edit the <code>/etc/cgroup/cgconfig.conf</code> and add:<br />
{{file|name=/etc/cgroup/cgconfig.conf|body=mount {<br />
"name=systemd" = /sys/fs/cgroup/systemd;<br />
}<br />
}}<br />
Then you need to start the cgconfig daemon:<br />
{{console|body=<br />
###i## rc-service cgconfig start<br />
}}<br />
The daemon can be started as needed, or automatically at system start by simply adding it to default group:<br />
{{console|body=<br />
###i## rc-update add cgconfig default<br />
}}<br />
<br />
<hr><br />
<hr><br />
<br />
== [[LXD/LXD in LXD|PART X - LXD in LXD]] ==<br />
== [[LXD/Docker in LXD|PART Y - Docker in LXD]] ==<br />
<br />
== [[LXD/FAQ|PART Z - LXD FAQ]] ==<br />
<br />
== List of tested and working images ==<br />
These are images from the https://images.linuxcontainers.org repository available by default in lxd. You can <br />
list all available images by typing following command (beware the list is very long):<br />
{{console|body=<br />
###i## lxc image list images:<br />
<nowiki>+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCH | SIZE | UPLOAD DATE |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.3 (3 more) | ef69c8dc37f6 | yes | Alpine 3.3 amd64 (20171018_17:50) | x86_64 | 2.00MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.3/armhf (1 more) | 5ce4c80edcf3 | yes | Alpine 3.3 armhf (20170103_17:50) | armv7l | 1.53MB | Jan 3, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.3/i386 (1 more) | cd1700cb7c97 | yes | Alpine 3.3 i386 (20171018_17:50) | i686 | 1.84MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.4 (3 more) | bd4f1ccfabb5 | yes | Alpine 3.4 amd64 (20171018_17:50) | x86_64 | 2.04MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.4/armhf (1 more) | 9fe7c201924c | yes | Alpine 3.4 armhf (20170111_20:27) | armv7l | 1.58MB | Jan 11, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.4/i386 (1 more) | 188a31315773 | yes | Alpine 3.4 i386 (20171018_17:50) | i686 | 1.88MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.5 (3 more) | 63bebc672163 | yes | Alpine 3.5 amd64 (20171018_17:50) | x86_64 | 1.70MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| alpine/3.5/i386 (1 more) | 48045e297515 | yes | Alpine 3.5 i386 (20171018_17:50) | i686 | 1.73MB | Oct 18, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
...<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| | fd95a7a754a0 | yes | Alpine 3.5 amd64 (20171016_17:50) | x86_64 | 1.70MB | Oct 16, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| | fef66668f5a2 | yes | Debian stretch arm64 (20171016_22:42) | aarch64 | 96.56MB | Oct 16, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| | ff18aa2c11d7 | yes | Opensuse 42.3 amd64 (20171017_00:53) | x86_64 | 58.92MB | Oct 17, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
| | ff4ef0d824b6 | yes | Ubuntu zesty s390x (20171017_03:49) | s390x | 86.88MB | Oct 17, 2017 at 12:00am (UTC) |<br />
+---------------------------------+--------------+--------+------------------------------------------+---------+----------+-------------------------------+<br />
</nowiki>}}<br />
<br />
These are the images that are known to work with current LXD setup on Funtoo Linux:<br />
{| class="wikitable sortable"<br />
|-<br />
! Image !! Init !! Status<br />
|-<br />
| CentOS 7 || systemd || Working<br />
|-<br />
| Debian Jessie (8) - EOL April/May 2020|| systemd || Working (systemd - no failed units)<br />
|-<br />
| Debian Stretch (9) - EOL June 2022|| systemd || Working<br />
|-<br />
| Fedora 26 || systemd with cgroup v2|| Not Working<br />
|-<br />
| Fedora 25 || systemd || Working<br />
|-<br />
| Fedora 24 || systemd || Working<br />
|-<br />
| Oracle 7 || systemd || Working (systemd - no failed units)<br />
|-<br />
| OpenSUSE 42.2 || systemd || Working<br />
|-<br />
| OpenSUSE 42.3 || systemd || Working<br />
|-<br />
| Ubuntu Xenial (16.04 LTS) - EOL 2021-04 || systemd || Working<br />
|-<br />
| Ubuntu Zesty (17.04) - EOL 2018-01 || systemd || Working<br />
|-<br />
| Alpine 3.3 || OpenRC || Working<br />
|-<br />
| Alpine 3.4 || OpenRC || Working<br />
|-<br />
| Alpine 3.5 || OpenRC || Working<br />
|-<br />
| Alpine 3.6 || OpenRC || Working<br />
|-<br />
| Alpine Edge || OpenRC || Working<br />
|-<br />
| Archlinux || systemd with cgroup v2 || Not Working<br />
|-<br />
| CentOS 6 || upstart || Working (systemd - no failed units)<br />
|-<br />
| Debian Buster || systemd with cgroup v2 || Not Working<br />
|-<br />
| Debian Sid || systemd with cgroup v2 || Not working<br />
|-<br />
| Debian Wheezy (7) - EOL May 2018 || ? || ? (more testing needed)<br />
|-<br />
| Gentoo || OpenRC || Working (all services started)<br />
|-<br />
| Oracle 6 || upstart || ? (mount outputs nothing)<br />
|-<br />
| Plamo 5 || ? || ?<br />
|-<br />
| Plamo 6 || ? || ?<br />
|-<br />
| Sabayon || systemd with cgroup v2 || Not Working<br />
|-<br />
| Ubuntu Artful (17.10) - EOL 2018-07|| systemd with cgroup v2 || Not Working<br />
|-<br />
| Ubuntu Core 16 || ? || ?<br />
|-<br />
| Ubuntu Trusty (14.04 LTS) - EOL 2019-04 || upstart || Working<br />
|}<br />
<br />
[[Category:Virtualization]]</div>Oleghttps://www.funtoo.org/index.php?title=Package:Xfce4-meta&diff=22710Package:Xfce4-meta2018-09-11T16:29:37Z<p>Oleg: </p>
<hr />
<div>{{Ebuild<br />
|Summary=Meta package for XFCE desktop<br />
|CatPkg=xfce-base/xfce4-meta<br />
|Maintainer=Oleg<br />
}}<br />
== About Xfce ==<br />
<blockquote><br />
Xfce is a lightweight desktop environment for UNIX-like operating systems. It aims to be fast and low on system resources, while still being visually appealing and user friendly.<br />
<br />
Xfce embodies the traditional UNIX philosophy of modularity and re-usability. It consists of a number of components that provide the full functionality one can expect of a modern desktop environment. They are packaged separately and you can pick among the available packages to create the optimal personal working environment.<br />
<br />
Another priority of Xfce is adhereance to standards, specifically those defined at [http://freedesktop.org/ freedesktop.org]. [[http://www.xfce.org/about/ About XFCE]]<br />
</blockquote><br />
== Installation ==<br />
<br />
Before emerging Xfce you need to:<br />
<br />
Check that you have enabled the USE flags necessary for a XFCE desktop environment. Set your profile to "xfce" via [[Funtoo_Linux_First_Steps#Changing_profile|mix-ins]], as follows:<br />
{{console|body=<br />
###i## epro mix-ins +xfce<br />
}}<br />
Then, verify that you have the [[Funtoo_Linux_First_Steps#X.Org|X.Org Server]] configured properly. If X.Org is installed on your system, you are now ready to install Xfce. Install the meta-package, which pulls in all of the components you need for a minimal Xfce installation:<br />
<br />
{{console|body=<br />
###i## emerge -auDN @world<br />
###i## emerge xfce4-meta<br />
}}<br />
== Running Xfce ==<br />
<br />
There are several different ways to start Xfce:<br />
<br />
* Editing the {{c|.xinitrc}} file in your user's home directory and using {{c|startx}} from a text console.<br />
* Using a display manager (e.g. SLiM, GDM, and KDM).<br />
* Configuring your shell profile to automatically start Xfce upon a successful login.<br />
=== .xinitrc ===<br />
<br />
The most common way to start the environment is to configure {{c|~/.xinitrc}} to launch an Xfce session. The proper way to launch Xfce from the terminal is by using {{c|startxfce4}} combined with {{c|ck-launch-session}}.<br />
<br />
{{console|body=<br />
$##bl## echo "exec startxfce4 --with-ck-launch" > ~/.xinitrc<br />
}}<br />
Note that {{c|--with-ck-launch}} (from ConsoleKit) is required by Xfce for important tasks such as shutting down, suspending, and automatically mounting removable storage media. Some applications will not work properly without it. Also, configuring the GTK+ theme and other forms of theming through {{Package|Xfce4-settings}} do not often work without the {{c|--with-ck-launch}} command line option. Because we are launching XFCE with a consolekit command line option, we need to add ConsoleKit to the {{c|default}} runlevel, executing the following command as root:<br />
<br />
{{console|body=<br />
###i## rc-update add consolekit default<br />
###i## rc<br />
}}<br />
Finally, to start the graphical environment, run {{c|startx}} from a login shell:<br />
<br />
{{console|body=<br />
$##bl## startx<br />
}}<br />
After running this command, you should be greeted by your shiny new XFCE desktop. For more information about configuring XFCE and some of the default applications that come with it, consider looking at [[Package:xfce4-meta#XFCE configuration and XFCE applications|XFCE configuration and XFCE applications]].<br />
== Display Manager ==<br />
<br />
A display manager is a program that manages sessions and acts as a login screen. Here's a short list of a few of the display managers available from the Funtoo repositories:<br />
* XDM ({{c|x11-apps/xdm}}): X.Org's standard display manager.<br />
* LightDM ({{c|x11-misc/lightdm}}): A lightweight, but highly configurable display manager.<br />
* SLiM ({{c|x11-misc/slim}}): Simple Login Manager -- a lightweight display manager. Not very configurable.<br />
<br />
<br />
Any of these would make a fine choice. After you've chosen which display manager to use, install it:<br />
<br />
{{console|body=###i## emerge <display manager of choice>}}<br />
All that's left to do now is to add the name of the display manager to {{c|/etc/conf.d/xdm}}, add the {{c|xdm}} service and the {{c|dbus}} service to the {{c|default}} runlevel, and start the services:<br />
<br />
{{file|name=/etc/conf.d/xdm|desc=|body=<br />
DISPLAYMANAGER="<display manager name>"<br />
}}<br />
<br />
{{console|body=<br />
###i## rc-update add xdm default<br />
###i## rc-update add dbus default<br />
###i## rc<br />
}}<br />
==Power Group==<br />
If you wish to be able to shutdown/reboot/suspend/hibernate using xfce4-logout-session then you will need to add yourself or the user to the power group by:<br />
<br />
{{console|body=<br />
###i## gpasswd -a <username> power<br />
}}<br />
If the group does not exist then execute the following and try again:<br />
<br />
{{console|body=<br />
###i## groupadd power}}<br />
== XFCE configuration and XFCE applications ==<br />
For more information on configuring XFCE and the default applications provided by XFCE, such as {{c|xfce4-terminal}}, consider following some of the below listed links:<br />
* [[Package:Xfce4-settings|GUI settings configuration for XFCE ({{c|xfce4-settings}})]]<br />
* [[Package:Xfce4-terminal|The default terminal emulator for XFCE ({{c|xfce4-terminal}})]]<br />
* [[XFCE Panel Plugins|A list of the various plugins available for {{c|xfce4-panel}}]]<br />
* [[Thunar Plugins|A list of the plugins available for the Thunar file manager ]]<br />
<br />
[[Category:Desktop]]<br />
[[Category:First Steps]]<br />
[[Category:Official Documentation]]<br />
{{EbuildFooter}}</div>Oleghttps://www.funtoo.org/index.php?title=Installation_Troubleshooting&diff=22406Installation Troubleshooting2018-08-29T07:58:32Z<p>Oleg: /* Graphics Corruption at boot */</p>
<hr />
<div>This page shows you how to troubleshoot issues with your Funtoo Linux installation.<br />
<br />
== Boot failure ==<br />
<br />
If you did not get a {{c|login:}} prompt, it's possible that your install was not successful. You can reboot from the live CD and bind-mount your existing installation without repartitioning or recreating filesystems, chroot inside, and then check your system to ensure everything is configured properly:<br />
<br />
{{console|body=<br />
# ##i##mkdir /mnt/funtoo<br />
# ##i##mount /dev/sda4 /mnt/funtoo<br />
# ##i##mount /dev/sda1 /mnt/funtoo/boot<br />
# ##i##cd /mnt/funtoo<br />
# ##i##mount --bind /proc proc<br />
# ##i##mount --bind /dev dev<br />
# ##i##cp /etc/resolv.conf etc/<br />
# ##i##env -i HOME=/root TERM=$TERM PATH='/bin:/sbin:/usr/bin:/usr/sbin' chroot /mnt/funtoo<br />
}}<br />
<br />
Once you are done making changes, leave the chroot, unmount mounted filesystems, and reboot:<br />
<br />
{{console|body=<br />
(chroot) # ##i##exit<br />
# ##i##umount /mnt/funtoo/boot /mnt/funtoo/dev /mnt/funtoo/proc /mnt/funtoo <br />
# ##i##reboot<br />
}}<br />
<br />
If you continue to experience difficulties getting Funtoo Linux to boot properly, please ask for help in the [https://forums.funtoo.org/ forums] or #funtoo IRC channel.<br />
<br />
== Debugging system startup ==<br />
<br />
{{c|/etc/rc.conf}} contains system settings related to the system initialization scripts. If you are experiencing problems with the system startup scripts, you may find it beneficial to set {{c|rc_logger}} to {{c|YES}}. This will instruct OpenRC to launch a logging daemon to log the entire rc process to {{c|/var/log/rc.log}}.<br />
<br />
== Graphics Corruption at boot ==<br />
<br />
If you experience graphics corruption issues at boot time, you may be experiencing an issue with the [http://fedoraproject.org/wiki/Features/KernelModesetting Kernel Mode Setting subsystem], known as KMS. To resolve this problem, boot from the live CD again [[#boot failure|following the steps above]]. Then, from within the chroot, edit {{c|etc/boot.conf}} and add the line '''params += nomodeset''' to the "{{c|Funtoo Linux genkernel}}" section. Your {{c|/etc/boot.conf}} should now look like this:<br />
<br />
{{file|name=boot.conf|body=<br />
"Funtoo Linux genkernel" {<br />
kernel kernel[-v]<br />
initrd initramfs[-v]<br />
params += real_root=auto <br />
params += nomodeset<br />
} <br />
}}<br />
<br />
Now, save your changes and run <tt>boot-update</tt>:<br />
<br />
{{console|body=<br />
(chroot) # ##i##boot-update<br />
}}<br />
<br />
Now, exit the chroot, unmount filesystems and reboot following the [[#boot failure|boot failure]] instructions.<br />
<br />
[[Category:HOWTO]]<br />
[[Category:Install]]<br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Installation_Troubleshooting&diff=22405Installation Troubleshooting2018-08-29T04:49:24Z<p>Oleg: /* Boot failure */</p>
<hr />
<div>This page shows you how to troubleshoot issues with your Funtoo Linux installation.<br />
<br />
== Boot failure ==<br />
<br />
If you did not get a {{c|login:}} prompt, it's possible that your install was not successful. You can reboot from the live CD and bind-mount your existing installation without repartitioning or recreating filesystems, chroot inside, and then check your system to ensure everything is configured properly:<br />
<br />
{{console|body=<br />
# ##i##mkdir /mnt/funtoo<br />
# ##i##mount /dev/sda4 /mnt/funtoo<br />
# ##i##mount /dev/sda1 /mnt/funtoo/boot<br />
# ##i##cd /mnt/funtoo<br />
# ##i##mount --bind /proc proc<br />
# ##i##mount --bind /dev dev<br />
# ##i##cp /etc/resolv.conf etc/<br />
# ##i##env -i HOME=/root TERM=$TERM PATH='/bin:/sbin:/usr/bin:/usr/sbin' chroot /mnt/funtoo<br />
}}<br />
<br />
Once you are done making changes, leave the chroot, unmount mounted filesystems, and reboot:<br />
<br />
{{console|body=<br />
(chroot) # ##i##exit<br />
# ##i##umount /mnt/funtoo/boot /mnt/funtoo/dev /mnt/funtoo/proc /mnt/funtoo <br />
# ##i##reboot<br />
}}<br />
<br />
If you continue to experience difficulties getting Funtoo Linux to boot properly, please ask for help in the [https://forums.funtoo.org/ forums] or #funtoo IRC channel.<br />
<br />
== Debugging system startup ==<br />
<br />
{{c|/etc/rc.conf}} contains system settings related to the system initialization scripts. If you are experiencing problems with the system startup scripts, you may find it beneficial to set {{c|rc_logger}} to {{c|YES}}. This will instruct OpenRC to launch a logging daemon to log the entire rc process to {{c|/var/log/rc.log}}.<br />
<br />
== Graphics Corruption at boot ==<br />
<br />
If you experience graphics corruption issues at boot time, you may be experiencing an issue with the [http://fedoraproject.org/wiki/Features/KernelModesetting Kernel Mode Setting subsystem], known as KMS. To resolve this problem, boot from the live CD again [[#boot failure|following the steps above]]. Then, from within the chroot, edit <tt>/etc/boot.conf</tt> and add the line "<tt>params += nomodeset</tt>" to the "<tt>Funtoo Linux genkernel</tt>" section. Your <tt>/etc/boot.conf</tt> should now look like this:<br />
<br />
<pre><br />
"Funtoo Linux genkernel" {<br />
kernel kernel[-v]<br />
initrd initramfs[-v]<br />
params += real_root=auto <br />
params += nomodeset<br />
} <br />
</pre><br />
<br />
Now, save your changes and run <tt>boot-update</tt>:<br />
<br />
<console><br />
(chroot) # ##i##boot-update<br />
</console><br />
<br />
Now, exit the chroot, unmount filesystems and reboot following the [[#boot failure|boot failure]] instructions.<br />
<br />
[[Category:HOWTO]]<br />
[[Category:Install]]<br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Installation_Troubleshooting&diff=22404Installation Troubleshooting2018-08-29T04:47:16Z<p>Oleg: /* Debugging system startup */</p>
<hr />
<div>This page shows you how to troubleshoot issues with your Funtoo Linux installation.<br />
<br />
== Boot failure ==<br />
<br />
If you did not get a {{c|login:}} prompt, it's possible that your install was not successful. You can reboot from the live CD and bind-mount your existing installation without repartitioning or recreating filesystems, chroot inside, and then check your system to ensure everything is configured properly:<br />
<br />
{{console|body=<br />
# ##i##mkdir /mnt/funtoo<br />
# ##i##mount /dev/sda4 /mnt/funtoo<br />
# ##i##mount /dev/sda1 /mnt/funtoo/boot<br />
# ##i##cd /mnt/funtoo<br />
# ##i##mount --bind /proc proc<br />
# ##i##mount --bind /dev dev<br />
# ##i##cp /etc/resolv.conf etc/<br />
# ##i##env -i HOME=/root TERM=$TERM PATH='/bin:/sbin:/usr/bin:/usr/sbin' chroot /mnt/funtoo<br />
}}<br />
<br />
Once you are done making changes, leave the chroot, unmount mounted filesystems, and reboot:<br />
<br />
{{console|body=<br />
(chroot) # ##i##exit<br />
# ##i##umount /mnt/funtoo/boot /mnt/funtoo/dev /mnt/funtoo/proc /mnt/funtoo <br />
# ##i##reboot<br />
}}<br />
<br />
If you continue to experience difficulties getting Funtoo Linux to boot properly, please ask for help in the [http://forums.funtoo.org/ forums].<br />
<br />
== Debugging system startup ==<br />
<br />
{{c|/etc/rc.conf}} contains system settings related to the system initialization scripts. If you are experiencing problems with the system startup scripts, you may find it beneficial to set {{c|rc_logger}} to {{c|YES}}. This will instruct OpenRC to launch a logging daemon to log the entire rc process to {{c|/var/log/rc.log}}.<br />
<br />
== Graphics Corruption at boot ==<br />
<br />
If you experience graphics corruption issues at boot time, you may be experiencing an issue with the [http://fedoraproject.org/wiki/Features/KernelModesetting Kernel Mode Setting subsystem], known as KMS. To resolve this problem, boot from the live CD again [[#boot failure|following the steps above]]. Then, from within the chroot, edit <tt>/etc/boot.conf</tt> and add the line "<tt>params += nomodeset</tt>" to the "<tt>Funtoo Linux genkernel</tt>" section. Your <tt>/etc/boot.conf</tt> should now look like this:<br />
<br />
<pre><br />
"Funtoo Linux genkernel" {<br />
kernel kernel[-v]<br />
initrd initramfs[-v]<br />
params += real_root=auto <br />
params += nomodeset<br />
} <br />
</pre><br />
<br />
Now, save your changes and run <tt>boot-update</tt>:<br />
<br />
<console><br />
(chroot) # ##i##boot-update<br />
</console><br />
<br />
Now, exit the chroot, unmount filesystems and reboot following the [[#boot failure|boot failure]] instructions.<br />
<br />
[[Category:HOWTO]]<br />
[[Category:Install]]<br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Installation_Troubleshooting&diff=22403Installation Troubleshooting2018-08-29T04:47:04Z<p>Oleg: /* Debugging system startup */</p>
<hr />
<div>This page shows you how to troubleshoot issues with your Funtoo Linux installation.<br />
<br />
== Boot failure ==<br />
<br />
If you did not get a {{c|login:}} prompt, it's possible that your install was not successful. You can reboot from the live CD and bind-mount your existing installation without repartitioning or recreating filesystems, chroot inside, and then check your system to ensure everything is configured properly:<br />
<br />
{{console|body=<br />
# ##i##mkdir /mnt/funtoo<br />
# ##i##mount /dev/sda4 /mnt/funtoo<br />
# ##i##mount /dev/sda1 /mnt/funtoo/boot<br />
# ##i##cd /mnt/funtoo<br />
# ##i##mount --bind /proc proc<br />
# ##i##mount --bind /dev dev<br />
# ##i##cp /etc/resolv.conf etc/<br />
# ##i##env -i HOME=/root TERM=$TERM PATH='/bin:/sbin:/usr/bin:/usr/sbin' chroot /mnt/funtoo<br />
}}<br />
<br />
Once you are done making changes, leave the chroot, unmount mounted filesystems, and reboot:<br />
<br />
{{console|body=<br />
(chroot) # ##i##exit<br />
# ##i##umount /mnt/funtoo/boot /mnt/funtoo/dev /mnt/funtoo/proc /mnt/funtoo <br />
# ##i##reboot<br />
}}<br />
<br />
If you continue to experience difficulties getting Funtoo Linux to boot properly, please ask for help in the [http://forums.funtoo.org/ forums].<br />
<br />
== Debugging system startup ==<br />
<br />
{{c|/etc/rc.conf}} contains system settings related to the system initialization scripts. If you are experiencing problems with the system startup scripts, you may find it beneficial to set {{c|rc_logger}} to {{c|YES}}>. This will instruct OpenRC to launch a logging daemon to log the entire rc process to {{c|/var/log/rc.log}}.<br />
<br />
== Graphics Corruption at boot ==<br />
<br />
If you experience graphics corruption issues at boot time, you may be experiencing an issue with the [http://fedoraproject.org/wiki/Features/KernelModesetting Kernel Mode Setting subsystem], known as KMS. To resolve this problem, boot from the live CD again [[#boot failure|following the steps above]]. Then, from within the chroot, edit <tt>/etc/boot.conf</tt> and add the line "<tt>params += nomodeset</tt>" to the "<tt>Funtoo Linux genkernel</tt>" section. Your <tt>/etc/boot.conf</tt> should now look like this:<br />
<br />
<pre><br />
"Funtoo Linux genkernel" {<br />
kernel kernel[-v]<br />
initrd initramfs[-v]<br />
params += real_root=auto <br />
params += nomodeset<br />
} <br />
</pre><br />
<br />
Now, save your changes and run <tt>boot-update</tt>:<br />
<br />
<console><br />
(chroot) # ##i##boot-update<br />
</console><br />
<br />
Now, exit the chroot, unmount filesystems and reboot following the [[#boot failure|boot failure]] instructions.<br />
<br />
[[Category:HOWTO]]<br />
[[Category:Install]]<br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Installation_Troubleshooting&diff=22402Installation Troubleshooting2018-08-29T04:43:08Z<p>Oleg: /* Boot failure */</p>
<hr />
<div>This page shows you how to troubleshoot issues with your Funtoo Linux installation.<br />
<br />
== Boot failure ==<br />
<br />
If you did not get a {{c|login:}} prompt, it's possible that your install was not successful. You can reboot from the live CD and bind-mount your existing installation without repartitioning or recreating filesystems, chroot inside, and then check your system to ensure everything is configured properly:<br />
<br />
{{console|body=<br />
# ##i##mkdir /mnt/funtoo<br />
# ##i##mount /dev/sda4 /mnt/funtoo<br />
# ##i##mount /dev/sda1 /mnt/funtoo/boot<br />
# ##i##cd /mnt/funtoo<br />
# ##i##mount --bind /proc proc<br />
# ##i##mount --bind /dev dev<br />
# ##i##cp /etc/resolv.conf etc/<br />
# ##i##env -i HOME=/root TERM=$TERM PATH='/bin:/sbin:/usr/bin:/usr/sbin' chroot /mnt/funtoo<br />
}}<br />
<br />
Once you are done making changes, leave the chroot, unmount mounted filesystems, and reboot:<br />
<br />
{{console|body=<br />
(chroot) # ##i##exit<br />
# ##i##umount /mnt/funtoo/boot /mnt/funtoo/dev /mnt/funtoo/proc /mnt/funtoo <br />
# ##i##reboot<br />
}}<br />
<br />
If you continue to experience difficulties getting Funtoo Linux to boot properly, please ask for help in the [http://forums.funtoo.org/ forums].<br />
<br />
== Debugging system startup ==<br />
<br />
<tt>/etc/rc.conf</tt> contains system settings related to the system initialization scripts. If you are experiencing problems with the system startup scripts, you may find it beneficial to set <tt>rc_logger</tt> to <tt>YES</tt>. This will instruct OpenRC to launch a logging daemon to log the entire rc process to <tt>/var/log/rc.log</tt>.<br />
<br />
== Graphics Corruption at boot ==<br />
<br />
If you experience graphics corruption issues at boot time, you may be experiencing an issue with the [http://fedoraproject.org/wiki/Features/KernelModesetting Kernel Mode Setting subsystem], known as KMS. To resolve this problem, boot from the live CD again [[#boot failure|following the steps above]]. Then, from within the chroot, edit <tt>/etc/boot.conf</tt> and add the line "<tt>params += nomodeset</tt>" to the "<tt>Funtoo Linux genkernel</tt>" section. Your <tt>/etc/boot.conf</tt> should now look like this:<br />
<br />
<pre><br />
"Funtoo Linux genkernel" {<br />
kernel kernel[-v]<br />
initrd initramfs[-v]<br />
params += real_root=auto <br />
params += nomodeset<br />
} <br />
</pre><br />
<br />
Now, save your changes and run <tt>boot-update</tt>:<br />
<br />
<console><br />
(chroot) # ##i##boot-update<br />
</console><br />
<br />
Now, exit the chroot, unmount filesystems and reboot following the [[#boot failure|boot failure]] instructions.<br />
<br />
[[Category:HOWTO]]<br />
[[Category:Install]]<br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Funtoo:Metro&diff=22395Funtoo:Metro2018-08-27T19:05:47Z<p>Oleg: </p>
<hr />
<div><languages/><br />
<translate><br />
<!--T:1--><br />
{{#layout:doc}}{{#widget:AddThis}}Metro is the build system for Funtoo Linux and [[Gentoo Linux]] stages. It automates the bootstrapping process.<br />
<br />
<!--T:2--><br />
This tutorial will take you through installing, setting up and running Metro.<br />
<br />
<!--T:3--><br />
Other [[:Category:Metro|Metro Documentation]] is available.<br />
<br />
= Preface = <!--T:4--> <br />
<br />
{{Note|If you are unfamiliar with how metro works it is recommended that you continue reading this section so you can become familiar with it. If you are already familiar with it, you may wish to use the new [[Metro AutoSetup|autosetup]] script which uses a curses based menu and allows for quickly setting up and running builds base on your choices without requiring any manual steps. Please see the [[Metro AutoSetup]] page for more details}}<br />
<br />
== How Metro Works == <!--T:5--> <br />
<br />
<!--T:6--><br />
Metro is the Funtoo Linux automated build system, and is used to build Funtoo Linux stage tarballs.<br />
<br />
<!--T:7--><br />
Metro cannot create a stage tarball out of thin air. To build a new stage tarball, Metro must use an existing, older stage tarball called a "seed" stage. This seed stage typically is used as the ''build environment'' for creating the stage we want.<br />
<br />
<!--T:8--><br />
Metro can use two kinds of seed stages. Traditionally, Metro has used a stage3 as a seed stage. This stage3 is then used to build a new stage1, which in turn is used to build a new stage2, and then a new stage3. This is generally the most reliable way to build [[Gentoo Linux]] or Funtoo Linux, so it's the recommended approach.<br />
<br />
== Seeds and Build Isolation == <!--T:9--><br />
<br />
<!--T:10--><br />
Another important concept to mention here is something called ''build isolation''. Because Metro creates an isolated build environment, and the build environment is explicitly defined using existing, tangible entities -- a seed stage and a portage snapshot -- you will get consistent, repeatable results. In other words, the same seed stage, portage snapshot and build instructions will generate an essentially identical result, even if you perform the build a month later on someone else's workstation.<br />
<br />
== Local Build == <!--T:11--> <br />
<br />
<!--T:12--><br />
Say you wanted to build a new <tt>pentium4</tt> stage3 tarball. The recommended method of doing this would be to grab an existing <tt>pentium4</tt> stage3 tarball to use as your seed stage. Metro will be told to use this existing <tt>pentium4</tt> stage3 to build a new stage1 for the same <tt>pentium4</tt>. For this process, the generic <tt>pentium4</tt> stage3 would provide the ''build environment'' for creating our new stage1. Then, the new stage1 would serve as the build environment for creating the new <tt>pentium4</tt> stage2. And the new <tt>pentium4</tt> stage2 would serve as the build environment for creating the new <tt>pentium4</tt> stage3.<br />
<br />
<!--T:13--><br />
In the Metro terminology this is called a '''local build''', which means a stage3 of a given architecture is used to seed a brand new build of the same architecture. Incidentally this will be the first exercise we are going to perform in this tutorial.<br />
<br />
<!--T:14--><br />
A week later, you may want to build a brand new <tt>pentium4</tt> stage3 tarball. Rather than starting from the original <tt>pentium4</tt> stage3 again, you'd probably configure Metro to use the most-recently-built <tt>pentium4</tt> stage3 as the seed. Metro has built-in functionality to make this easy, allowing it to easily find and track the most recent stage3 seed available.<br />
<br />
== Remote Build == <!--T:15--> <br />
<br />
<!--T:16--><br />
Metro can also perform '''remote build''', where a stage3 of a different, but binary compatible, architecture is used as a seed to build a different architecture stage3. Consequentiality the second exercise we are going to perform in this tutorial will be to build a <tt>core2 32bit</tt> stage3 tarball from the <tt>pentium4</tt> stage3 tarball we have just built.<br />
<br />
<!--T:17--><br />
TODO: add caveats about what archs can be seeded and what can be not (maybe a table?)<br />
<br />
== Tailored Build == <!--T:18--> <br />
<br />
<!--T:19--><br />
Last, it's also worthy noting that both in <tt>local</tt> and <tt>remote builds</tt>, Metro can be configured to add and/or remove individual packages to the final tarball.<br />
Let's say you can't live without <tt>app-misc/screen</tt>, at the end of this tutorial, we will show how to have your tailored stage3 to include it.<br />
<br />
== Installing Metro == <!--T:20--><br />
<br />
<!--T:21--><br />
'''The recommended and supported method''' is to use the Git repository of Metro. <br />
<br />
<!--T:22--><br />
Ensure that {{Package|dev-vcs/git}}, {{Package|dev-python/requests}} and {{Package|dev-python/boto}} (optional; required for EC2 support) are installed on your system. <br />
<br />
<!--T:23--><br />
{{console|body=<br />
# ##i##emerge dev-vcs/git dev-python/requests dev-python/boto <br />
}}<br />
<br />
<!--T:24--><br />
Next, clone the master git repository as follows:<br />
<br />
<!--T:25--><br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone git://github.com/funtoo/metro.git<br />
# ##i##cp /root/metro/metro.conf ~/.metro<br />
}}<br />
<br />
<!--T:26--><br />
You will now have a directory called {{c|/root/metro}} that contains all the Metro source code.<br />
<br />
<!--T:27--><br />
=== Setting up ego===<br />
Now, we will set the {{c|ego}}, administration tool of Funtoo/Linux. The way it is used with metro is independent from {{c|app-admin/ego}} installed on your box. Setup is easy as follows:<br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone https://github.com/funtoo/ego,git<br />
}}<br />
This way you will have {{c|/root/ego}} directory with {{c|ego}} binary that is then used by metro.<br />
<br />
<!--T:28--><br />
Metro is now installed. It's time to customize it for your local system.<br />
<br />
= Configuring Metro = <!--T:28--><br />
<br />
<!--T:29--><br />
{{Note|Metro is not currently able to build Gentoo stages. See {{Bug|FL-901}}.}}<br />
<br />
<!--T:30--><br />
[[User:Drobbins|Daniel Robbins]] maintains Metro, so it comes pre-configured to successfully build Funtoo Linux releases. Before reading further, you might want to customize some basic settings like the number of concurrent jobs to fit your hardware's capabilities or the directory to use for produced stage archives. This is accomplished by editing {{c|~/.metro}} which is the Metro's master configuration file.<br />
<br />
<!--T:31--><br />
Please note that {{c|path/install}} must point to where metro was installed. Point {{c|path/distfiles}} to where your distfiles reside. Also set {{c|path/mirror/owner}} and {{c|path/mirror/group}} to the owner and group of all the files that will be written to the build repository directory, which by default (as per the configuration file) is at {{c|/home/mirror/funtoo}}. The cache directory normally resides inside the temp directory -- this can be modified as desired. The cache directory can end up holding many cached .tbz2 packages, and eat up a lot of storage. You may want to place the temp directory on faster storage, for faster compile times, and place the cache directory on slower, but more plentiful storage.<br />
<br />
<!--T:32--><br />
{{file|name=.metro|desc=Metro configuration|body=<br />
# Main metro configuration file - these settings need to be tailored to your install:<br />
<br />
<!--T:33--><br />
[section path]<br />
install: /root/metro<br />
tmp: /var/tmp/metro<br />
cache: $[path/tmp]/cache<br />
distfiles: /var/src/distfiles<br />
work: $[path/tmp]/work/$[target/build]/$[target/name]<br />
<br />
<!--T:34--><br />
[section path/mirror]<br />
<br />
<!--T:35--><br />
: /home/mirror/funtoo<br />
owner: root<br />
group: repomgr<br />
dirmode: 775<br />
<br />
<!--T:36--><br />
[section portage]<br />
<br />
<!--T:37--><br />
MAKEOPTS: auto <br />
<br />
<!--T:38--><br />
[section emerge]<br />
<br />
<!--T:39--><br />
options: --jobs=4 --load-average=4 --keep-going=n<br />
<br />
<!--T:40--><br />
# This line should not be modified:<br />
[collect $[path/install]/etc/master.conf]<br />
}}<br />
<br />
== Arch and Subarch == <!--T:41--><br />
<br />
<!--T:42--><br />
In the following example we are creating a pentium4 stage 3 compiled for x86-32bit binary compatibility. Pentium4 is a subarch of the x86-32bit architecture. Once you have metro installed you may find a full list of each subarch in your {{f|/var/git/meta-repo/kits/core-kit/profiles/funtoo-1.0/linux-gnu/arch/x86-32bit/subarch}} directory:<br />
Example:<br />
{{console|body=<br />
# ##i##ls /var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-32bit/subarch/<br />
amd64-k8+sse3_32 athlon-4 athlon-xp core2_32 i486 k6-2 pentium pentium2 pentiumpro<br />
amd64-k8_32 athlon-mp atom_32 generic_32 i686 k6-3 pentium-m pentium3 prescott<br />
athlon athlon-tbird btver1 geode k6 native_32 pentium-mmx pentium4 xen-pentium4+sse3<br />
}}<br />
<br />
64-bit PC profiles can be found in the {{f|/var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-64bit/subarch/}} directory:<br />
{{console|body=<br />
# ##i##ls /var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-64bit/subarch/<br />
amd64-bulldozer amd64-k8+sse3 btver1_64 generic_64 intel64-nehalem native_64<br />
amd64-jaguar amd64-piledriver core-avx-i intel64-broadwell intel64-sandybridge nocona<br />
amd64-k10 amd64-steamroller core2_64 intel64-haswell intel64-silvermont opteron_64<br />
amd64-k8 atom_64 corei7 intel64-ivybridge intel64-westmere xen-pentium4+sse3_64<br />
}}<br />
<br />
= First stages build (local build) = <!--T:43--><br />
<br />
<!--T:44--><br />
To get this all started, we need to bootstrap the process by downloading an initial seed stage3 to use for building and place it in its proper location in {{f|/home/mirror/funtoo}}, so that Metro can find it. We will also need to create some special &quot;control&quot; files in {{f|/home/mirror/funtoo}}, which will allow Metro to understand how it is supposed to proceed.<br />
<br />
== Step 1: Set up pentium4 repository (local build) == <!--T:45--><br />
<br />
<!--T:46--><br />
Assuming we're following the basic steps outlined in the previous section, and building {{f|funtoo-current}} build for the {{f|pentium4}}, using a generic {{f|pentium4}} stage3 as a seed stage, then here the first set of steps we'd perform:<br />
<br />
<!--T:47--><br />
{{console|body=<br />
# ##i##install -d /home/mirror/funtoo/funtoo-current/x86-32bit/pentium4<br />
# ##i##install -d /home/mirror/funtoo/funtoo-current/snapshots<br />
# ##i##cd /home/mirror/funtoo/funtoo-current/x86-32bit/pentium4<br />
# ##i##install -d 2017-10-01<br />
# ##i##cd 2017-10-01<br />
# ##i##wget -c https://build.funtoo.org/funtoo-current/x86-32bit/pentium4/2017-10-01/stage3-pentium4-funtoo-current-2017-10-01.tar.xz<br />
# ##i##cd ..<br />
# ##i##install -d .control/version<br />
# ##i##echo "2017-10-01" > .control/version/stage3<br />
# ##i##install -d .control/strategy<br />
# ##i##echo local > .control/strategy/build<br />
# ##i##echo stage3 > .control/strategy/seed<br />
}}<br />
<br />
<!--T:48--><br />
OK, let's review the steps above. First, we create the directory {{f|/home/mirror/funtoo/funtoo-current/x86-32bit/pentium4}}, which is where Metro will expect to find {{f|funtoo-current}} pentium4 builds -- it is configured to look here by default. Then we create a specially-named directory to house our seed x86 stage3. Again, by default, Metro expects the directory to be named this way. We enter this directory, and download our seed x86 stage3 from funtoo.org. Note that the {{f|2017-10-01}} version stamp matches. Make sure that your directory name matches the stage3 name too. Everything has been set up to match Metro's default filesystem layout.<br />
<br />
<!--T:49--><br />
Next, we go back to the {{f|/home/mirror/metro/funtoo-current/x86-32bit/pentium4}} directory, and inside it, we create a {{f|.control}} directory. This directory and its subdirectories contain special files that Metro references to determine certain aspects of its behavior. The {{f|.control/version/stage3}} file is used by Metro to track the most recently-built stage3 for this particular build and subarch. Metro will automatically update this file with a new version stamp after it successfully builds a new stage3. But because Metro didn't actually ''build'' this stage3, we need to set up the {{f|.control/version/stage3}} file manually. This will allow Metro to find our downloaded stage3 when we set up our pentium4 build to use it as a seed. Also note that Metro will create a similar {{f|.control/version/stage1}} file after it successfully builds an pentium4 funtoo-current stage1.<br />
<br />
<!--T:50--><br />
We also set up {{f|.control/strategy/build}} and {{f|.control/strategy/seed}} files with values of {{f|local}} and {{f|stage3}} respectively. These files define the building strategy Metro will use when we build pentium4 funtoo-current stages. With a build strategy of {{f|local}}, Metro will source its seed stage from funtoo-current pentium4, the current directory. And with a seed strategy of {{f|stage3}}, Metro will use a stage3 as a seed, and use this seed to build a new stage1, stage2 and stage3.<br />
<br />
== Step 2: Building the pentium4 stages == <!--T:51--><br />
<br />
<!--T:52--><br />
Incidentally, if all you wanted to do at this point was to build a new pentium4 funtoo-current stage1/2/3 (plus openvz and vserver templates). You would begin the process by typing:<br />
<br />
<!--T:53--><br />
{{console|body=<br />
# ##i##cd /root/metro<br />
# ##i##scripts/ezbuild.sh funtoo-current x86-32bit pentium4<br />
}}<br />
<br />
<!--T:54--><br />
If you have a slow machine, it could take several hours to be completed because several "heavy" components like gcc or glibc have to be recompiled in each stage. Once a stage has been successfully completed, it is placed in the {{f|"${METRO_MIRROR}/funtoo-current/x32-bit/pentium4/YYYY-MM-DD"}} subdirectory, where {{f|YYYY-MM-DD}} is today's date at the time the {{f|ezbuild.sh}} script was started or the date you put on the ezscript.sh command line.<br />
<br />
= Building for another binary compatible architecture (remote build) = <!--T:55--><br />
<br />
<!--T:56--><br />
As written above, Metro is able to perform '''remote build''' building different architecture stage3 from a binary compatible seeding stage3 (e.g. using a pentium4 stage3 to seed a <tt>Intel Core2 32bits</tt> stage3). <br />
<br />
<!--T:57--><br />
In the Metro terminology this is called a '''remote build''' (a stage 3 of a different, but binary compatible, architecture is used as a seed). <br />
What's not compatible? You can't use a <tt>Sparc</tt> architecture to generate an <tt>x86</tt> or <tt>ARM</tt> based stage and vice-versa. If you use a 32bit stage then you don't want to seed a 64bit build from it. Be sure that you are using a stage from the same architecture that you are trying to seed. Check [http://ftp.osuosl.org/pub/funtoo/funtoo-current/ Funtoo-current FTP Mirror] for a stage that is from the same Architecture that you will be building. <br />
<br />
<!--T:58--><br />
{{Note|Often, one build (ie. funtoo-current) can be used as a seed for another build such as funtoo-stable. However, hardened builds require hardened stages as seeds in order for the build to complete successfully.}}<br />
<br />
== Step 1: Set up Core_2 32bit repository == <!--T:59--><br />
<br />
<!--T:60--><br />
In this example, we're going to use this pentium4 funtoo-current stage3 to seed a new Core_2 32bit funtoo-current build. To get that done, we need to set up the pentium4 build directory as follows:<br />
<br />
<!--T:61--><br />
{{console|body=<br />
# ##i## cd /home/mirror/funtoo/funtoo-current/x86-32bit<br />
# ##i##install -d core2_32<br />
# ##i##cd core2_32<br />
# ##i##install -d .control/strategy<br />
# ##i##echo remote > .control/strategy/build<br />
# ##i##echo stage3 > .control/strategy/seed<br />
# ##i##install -d .control/remote<br />
# ##i##echo funtoo-current > .control/remote/build<br />
# ##i##echo x86-32bit > .control/remote/arch_desc<br />
# ##i##echo pentium4 > .control/remote/subarch<br />
}}<br />
<br />
<!--T:62--><br />
The steps we follow are similar to those we performed for a ''local build'' to set up our pentium4 directory for local build. However, note the differences. We didn't download a stage, because we are going to use the pentium4 stage to build a new Core_2 32bit stage. We also didn't create the <tt>.control/version/stage{1,3}</tt> files because Metro will create them for us after it successfully builds a new stage1 and stage3. We are still using a <tt>stage3</tt> seed strategy, but we've set the build strategy to <tt>remote</tt>, which means that we're going to use a seed stage that's not from this particular subdirectory. Where are we going to get it from? The <tt>.control/remote</tt> directory contains this information, and lets Metro know that it should look for its seed stage3 in the <tt>/home/mirror/funtoo/funtoo-current/x86-32bit/pentium4</tt> directory. Which one will it grab? You guessed it -- the most recently built ''stage3'' (since our seed strategy was set to <tt>stage3</tt>) that has the version stamp of <tt>2010-12-24</tt>, as recorded in <tt>/home/mirror/funtoo-current/x86-32bit/pentium4/.control/version/stage</tt>. Now you can see how all those control files come together to direct Metro to do the right thing.<br />
<br />
<!--T:63--><br />
{{Note|<code>arch_desc</code> should be set to one of: <code>x86-32bit</code>, <code>x86-64bit</code> or <code>pure64</code> for PC-compatible systems. You must use a 32-bit build as a seed for other 32-bit builds, and a 64-bit build as a seed for other 64-bit builds.}}<br />
<br />
== Step 2: Building the Core_2 32bit stages == <!--T:64--><br />
<br />
<!--T:65--><br />
Now, you could start building your new Core_2 32bit stage1/2/3 (plus openvz and vserver templates) by typing the following:<br />
<br />
<!--T:66--><br />
{{console|body=<br />
# ##i##/root/metro/scripts/ezbuild.sh funtoo-current x86-32bit core2_32<br />
}}<br />
<br />
<!--T:67--><br />
In that case, the produced stages are placed in the <tt>/home/mirror/funtoo/funtoo-current/x32-bit/core2_32/YYYY-MM-DD</tt> subdirectory.<br />
<br />
== Step 3: The Next Build == <!--T:68--><br />
<br />
<!--T:69--><br />
At this point, you now have a new Core_2 32bit stage3, built using a "remote" pentium4 stage3. Once the first remote build completes successfully, metro will automatically change {{c|.control/strategy/build}} to be {{c|local}} instead of {{c|remote}}, so it will use the most recently-built Core_2 32bit stage3 as a seed for any new Core_2 32bit builds from now on.<br />
<br />
= Build your own tailored stage3 = <!--T:70--><br />
<br />
<!--T:71--><br />
Metro can be easily configured for building custom stage3 by including additional packages. You can find following directory {{c|/etc/builds/packages}} in your copy of metro repository and a corresponding {{c|arch}} configuration files inside:<br />
{{file|name=/etc/builds/packages/x86-64bit.conf|body=<br />
[section emerge]<br />
<br />
packages: [<br />
sys-kernel/debian-sources<br />
]<br />
}}<br />
Notice a {{c|debian-sources}} ebuild is added for all 64-bit stages. Modify the file to include (or exclude in case Funtoo add additional) packages of your choice. They will be included in your custom stage3 portage's world file.<br />
<br />
= Building Gentoo stages = <!--T:98--><br />
<br />
<!--T:99--><br />
Metro can also build Gentoo stages. After switching to Funtoo profile, see http://www.funtoo.org/Funtoo_Profiles metro require additional steps for this. We have an open bug for this -- it is simply due to the fact that we focus on ensuring Funtoo Linux builds and building Gentoo is a lower priority. Historical note: Funtoo Linux originally started as a fork of Gentoo Linux so that metro could reliably build Gentoo stages.<br />
http://www.funtoo.org/Funtoo_Profiles<br />
<br />
= Advanced Features = <!--T:100--><br />
<br />
<!--T:101--><br />
Metro also includes a number of advanced features that can be used to automate builds and set up distributed build servers. These features require you to {{c|emerge sqlalchemy}}, as SQLite is used as a dependency and also {{c|emerge dev-python/lxml}} as this is needed for index file generation.<br />
<br />
== Repository Management == <!--T:102--><br />
<br />
<!--T:103--><br />
Metro includes a script in the {{c|scripts}} directory called {{c|buildrepo}}. Buildrepo serves as the heart of Metro's advanced repository management features.<br />
<br />
=== Initial Setup === <!--T:104--><br />
<br />
<!--T:105--><br />
To use {{c|buildrepo}}, you will first need to create a {{f|.buildbot}} configuration file. Here is the file I use on my AMD Jaguar build server:<br />
<br />
<!--T:106--><br />
{{file|name=/root/.buildbot|lang=python|body=<br />
builds = (<br />
"funtoo-current",<br />
"funtoo-current-hardened",<br />
)<br />
<br />
<!--T:107--><br />
arches = (<br />
"x86-64bit",<br />
"pure64"<br />
)<br />
<br />
<!--T:108--><br />
subarches = (<br />
"amd64-jaguar",<br />
"amd64-jaguar-pure64",<br />
)<br />
<br />
<!--T:109--><br />
def map_build(build, subarch, full, full_date):<br />
# arguments refer to last build...<br />
if full == True:<br />
buildtype = ( "freshen", )<br />
else:<br />
buildtype = ("full", )<br />
# return value can be a string like "full+openvz" or a sequence type like [ "freshen", "openvz" ]<br />
return buildtype<br />
}}<br />
<br />
<!--T:110--><br />
This file is actually a python source file that defines the tuples {{c|builds}}, {{c|arches}} and {{c|subarches}}. These variables tell {{c|buildrepo}} which builds, arches and subarches it should manage. A {{c|map_build()}} function is also defined which {{c|buildbot}} uses to determine what kind of build to perform. The arguments passed to the function are based on the last successful build. The function can read these arguments and return a string to define the type of the next build. In the above example, the {{c|map_build()}} function will cause the next build after a freshen build to be a full build, and the next build after a full build to be a freshen build, so that the build will alternate between full and freshen.<br />
<br />
== Automated Builds == <!--T:111--><br />
<br />
<!--T:112--><br />
Once the {{c|.buildbot}} file has been created, the {{c|buildrepo}} and {{c|buildbot.sh}} tools are ready to use. Here's how they work. These tools are designed to keep your repository ({{c|path/mirror}} in {{f|/root/.metro}} up-to-date by inspecting your repository and looking for stages that are out-of-date. <br />
<br />
<!--T:113--><br />
To list the next build that will be performed, do this -- this is from my ARM build server:<br />
<br />
<!--T:114--><br />
{{console|body=<br />
# ##i##./buildrepo nextbuild<br />
build=funtoo-current<br />
arch_desc=arm-32bit<br />
subarch=armv7a_hardfp<br />
fulldate=2015-02-08<br />
nextdate=2015-02-20<br />
failcount=0<br />
target=full<br />
extras=''<br />
}}<br />
<br />
<!--T:115--><br />
If no output is displayed, then all your builds are up-to-date.<br />
<br />
<!--T:116--><br />
To actually run the next build, run {{c|buildbot.sh}}:<br />
<br />
<!--T:117--><br />
{{console|body=<br />
# ##i##./buildbot.sh<br />
}}<br />
<br />
<!--T:118--><br />
If you're thinking that {{c|buildbot.sh}} would be a good candidate for a cron job, you've got the right idea!<br />
<br />
=== List Builds === <!--T:119--><br />
<br />
<!--T:120--><br />
To get a quick look at our repository, let's run the {{c|buildrepo fails}} command:<br />
<br />
<!--T:121--><br />
{{console|body=<br />
# ##i##./buildrepo fails<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current/x86-64bit/amd64-jaguar<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current/pure64/amd64-jaguar-pure64<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current-hardened/x86-64bit/amd64-jaguar<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current-hardened/pure64/amd64-jaguar-pure64<br />
}}<br />
<br />
<!--T:122--><br />
On my AMD Jaguar build server, on Feb 20, 2015, this lists all the builds that {{c|buildrepo}} has been configured to manage. The first number on each line is a '''failcount''', which is the number of consecutive times that the build has failed. A zero value indicates that everything's okay. The failcount is an important feature of the advanced repository management features. Here are a number of behaviors that are implemented based on failcount:<br />
<br />
<!--T:123--><br />
* If {{c|buildbot.sh}} tries to build a stage and the build fails, the failcount is incremented.<br />
* If the build succeeds for a particular build, the failcount is reset to zero.<br />
* Builds with the lowest failcount are prioritized by {{buildrepo}} to build next, to steer towards builds that are more likely to complete successfully.<br />
* Once the failcount reaches 3 for a particular build, it is removed from the build rotation.<br />
<br />
=== Resetting Failcount === <!--T:124--><br />
<br />
<!--T:125--><br />
If a build has issues, the failcount for a build will reach 3, at which point it will be pulled out of build rotation. To clear failcount, so that these builds are attempted again -- possibly fixed by new updates to the Portage tree -- use {{c|buildrepo zap}}:<br />
<br />
<!--T:126--><br />
{{console|body=<br />
# /root/metro/scripts/buildrepo zap<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv7a_hardfp/.control/.failcount...<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv6j_hardfp/.control/.failcount...<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv5te/.control/.failcount...<br />
}}<br />
<br />
== Repository Maintenance == <!--T:127--><br />
<br />
<!--T:128--><br />
A couple of repository maintenance tools are provided:<br />
<br />
<!--T:129--><br />
* {{c|buildrepo digestgen}} will generate hash files for the archives in your repository, and clean up stale hashes. <br />
* {{c|buildrepo index.xml}} will create an index.xml file at the root of your repository, listing all builds available.<br />
* {{c|buildrepo clean}} will output a shell script that will remove old stages. No more than the three most recent stage builds for each build/arch/subarch are kept.<br />
<br />
== Distributed Repositories == <!--T:130--><br />
<br />
<!--T:131--><br />
In many situation, you will have a number of build servers, and each will build a subset of your master repository, and then upload builds to the master repository. This is an area of Metro that is being actively developed. For now, automated upload functionality is not enabled, but is expected to be implemented in the relatively near future. However, it is possible to have your master repository differentiate between subarches that are built locally, and thus should be part of that system's {{c|buildbot}} build rotation, and those that are stored locally and built remotely. These builds should be cleaned when {{c|buildrepo clean}} is run, but should not enter the local build rotation. To set this up, modify {{f|/root/.buildbot}} and use the {{c|subarches}} and {{c|all_subarches}} variables:<br />
<br />
<!--T:132--><br />
{{file|name=/root/.buildbot|desc=Excerpt of .buildbot config for master repository|body=<br />
# subarches we are building locally:<br />
<br />
<!--T:133--><br />
subarches = ( <br />
"pentium4",<br />
"athlon-xp",<br />
"corei7",<br />
"corei7-pure64",<br />
"generic_32", <br />
"i686", <br />
"amd64-k8",<br />
"amd64-k8-pure64",<br />
"core2_64",<br />
"core2_64-pure64",<br />
"generic_64",<br />
"generic_64-pure64",<br />
) <br />
<br />
# Things we need to clean, even if we may not be building:<br />
<br />
all_subarches = subarches + (<br />
"atom_32",<br />
"atom_64",<br />
"atom_64-pure64",<br />
"amd64-k10",<br />
"amd64-k10-pure64",<br />
"amd64-bulldozer",<br />
"amd64-bulldozer-pure64",<br />
"amd64-steamroller",<br />
"amd64-steamroller-pure64",<br />
"amd64-piledriver",<br />
"amd64-piledriver-pure64",<br />
"amd64-jaguar",<br />
"amd64-jaguar-pure64",<br />
"intel64-haswell",<br />
"intel64-haswell-pure64",<br />
"intel64-ivybridge-pure64",<br />
"intel64-ivybridge",<br />
"armv7a_hardfp",<br />
"armv6j_hardfp",<br />
"armv5te"<br />
) <br />
}}<br />
== Using binary cache ==<br />
Metro has built-in feature which allows to use binary packages cache rather then building same list of packages from sources. For example, core packages, such as @system are updated at slower pace and it makes sense to enable binary cache to make stage building blazing fast. However, the real disadvantage with using binary cache could be a core package update that due to internal ABI changes require rebuilding of numerous packages from sources. Good example is {{c|sys-libs/ncurses-5}} to {{c|sys-libs/ncurses-6}} major update. This is the case when you would need to disable binary cache and use regular ebuild installation from sources. To enable binary cache, in your metro git repository copy, edit the {{c|common.conf}} <br />
{{file|name=/etc/builds/common.conf|desc=Excerpt of default common.conf|body=<br />
[section metro]<br />
<br />
options:<br />
options/stage:<br />
target: gentoo<br />
}}<br />
and set {{c|cache/package}}<br />
{{file|name=/etc/builds/common.conf|desc=Excerpt of common.conf with binary cache enabled|body=<br />
[section metro]<br />
<br />
options:<br />
options/stage: cache/package<br />
target: gentoo<br />
}}<br />
<br />
During stage build metro will save package cache in {{c|/var/tmp/metro/cache/package-cache}}. With any next builds this binary package cache will be used.<br />
<!--T:134--><br />
[[Category:HOWTO]]<br />
[[Category:Metro]]<br />
__TOC__<br />
</translate><br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Funtoo:Metro&diff=22394Funtoo:Metro2018-08-27T19:05:26Z<p>Oleg: </p>
<hr />
<div><languages/><br />
<translate><br />
<!--T:1--><br />
{{#layout:doc}}{{#widget:AddThis}}Metro is the build system for Funtoo Linux and [[Gentoo Linux]] stages. It automates the bootstrapping process.<br />
<br />
<!--T:2--><br />
This tutorial will take you through installing, setting up and running Metro.<br />
<br />
<!--T:3--><br />
Other [[:Category:Metro|Metro Documentation]] is available.<br />
<br />
= Preface = <!--T:4--> <br />
<br />
{{Note|If you are unfamiliar with how metro works it is recommended that you continue reading this section so you can become familiar with it. If you are already familiar with it, you may wish to use the new [[Metro AutoSetup|autosetup]] script which uses a curses based menu and allows for quickly setting up and running builds base on your choices without requiring any manual steps. Please see the [[Metro AutoSetup]] page for more details}}<br />
<br />
== How Metro Works == <!--T:5--> <br />
<br />
<!--T:6--><br />
Metro is the Funtoo Linux automated build system, and is used to build Funtoo Linux stage tarballs.<br />
<br />
<!--T:7--><br />
Metro cannot create a stage tarball out of thin air. To build a new stage tarball, Metro must use an existing, older stage tarball called a "seed" stage. This seed stage typically is used as the ''build environment'' for creating the stage we want.<br />
<br />
<!--T:8--><br />
Metro can use two kinds of seed stages. Traditionally, Metro has used a stage3 as a seed stage. This stage3 is then used to build a new stage1, which in turn is used to build a new stage2, and then a new stage3. This is generally the most reliable way to build [[Gentoo Linux]] or Funtoo Linux, so it's the recommended approach.<br />
<br />
== Seeds and Build Isolation == <!--T:9--><br />
<br />
<!--T:10--><br />
Another important concept to mention here is something called ''build isolation''. Because Metro creates an isolated build environment, and the build environment is explicitly defined using existing, tangible entities -- a seed stage and a portage snapshot -- you will get consistent, repeatable results. In other words, the same seed stage, portage snapshot and build instructions will generate an essentially identical result, even if you perform the build a month later on someone else's workstation.<br />
<br />
== Local Build == <!--T:11--> <br />
<br />
<!--T:12--><br />
Say you wanted to build a new <tt>pentium4</tt> stage3 tarball. The recommended method of doing this would be to grab an existing <tt>pentium4</tt> stage3 tarball to use as your seed stage. Metro will be told to use this existing <tt>pentium4</tt> stage3 to build a new stage1 for the same <tt>pentium4</tt>. For this process, the generic <tt>pentium4</tt> stage3 would provide the ''build environment'' for creating our new stage1. Then, the new stage1 would serve as the build environment for creating the new <tt>pentium4</tt> stage2. And the new <tt>pentium4</tt> stage2 would serve as the build environment for creating the new <tt>pentium4</tt> stage3.<br />
<br />
<!--T:13--><br />
In the Metro terminology this is called a '''local build''', which means a stage3 of a given architecture is used to seed a brand new build of the same architecture. Incidentally this will be the first exercise we are going to perform in this tutorial.<br />
<br />
<!--T:14--><br />
A week later, you may want to build a brand new <tt>pentium4</tt> stage3 tarball. Rather than starting from the original <tt>pentium4</tt> stage3 again, you'd probably configure Metro to use the most-recently-built <tt>pentium4</tt> stage3 as the seed. Metro has built-in functionality to make this easy, allowing it to easily find and track the most recent stage3 seed available.<br />
<br />
== Remote Build == <!--T:15--> <br />
<br />
<!--T:16--><br />
Metro can also perform '''remote build''', where a stage3 of a different, but binary compatible, architecture is used as a seed to build a different architecture stage3. Consequentiality the second exercise we are going to perform in this tutorial will be to build a <tt>core2 32bit</tt> stage3 tarball from the <tt>pentium4</tt> stage3 tarball we have just built.<br />
<br />
<!--T:17--><br />
TODO: add caveats about what archs can be seeded and what can be not (maybe a table?)<br />
<br />
== Tailored Build == <!--T:18--> <br />
<br />
<!--T:19--><br />
Last, it's also worthy noting that both in <tt>local</tt> and <tt>remote builds</tt>, Metro can be configured to add and/or remove individual packages to the final tarball.<br />
Let's say you can't live without <tt>app-misc/screen</tt>, at the end of this tutorial, we will show how to have your tailored stage3 to include it.<br />
<br />
== Installing Metro == <!--T:20--><br />
<br />
<!--T:21--><br />
'''The recommended and supported method''' is to use the Git repository of Metro. <br />
<br />
<!--T:22--><br />
Ensure that {{Package|dev-vcs/git}}, {{Package|dev-python/requests}} and {{Package|dev-python/boto}} (optional; required for EC2 support) are installed on your system. <br />
<br />
<!--T:23--><br />
{{console|body=<br />
# ##i##emerge dev-vcs/git dev-python/requests dev-python/boto <br />
}}<br />
<br />
<!--T:24--><br />
Next, clone the master git repository as follows:<br />
<br />
<!--T:25--><br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone git://github.com/funtoo/metro.git<br />
# ##i##cp /root/metro/metro.conf ~/.metro<br />
}}<br />
<br />
<!--T:26--><br />
You will now have a directory called {{c|/root/metro}} that contains all the Metro source code.<br />
<br />
<!--T:27--><br />
=== Setting up ego===<br />
Now, we will set the {{c|ego}}, administration tool of Funtoo/Linux. The way it is used with metro is independent from {{c|app-admin/ego}} installed on your box. Setup is easy as follows:<br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone https://github.com/funtoo/ego,git<br />
}}<br />
This way you will have {{c|/root/ego}} directory with {{c|ego}} binary that is then used by metro.<br />
<br />
<!--T:28--><br />
Metro is now installed. It's time to customize it for your local system.<br />
<br />
= Configuring Metro = <!--T:28--><br />
<br />
<!--T:29--><br />
{{Note|Metro is not currently able to build Gentoo stages. See {{Bug|FL-901}}.}}<br />
<br />
<!--T:30--><br />
[[User:Drobbins|Daniel Robbins]] maintains Metro, so it comes pre-configured to successfully build Funtoo Linux releases. Before reading further, you might want to customize some basic settings like the number of concurrent jobs to fit your hardware's capabilities or the directory to use for produced stage archives. This is accomplished by editing {{c|~/.metro}} which is the Metro's master configuration file.<br />
<br />
<!--T:31--><br />
Please note that {{c|path/install}} must point to where metro was installed. Point {{c|path/distfiles}} to where your distfiles reside. Also set {{c|path/mirror/owner}} and {{c|path/mirror/group}} to the owner and group of all the files that will be written to the build repository directory, which by default (as per the configuration file) is at {{c|/home/mirror/funtoo}}. The cache directory normally resides inside the temp directory -- this can be modified as desired. The cache directory can end up holding many cached .tbz2 packages, and eat up a lot of storage. You may want to place the temp directory on faster storage, for faster compile times, and place the cache directory on slower, but more plentiful storage.<br />
<br />
<!--T:32--><br />
{{file|name=.metro|desc=Metro configuration|body=<br />
# Main metro configuration file - these settings need to be tailored to your install:<br />
<br />
<!--T:33--><br />
[section path]<br />
install: /root/metro<br />
tmp: /var/tmp/metro<br />
cache: $[path/tmp]/cache<br />
distfiles: /var/src/distfiles<br />
work: $[path/tmp]/work/$[target/build]/$[target/name]<br />
<br />
<!--T:34--><br />
[section path/mirror]<br />
<br />
<!--T:35--><br />
: /home/mirror/funtoo<br />
owner: root<br />
group: repomgr<br />
dirmode: 775<br />
<br />
<!--T:36--><br />
[section portage]<br />
<br />
<!--T:37--><br />
MAKEOPTS: auto <br />
<br />
<!--T:38--><br />
[section emerge]<br />
<br />
<!--T:39--><br />
options: --jobs=4 --load-average=4 --keep-going=n<br />
<br />
<!--T:40--><br />
# This line should not be modified:<br />
[collect $[path/install]/etc/master.conf]<br />
}}<br />
<br />
== Arch and Subarch == <!--T:41--><br />
<br />
<!--T:42--><br />
In the following example we are creating a pentium4 stage 3 compiled for x86-32bit binary compatibility. Pentium4 is a subarch of the x86-32bit architecture. Once you have metro installed you may find a full list of each subarch in your {{f|/var/git/meta-repo/kits/core-kit/profiles/funtoo-1.0/linux-gnu/arch/x86-32bit/subarch}} directory:<br />
Example:<br />
{{console|body=<br />
# ##i##ls /var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-32bit/subarch/<br />
amd64-k8+sse3_32 athlon-4 athlon-xp core2_32 i486 k6-2 pentium pentium2 pentiumpro<br />
amd64-k8_32 athlon-mp atom_32 generic_32 i686 k6-3 pentium-m pentium3 prescott<br />
athlon athlon-tbird btver1 geode k6 native_32 pentium-mmx pentium4 xen-pentium4+sse3<br />
}}<br />
<br />
64-bit PC profiles can be found in the {{f|/var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-64bit/subarch/}} directory:<br />
{{console|body=<br />
# ##i##ls /var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-64bit/subarch/<br />
amd64-bulldozer amd64-k8+sse3 btver1_64 generic_64 intel64-nehalem native_64<br />
amd64-jaguar amd64-piledriver core-avx-i intel64-broadwell intel64-sandybridge nocona<br />
amd64-k10 amd64-steamroller core2_64 intel64-haswell intel64-silvermont opteron_64<br />
amd64-k8 atom_64 corei7 intel64-ivybridge intel64-westmere xen-pentium4+sse3_64<br />
}}<br />
<br />
= First stages build (local build) = <!--T:43--><br />
<br />
<!--T:44--><br />
To get this all started, we need to bootstrap the process by downloading an initial seed stage3 to use for building and place it in its proper location in {{f|/home/mirror/funtoo}}, so that Metro can find it. We will also need to create some special &quot;control&quot; files in {{f|/home/mirror/funtoo}}, which will allow Metro to understand how it is supposed to proceed.<br />
<br />
== Step 1: Set up pentium4 repository (local build) == <!--T:45--><br />
<br />
<!--T:46--><br />
Assuming we're following the basic steps outlined in the previous section, and building {{f|funtoo-current}} build for the {{f|pentium4}}, using a generic {{f|pentium4}} stage3 as a seed stage, then here the first set of steps we'd perform:<br />
<br />
<!--T:47--><br />
{{console|body=<br />
# ##i##install -d /home/mirror/funtoo/funtoo-current/x86-32bit/pentium4<br />
# ##i##install -d /home/mirror/funtoo/funtoo-current/snapshots<br />
# ##i##cd /home/mirror/funtoo/funtoo-current/x86-32bit/pentium4<br />
# ##i##install -d 2017-10-01<br />
# ##i##cd 2017-10-01<br />
# ##i##wget -c https://build.funtoo.org/funtoo-current/x86-32bit/pentium4/2017-10-01/stage3-pentium4-funtoo-current-2017-10-01.tar.xz<br />
# ##i##cd ..<br />
# ##i##install -d .control/version<br />
# ##i##echo "2017-10-01" > .control/version/stage3<br />
# ##i##install -d .control/strategy<br />
# ##i##echo local > .control/strategy/build<br />
# ##i##echo stage3 > .control/strategy/seed<br />
}}<br />
<br />
<!--T:48--><br />
OK, let's review the steps above. First, we create the directory {{f|/home/mirror/funtoo/funtoo-current/x86-32bit/pentium4}}, which is where Metro will expect to find {{f|funtoo-current}} pentium4 builds -- it is configured to look here by default. Then we create a specially-named directory to house our seed x86 stage3. Again, by default, Metro expects the directory to be named this way. We enter this directory, and download our seed x86 stage3 from funtoo.org. Note that the {{f|2017-10-01}} version stamp matches. Make sure that your directory name matches the stage3 name too. Everything has been set up to match Metro's default filesystem layout.<br />
<br />
<!--T:49--><br />
Next, we go back to the {{f|/home/mirror/metro/funtoo-current/x86-32bit/pentium4}} directory, and inside it, we create a {{f|.control}} directory. This directory and its subdirectories contain special files that Metro references to determine certain aspects of its behavior. The {{f|.control/version/stage3}} file is used by Metro to track the most recently-built stage3 for this particular build and subarch. Metro will automatically update this file with a new version stamp after it successfully builds a new stage3. But because Metro didn't actually ''build'' this stage3, we need to set up the {{f|.control/version/stage3}} file manually. This will allow Metro to find our downloaded stage3 when we set up our pentium4 build to use it as a seed. Also note that Metro will create a similar {{f|.control/version/stage1}} file after it successfully builds an pentium4 funtoo-current stage1.<br />
<br />
<!--T:50--><br />
We also set up {{f|.control/strategy/build}} and {{f|.control/strategy/seed}} files with values of {{f|local}} and {{f|stage3}} respectively. These files define the building strategy Metro will use when we build pentium4 funtoo-current stages. With a build strategy of {{f|local}}, Metro will source its seed stage from funtoo-current pentium4, the current directory. And with a seed strategy of {{f|stage3}}, Metro will use a stage3 as a seed, and use this seed to build a new stage1, stage2 and stage3.<br />
<br />
== Step 2: Building the pentium4 stages == <!--T:51--><br />
<br />
<!--T:52--><br />
Incidentally, if all you wanted to do at this point was to build a new pentium4 funtoo-current stage1/2/3 (plus openvz and vserver templates). You would begin the process by typing:<br />
<br />
<!--T:53--><br />
{{console|body=<br />
# ##i##cd /root/metro<br />
# ##i##scripts/ezbuild.sh funtoo-current x86-32bit pentium4<br />
}}<br />
<br />
<!--T:54--><br />
If you have a slow machine, it could take several hours to be completed because several "heavy" components like gcc or glibc have to be recompiled in each stage. Once a stage has been successfully completed, it is placed in the {{f|"${METRO_MIRROR}/funtoo-current/x32-bit/pentium4/YYYY-MM-DD"}} subdirectory, where {{f|YYYY-MM-DD}} is today's date at the time the {{f|ezbuild.sh}} script was started or the date you put on the ezscript.sh command line.<br />
<br />
= Building for another binary compatible architecture (remote build) = <!--T:55--><br />
<br />
<!--T:56--><br />
As written above, Metro is able to perform '''remote build''' building different architecture stage3 from a binary compatible seeding stage3 (e.g. using a pentium4 stage3 to seed a <tt>Intel Core2 32bits</tt> stage3). <br />
<br />
<!--T:57--><br />
In the Metro terminology this is called a '''remote build''' (a stage 3 of a different, but binary compatible, architecture is used as a seed). <br />
What's not compatible? You can't use a <tt>Sparc</tt> architecture to generate an <tt>x86</tt> or <tt>ARM</tt> based stage and vice-versa. If you use a 32bit stage then you don't want to seed a 64bit build from it. Be sure that you are using a stage from the same architecture that you are trying to seed. Check [http://ftp.osuosl.org/pub/funtoo/funtoo-current/ Funtoo-current FTP Mirror] for a stage that is from the same Architecture that you will be building. <br />
<br />
<!--T:58--><br />
{{Note|Often, one build (ie. funtoo-current) can be used as a seed for another build such as funtoo-stable. However, hardened builds require hardened stages as seeds in order for the build to complete successfully.}}<br />
<br />
== Step 1: Set up Core_2 32bit repository == <!--T:59--><br />
<br />
<!--T:60--><br />
In this example, we're going to use this pentium4 funtoo-current stage3 to seed a new Core_2 32bit funtoo-current build. To get that done, we need to set up the pentium4 build directory as follows:<br />
<br />
<!--T:61--><br />
{{console|body=<br />
# ##i## cd /home/mirror/funtoo/funtoo-current/x86-32bit<br />
# ##i##install -d core2_32<br />
# ##i##cd core2_32<br />
# ##i##install -d .control/strategy<br />
# ##i##echo remote > .control/strategy/build<br />
# ##i##echo stage3 > .control/strategy/seed<br />
# ##i##install -d .control/remote<br />
# ##i##echo funtoo-current > .control/remote/build<br />
# ##i##echo x86-32bit > .control/remote/arch_desc<br />
# ##i##echo pentium4 > .control/remote/subarch<br />
}}<br />
<br />
<!--T:62--><br />
The steps we follow are similar to those we performed for a ''local build'' to set up our pentium4 directory for local build. However, note the differences. We didn't download a stage, because we are going to use the pentium4 stage to build a new Core_2 32bit stage. We also didn't create the <tt>.control/version/stage{1,3}</tt> files because Metro will create them for us after it successfully builds a new stage1 and stage3. We are still using a <tt>stage3</tt> seed strategy, but we've set the build strategy to <tt>remote</tt>, which means that we're going to use a seed stage that's not from this particular subdirectory. Where are we going to get it from? The <tt>.control/remote</tt> directory contains this information, and lets Metro know that it should look for its seed stage3 in the <tt>/home/mirror/funtoo/funtoo-current/x86-32bit/pentium4</tt> directory. Which one will it grab? You guessed it -- the most recently built ''stage3'' (since our seed strategy was set to <tt>stage3</tt>) that has the version stamp of <tt>2010-12-24</tt>, as recorded in <tt>/home/mirror/funtoo-current/x86-32bit/pentium4/.control/version/stage</tt>. Now you can see how all those control files come together to direct Metro to do the right thing.<br />
<br />
<!--T:63--><br />
{{Note|<code>arch_desc</code> should be set to one of: <code>x86-32bit</code>, <code>x86-64bit</code> or <code>pure64</code> for PC-compatible systems. You must use a 32-bit build as a seed for other 32-bit builds, and a 64-bit build as a seed for other 64-bit builds.}}<br />
<br />
== Step 2: Building the Core_2 32bit stages == <!--T:64--><br />
<br />
<!--T:65--><br />
Now, you could start building your new Core_2 32bit stage1/2/3 (plus openvz and vserver templates) by typing the following:<br />
<br />
<!--T:66--><br />
<console><br />
# ##i##/root/metro/scripts/ezbuild.sh funtoo-current x86-32bit core2_32<br />
</console><br />
<br />
<!--T:67--><br />
In that case, the produced stages are placed in the <tt>/home/mirror/funtoo/funtoo-current/x32-bit/core2_32/YYYY-MM-DD</tt> subdirectory.<br />
<br />
== Step 3: The Next Build == <!--T:68--><br />
<br />
<!--T:69--><br />
At this point, you now have a new Core_2 32bit stage3, built using a "remote" pentium4 stage3. Once the first remote build completes successfully, metro will automatically change {{c|.control/strategy/build}} to be {{c|local}} instead of {{c|remote}}, so it will use the most recently-built Core_2 32bit stage3 as a seed for any new Core_2 32bit builds from now on.<br />
<br />
= Build your own tailored stage3 = <!--T:70--><br />
<br />
<!--T:71--><br />
Metro can be easily configured for building custom stage3 by including additional packages. You can find following directory {{c|/etc/builds/packages}} in your copy of metro repository and a corresponding {{c|arch}} configuration files inside:<br />
{{file|name=/etc/builds/packages/x86-64bit.conf|body=<br />
[section emerge]<br />
<br />
packages: [<br />
sys-kernel/debian-sources<br />
]<br />
}}<br />
Notice a {{c|debian-sources}} ebuild is added for all 64-bit stages. Modify the file to include (or exclude in case Funtoo add additional) packages of your choice. They will be included in your custom stage3 portage's world file.<br />
<br />
= Building Gentoo stages = <!--T:98--><br />
<br />
<!--T:99--><br />
Metro can also build Gentoo stages. After switching to Funtoo profile, see http://www.funtoo.org/Funtoo_Profiles metro require additional steps for this. We have an open bug for this -- it is simply due to the fact that we focus on ensuring Funtoo Linux builds and building Gentoo is a lower priority. Historical note: Funtoo Linux originally started as a fork of Gentoo Linux so that metro could reliably build Gentoo stages.<br />
http://www.funtoo.org/Funtoo_Profiles<br />
<br />
= Advanced Features = <!--T:100--><br />
<br />
<!--T:101--><br />
Metro also includes a number of advanced features that can be used to automate builds and set up distributed build servers. These features require you to {{c|emerge sqlalchemy}}, as SQLite is used as a dependency and also {{c|emerge dev-python/lxml}} as this is needed for index file generation.<br />
<br />
== Repository Management == <!--T:102--><br />
<br />
<!--T:103--><br />
Metro includes a script in the {{c|scripts}} directory called {{c|buildrepo}}. Buildrepo serves as the heart of Metro's advanced repository management features.<br />
<br />
=== Initial Setup === <!--T:104--><br />
<br />
<!--T:105--><br />
To use {{c|buildrepo}}, you will first need to create a {{f|.buildbot}} configuration file. Here is the file I use on my AMD Jaguar build server:<br />
<br />
<!--T:106--><br />
{{file|name=/root/.buildbot|lang=python|body=<br />
builds = (<br />
"funtoo-current",<br />
"funtoo-current-hardened",<br />
)<br />
<br />
<!--T:107--><br />
arches = (<br />
"x86-64bit",<br />
"pure64"<br />
)<br />
<br />
<!--T:108--><br />
subarches = (<br />
"amd64-jaguar",<br />
"amd64-jaguar-pure64",<br />
)<br />
<br />
<!--T:109--><br />
def map_build(build, subarch, full, full_date):<br />
# arguments refer to last build...<br />
if full == True:<br />
buildtype = ( "freshen", )<br />
else:<br />
buildtype = ("full", )<br />
# return value can be a string like "full+openvz" or a sequence type like [ "freshen", "openvz" ]<br />
return buildtype<br />
}}<br />
<br />
<!--T:110--><br />
This file is actually a python source file that defines the tuples {{c|builds}}, {{c|arches}} and {{c|subarches}}. These variables tell {{c|buildrepo}} which builds, arches and subarches it should manage. A {{c|map_build()}} function is also defined which {{c|buildbot}} uses to determine what kind of build to perform. The arguments passed to the function are based on the last successful build. The function can read these arguments and return a string to define the type of the next build. In the above example, the {{c|map_build()}} function will cause the next build after a freshen build to be a full build, and the next build after a full build to be a freshen build, so that the build will alternate between full and freshen.<br />
<br />
== Automated Builds == <!--T:111--><br />
<br />
<!--T:112--><br />
Once the {{c|.buildbot}} file has been created, the {{c|buildrepo}} and {{c|buildbot.sh}} tools are ready to use. Here's how they work. These tools are designed to keep your repository ({{c|path/mirror}} in {{f|/root/.metro}} up-to-date by inspecting your repository and looking for stages that are out-of-date. <br />
<br />
<!--T:113--><br />
To list the next build that will be performed, do this -- this is from my ARM build server:<br />
<br />
<!--T:114--><br />
{{console|body=<br />
# ##i##./buildrepo nextbuild<br />
build=funtoo-current<br />
arch_desc=arm-32bit<br />
subarch=armv7a_hardfp<br />
fulldate=2015-02-08<br />
nextdate=2015-02-20<br />
failcount=0<br />
target=full<br />
extras=''<br />
}}<br />
<br />
<!--T:115--><br />
If no output is displayed, then all your builds are up-to-date.<br />
<br />
<!--T:116--><br />
To actually run the next build, run {{c|buildbot.sh}}:<br />
<br />
<!--T:117--><br />
{{console|body=<br />
# ##i##./buildbot.sh<br />
}}<br />
<br />
<!--T:118--><br />
If you're thinking that {{c|buildbot.sh}} would be a good candidate for a cron job, you've got the right idea!<br />
<br />
=== List Builds === <!--T:119--><br />
<br />
<!--T:120--><br />
To get a quick look at our repository, let's run the {{c|buildrepo fails}} command:<br />
<br />
<!--T:121--><br />
{{console|body=<br />
# ##i##./buildrepo fails<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current/x86-64bit/amd64-jaguar<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current/pure64/amd64-jaguar-pure64<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current-hardened/x86-64bit/amd64-jaguar<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current-hardened/pure64/amd64-jaguar-pure64<br />
}}<br />
<br />
<!--T:122--><br />
On my AMD Jaguar build server, on Feb 20, 2015, this lists all the builds that {{c|buildrepo}} has been configured to manage. The first number on each line is a '''failcount''', which is the number of consecutive times that the build has failed. A zero value indicates that everything's okay. The failcount is an important feature of the advanced repository management features. Here are a number of behaviors that are implemented based on failcount:<br />
<br />
<!--T:123--><br />
* If {{c|buildbot.sh}} tries to build a stage and the build fails, the failcount is incremented.<br />
* If the build succeeds for a particular build, the failcount is reset to zero.<br />
* Builds with the lowest failcount are prioritized by {{buildrepo}} to build next, to steer towards builds that are more likely to complete successfully.<br />
* Once the failcount reaches 3 for a particular build, it is removed from the build rotation.<br />
<br />
=== Resetting Failcount === <!--T:124--><br />
<br />
<!--T:125--><br />
If a build has issues, the failcount for a build will reach 3, at which point it will be pulled out of build rotation. To clear failcount, so that these builds are attempted again -- possibly fixed by new updates to the Portage tree -- use {{c|buildrepo zap}}:<br />
<br />
<!--T:126--><br />
{{console|body=<br />
# /root/metro/scripts/buildrepo zap<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv7a_hardfp/.control/.failcount...<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv6j_hardfp/.control/.failcount...<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv5te/.control/.failcount...<br />
}}<br />
<br />
== Repository Maintenance == <!--T:127--><br />
<br />
<!--T:128--><br />
A couple of repository maintenance tools are provided:<br />
<br />
<!--T:129--><br />
* {{c|buildrepo digestgen}} will generate hash files for the archives in your repository, and clean up stale hashes. <br />
* {{c|buildrepo index.xml}} will create an index.xml file at the root of your repository, listing all builds available.<br />
* {{c|buildrepo clean}} will output a shell script that will remove old stages. No more than the three most recent stage builds for each build/arch/subarch are kept.<br />
<br />
== Distributed Repositories == <!--T:130--><br />
<br />
<!--T:131--><br />
In many situation, you will have a number of build servers, and each will build a subset of your master repository, and then upload builds to the master repository. This is an area of Metro that is being actively developed. For now, automated upload functionality is not enabled, but is expected to be implemented in the relatively near future. However, it is possible to have your master repository differentiate between subarches that are built locally, and thus should be part of that system's {{c|buildbot}} build rotation, and those that are stored locally and built remotely. These builds should be cleaned when {{c|buildrepo clean}} is run, but should not enter the local build rotation. To set this up, modify {{f|/root/.buildbot}} and use the {{c|subarches}} and {{c|all_subarches}} variables:<br />
<br />
<!--T:132--><br />
{{file|name=/root/.buildbot|desc=Excerpt of .buildbot config for master repository|body=<br />
# subarches we are building locally:<br />
<br />
<!--T:133--><br />
subarches = ( <br />
"pentium4",<br />
"athlon-xp",<br />
"corei7",<br />
"corei7-pure64",<br />
"generic_32", <br />
"i686", <br />
"amd64-k8",<br />
"amd64-k8-pure64",<br />
"core2_64",<br />
"core2_64-pure64",<br />
"generic_64",<br />
"generic_64-pure64",<br />
) <br />
<br />
# Things we need to clean, even if we may not be building:<br />
<br />
all_subarches = subarches + (<br />
"atom_32",<br />
"atom_64",<br />
"atom_64-pure64",<br />
"amd64-k10",<br />
"amd64-k10-pure64",<br />
"amd64-bulldozer",<br />
"amd64-bulldozer-pure64",<br />
"amd64-steamroller",<br />
"amd64-steamroller-pure64",<br />
"amd64-piledriver",<br />
"amd64-piledriver-pure64",<br />
"amd64-jaguar",<br />
"amd64-jaguar-pure64",<br />
"intel64-haswell",<br />
"intel64-haswell-pure64",<br />
"intel64-ivybridge-pure64",<br />
"intel64-ivybridge",<br />
"armv7a_hardfp",<br />
"armv6j_hardfp",<br />
"armv5te"<br />
) <br />
}}<br />
== Using binary cache ==<br />
Metro has built-in feature which allows to use binary packages cache rather then building same list of packages from sources. For example, core packages, such as @system are updated at slower pace and it makes sense to enable binary cache to make stage building blazing fast. However, the real disadvantage with using binary cache could be a core package update that due to internal ABI changes require rebuilding of numerous packages from sources. Good example is {{c|sys-libs/ncurses-5}} to {{c|sys-libs/ncurses-6}} major update. This is the case when you would need to disable binary cache and use regular ebuild installation from sources. To enable binary cache, in your metro git repository copy, edit the {{c|common.conf}} <br />
{{file|name=/etc/builds/common.conf|desc=Excerpt of default common.conf|body=<br />
[section metro]<br />
<br />
options:<br />
options/stage:<br />
target: gentoo<br />
}}<br />
and set {{c|cache/package}}<br />
{{file|name=/etc/builds/common.conf|desc=Excerpt of common.conf with binary cache enabled|body=<br />
[section metro]<br />
<br />
options:<br />
options/stage: cache/package<br />
target: gentoo<br />
}}<br />
<br />
During stage build metro will save package cache in {{c|/var/tmp/metro/cache/package-cache}}. With any next builds this binary package cache will be used.<br />
<!--T:134--><br />
[[Category:HOWTO]]<br />
[[Category:Metro]]<br />
__TOC__<br />
</translate><br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Funtoo:Metro&diff=22393Funtoo:Metro2018-08-27T19:04:59Z<p>Oleg: </p>
<hr />
<div><languages/><br />
<translate><br />
<!--T:1--><br />
{{#layout:doc}}{{#widget:AddThis}}Metro is the build system for Funtoo Linux and [[Gentoo Linux]] stages. It automates the bootstrapping process.<br />
<br />
<!--T:2--><br />
This tutorial will take you through installing, setting up and running Metro.<br />
<br />
<!--T:3--><br />
Other [[:Category:Metro|Metro Documentation]] is available.<br />
<br />
= Preface = <!--T:4--> <br />
<br />
{{Note|If you are unfamiliar with how metro works it is recommended that you continue reading this section so you can become familiar with it. If you are already familiar with it, you may wish to use the new [[Metro AutoSetup|autosetup]] script which uses a curses based menu and allows for quickly setting up and running builds base on your choices without requiring any manual steps. Please see the [[Metro AutoSetup]] page for more details}}<br />
<br />
== How Metro Works == <!--T:5--> <br />
<br />
<!--T:6--><br />
Metro is the Funtoo Linux automated build system, and is used to build Funtoo Linux stage tarballs.<br />
<br />
<!--T:7--><br />
Metro cannot create a stage tarball out of thin air. To build a new stage tarball, Metro must use an existing, older stage tarball called a "seed" stage. This seed stage typically is used as the ''build environment'' for creating the stage we want.<br />
<br />
<!--T:8--><br />
Metro can use two kinds of seed stages. Traditionally, Metro has used a stage3 as a seed stage. This stage3 is then used to build a new stage1, which in turn is used to build a new stage2, and then a new stage3. This is generally the most reliable way to build [[Gentoo Linux]] or Funtoo Linux, so it's the recommended approach.<br />
<br />
== Seeds and Build Isolation == <!--T:9--><br />
<br />
<!--T:10--><br />
Another important concept to mention here is something called ''build isolation''. Because Metro creates an isolated build environment, and the build environment is explicitly defined using existing, tangible entities -- a seed stage and a portage snapshot -- you will get consistent, repeatable results. In other words, the same seed stage, portage snapshot and build instructions will generate an essentially identical result, even if you perform the build a month later on someone else's workstation.<br />
<br />
== Local Build == <!--T:11--> <br />
<br />
<!--T:12--><br />
Say you wanted to build a new <tt>pentium4</tt> stage3 tarball. The recommended method of doing this would be to grab an existing <tt>pentium4</tt> stage3 tarball to use as your seed stage. Metro will be told to use this existing <tt>pentium4</tt> stage3 to build a new stage1 for the same <tt>pentium4</tt>. For this process, the generic <tt>pentium4</tt> stage3 would provide the ''build environment'' for creating our new stage1. Then, the new stage1 would serve as the build environment for creating the new <tt>pentium4</tt> stage2. And the new <tt>pentium4</tt> stage2 would serve as the build environment for creating the new <tt>pentium4</tt> stage3.<br />
<br />
<!--T:13--><br />
In the Metro terminology this is called a '''local build''', which means a stage3 of a given architecture is used to seed a brand new build of the same architecture. Incidentally this will be the first exercise we are going to perform in this tutorial.<br />
<br />
<!--T:14--><br />
A week later, you may want to build a brand new <tt>pentium4</tt> stage3 tarball. Rather than starting from the original <tt>pentium4</tt> stage3 again, you'd probably configure Metro to use the most-recently-built <tt>pentium4</tt> stage3 as the seed. Metro has built-in functionality to make this easy, allowing it to easily find and track the most recent stage3 seed available.<br />
<br />
== Remote Build == <!--T:15--> <br />
<br />
<!--T:16--><br />
Metro can also perform '''remote build''', where a stage3 of a different, but binary compatible, architecture is used as a seed to build a different architecture stage3. Consequentiality the second exercise we are going to perform in this tutorial will be to build a <tt>core2 32bit</tt> stage3 tarball from the <tt>pentium4</tt> stage3 tarball we have just built.<br />
<br />
<!--T:17--><br />
TODO: add caveats about what archs can be seeded and what can be not (maybe a table?)<br />
<br />
== Tailored Build == <!--T:18--> <br />
<br />
<!--T:19--><br />
Last, it's also worthy noting that both in <tt>local</tt> and <tt>remote builds</tt>, Metro can be configured to add and/or remove individual packages to the final tarball.<br />
Let's say you can't live without <tt>app-misc/screen</tt>, at the end of this tutorial, we will show how to have your tailored stage3 to include it.<br />
<br />
== Installing Metro == <!--T:20--><br />
<br />
<!--T:21--><br />
'''The recommended and supported method''' is to use the Git repository of Metro. <br />
<br />
<!--T:22--><br />
Ensure that {{Package|dev-vcs/git}}, {{Package|dev-python/requests}} and {{Package|dev-python/boto}} (optional; required for EC2 support) are installed on your system. <br />
<br />
<!--T:23--><br />
{{console|body=<br />
# ##i##emerge dev-vcs/git dev-python/requests dev-python/boto <br />
}}<br />
<br />
<!--T:24--><br />
Next, clone the master git repository as follows:<br />
<br />
<!--T:25--><br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone git://github.com/funtoo/metro.git<br />
# ##i##cp /root/metro/metro.conf ~/.metro<br />
}}<br />
<br />
<!--T:26--><br />
You will now have a directory called {{c|/root/metro}} that contains all the Metro source code.<br />
<br />
<!--T:27--><br />
=== Setting up ego===<br />
Now, we will set the {{c|ego}}, administration tool of Funtoo/Linux. The way it is used with metro is independent from {{c|app-admin/ego}} installed on your box. Setup is easy as follows:<br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone https://github.com/funtoo/ego,git<br />
}}<br />
This way you will have {{c|/root/ego}} directory with {{c|ego}} binary that is then used by metro.<br />
<br />
<!--T:28--><br />
Metro is now installed. It's time to customize it for your local system.<br />
<br />
= Configuring Metro = <!--T:28--><br />
<br />
<!--T:29--><br />
{{Note|Metro is not currently able to build Gentoo stages. See {{Bug|FL-901}}.}}<br />
<br />
<!--T:30--><br />
[[User:Drobbins|Daniel Robbins]] maintains Metro, so it comes pre-configured to successfully build Funtoo Linux releases. Before reading further, you might want to customize some basic settings like the number of concurrent jobs to fit your hardware's capabilities or the directory to use for produced stage archives. This is accomplished by editing {{c|~/.metro}} which is the Metro's master configuration file.<br />
<br />
<!--T:31--><br />
Please note that {{c|path/install}} must point to where metro was installed. Point {{c|path/distfiles}} to where your distfiles reside. Also set {{c|path/mirror/owner}} and {{c|path/mirror/group}} to the owner and group of all the files that will be written to the build repository directory, which by default (as per the configuration file) is at {{c|/home/mirror/funtoo}}. The cache directory normally resides inside the temp directory -- this can be modified as desired. The cache directory can end up holding many cached .tbz2 packages, and eat up a lot of storage. You may want to place the temp directory on faster storage, for faster compile times, and place the cache directory on slower, but more plentiful storage.<br />
<br />
<!--T:32--><br />
{{file|name=.metro|desc=Metro configuration|body=<br />
# Main metro configuration file - these settings need to be tailored to your install:<br />
<br />
<!--T:33--><br />
[section path]<br />
install: /root/metro<br />
tmp: /var/tmp/metro<br />
cache: $[path/tmp]/cache<br />
distfiles: /var/src/distfiles<br />
work: $[path/tmp]/work/$[target/build]/$[target/name]<br />
<br />
<!--T:34--><br />
[section path/mirror]<br />
<br />
<!--T:35--><br />
: /home/mirror/funtoo<br />
owner: root<br />
group: repomgr<br />
dirmode: 775<br />
<br />
<!--T:36--><br />
[section portage]<br />
<br />
<!--T:37--><br />
MAKEOPTS: auto <br />
<br />
<!--T:38--><br />
[section emerge]<br />
<br />
<!--T:39--><br />
options: --jobs=4 --load-average=4 --keep-going=n<br />
<br />
<!--T:40--><br />
# This line should not be modified:<br />
[collect $[path/install]/etc/master.conf]<br />
}}<br />
<br />
== Arch and Subarch == <!--T:41--><br />
<br />
<!--T:42--><br />
In the following example we are creating a pentium4 stage 3 compiled for x86-32bit binary compatibility. Pentium4 is a subarch of the x86-32bit architecture. Once you have metro installed you may find a full list of each subarch in your {{f|/var/git/meta-repo/kits/core-kit/profiles/funtoo-1.0/linux-gnu/arch/x86-32bit/subarch}} directory:<br />
Example:<br />
{{console|body=<br />
# ##i##ls /var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-32bit/subarch/<br />
amd64-k8+sse3_32 athlon-4 athlon-xp core2_32 i486 k6-2 pentium pentium2 pentiumpro<br />
amd64-k8_32 athlon-mp atom_32 generic_32 i686 k6-3 pentium-m pentium3 prescott<br />
athlon athlon-tbird btver1 geode k6 native_32 pentium-mmx pentium4 xen-pentium4+sse3<br />
}}<br />
<br />
64-bit PC profiles can be found in the {{f|/var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-64bit/subarch/}} directory:<br />
{{console|body=<br />
# ##i##ls /var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-64bit/subarch/<br />
amd64-bulldozer amd64-k8+sse3 btver1_64 generic_64 intel64-nehalem native_64<br />
amd64-jaguar amd64-piledriver core-avx-i intel64-broadwell intel64-sandybridge nocona<br />
amd64-k10 amd64-steamroller core2_64 intel64-haswell intel64-silvermont opteron_64<br />
amd64-k8 atom_64 corei7 intel64-ivybridge intel64-westmere xen-pentium4+sse3_64<br />
}}<br />
<br />
= First stages build (local build) = <!--T:43--><br />
<br />
<!--T:44--><br />
To get this all started, we need to bootstrap the process by downloading an initial seed stage3 to use for building and place it in its proper location in {{f|/home/mirror/funtoo}}, so that Metro can find it. We will also need to create some special &quot;control&quot; files in {{f|/home/mirror/funtoo}}, which will allow Metro to understand how it is supposed to proceed.<br />
<br />
== Step 1: Set up pentium4 repository (local build) == <!--T:45--><br />
<br />
<!--T:46--><br />
Assuming we're following the basic steps outlined in the previous section, and building {{f|funtoo-current}} build for the {{f|pentium4}}, using a generic {{f|pentium4}} stage3 as a seed stage, then here the first set of steps we'd perform:<br />
<br />
<!--T:47--><br />
{{console|body=<br />
# ##i##install -d /home/mirror/funtoo/funtoo-current/x86-32bit/pentium4<br />
# ##i##install -d /home/mirror/funtoo/funtoo-current/snapshots<br />
# ##i##cd /home/mirror/funtoo/funtoo-current/x86-32bit/pentium4<br />
# ##i##install -d 2017-10-01<br />
# ##i##cd 2017-10-01<br />
# ##i##wget -c https://build.funtoo.org/funtoo-current/x86-32bit/pentium4/2017-10-01/stage3-pentium4-funtoo-current-2017-10-01.tar.xz<br />
# ##i##cd ..<br />
# ##i##install -d .control/version<br />
# ##i##echo "2017-10-01" > .control/version/stage3<br />
# ##i##install -d .control/strategy<br />
# ##i##echo local > .control/strategy/build<br />
# ##i##echo stage3 > .control/strategy/seed<br />
}}<br />
<br />
<!--T:48--><br />
OK, let's review the steps above. First, we create the directory {{f|/home/mirror/funtoo/funtoo-current/x86-32bit/pentium4}}, which is where Metro will expect to find {{f|funtoo-current}} pentium4 builds -- it is configured to look here by default. Then we create a specially-named directory to house our seed x86 stage3. Again, by default, Metro expects the directory to be named this way. We enter this directory, and download our seed x86 stage3 from funtoo.org. Note that the {{f|2017-10-01}} version stamp matches. Make sure that your directory name matches the stage3 name too. Everything has been set up to match Metro's default filesystem layout.<br />
<br />
<!--T:49--><br />
Next, we go back to the {{f|/home/mirror/metro/funtoo-current/x86-32bit/pentium4}} directory, and inside it, we create a {{f|.control}} directory. This directory and its subdirectories contain special files that Metro references to determine certain aspects of its behavior. The {{f|.control/version/stage3}} file is used by Metro to track the most recently-built stage3 for this particular build and subarch. Metro will automatically update this file with a new version stamp after it successfully builds a new stage3. But because Metro didn't actually ''build'' this stage3, we need to set up the {{f|.control/version/stage3}} file manually. This will allow Metro to find our downloaded stage3 when we set up our pentium4 build to use it as a seed. Also note that Metro will create a similar {{f|.control/version/stage1}} file after it successfully builds an pentium4 funtoo-current stage1.<br />
<br />
<!--T:50--><br />
We also set up {{f|.control/strategy/build}} and {{f|.control/strategy/seed}} files with values of {{f|local}} and {{f|stage3}} respectively. These files define the building strategy Metro will use when we build pentium4 funtoo-current stages. With a build strategy of {{f|local}}, Metro will source its seed stage from funtoo-current pentium4, the current directory. And with a seed strategy of {{f|stage3}}, Metro will use a stage3 as a seed, and use this seed to build a new stage1, stage2 and stage3.<br />
<br />
== Step 2: Building the pentium4 stages == <!--T:51--><br />
<br />
<!--T:52--><br />
Incidentally, if all you wanted to do at this point was to build a new pentium4 funtoo-current stage1/2/3 (plus openvz and vserver templates). You would begin the process by typing:<br />
<br />
<!--T:53--><br />
{{console|body=<br />
# ##i##cd /root/metro<br />
# ##i##scripts/ezbuild.sh funtoo-current x86-32bit pentium4<br />
}}<br />
<br />
<!--T:54--><br />
If you have a slow machine, it could take several hours to be completed because several "heavy" components like gcc or glibc have to be recompiled in each stage. Once a stage has been successfully completed, it is placed in the {{f|"${METRO_MIRROR}/funtoo-current/x32-bit/pentium4/YYYY-MM-DD"}} subdirectory, where {{f|YYYY-MM-DD}} is today's date at the time the {{f|ezbuild.sh}} script was started or the date you put on the ezscript.sh command line.<br />
<br />
= Building for another binary compatible architecture (remote build) = <!--T:55--><br />
<br />
<!--T:56--><br />
As written above, Metro is able to perform '''remote build''' building different architecture stage3 from a binary compatible seeding stage3 (e.g. using a pentium4 stage3 to seed a <tt>Intel Core2 32bits</tt> stage3). <br />
<br />
<!--T:57--><br />
In the Metro terminology this is called a '''remote build''' (a stage 3 of a different, but binary compatible, architecture is used as a seed). <br />
What's not compatible? You can't use a <tt>Sparc</tt> architecture to generate an <tt>x86</tt> or <tt>ARM</tt> based stage and vice-versa. If you use a 32bit stage then you don't want to seed a 64bit build from it. Be sure that you are using a stage from the same architecture that you are trying to seed. Check [http://ftp.osuosl.org/pub/funtoo/funtoo-current/ Funtoo-current FTP Mirror] for a stage that is from the same Architecture that you will be building. <br />
<br />
<!--T:58--><br />
{{Note|Often, one build (ie. funtoo-current) can be used as a seed for another build such as funtoo-stable. However, hardened builds require hardened stages as seeds in order for the build to complete successfully.}}<br />
<br />
== Step 1: Set up Core_2 32bit repository == <!--T:59--><br />
<br />
<!--T:60--><br />
In this example, we're going to use this pentium4 funtoo-current stage3 to seed a new Core_2 32bit funtoo-current build. To get that done, we need to set up the pentium4 build directory as follows:<br />
<br />
<!--T:61--><br />
<console><br />
# ##i## cd /home/mirror/funtoo/funtoo-current/x86-32bit<br />
# ##i##install -d core2_32<br />
# ##i##cd core2_32<br />
# ##i##install -d .control/strategy<br />
# ##i##echo remote > .control/strategy/build<br />
# ##i##echo stage3 > .control/strategy/seed<br />
# ##i##install -d .control/remote<br />
# ##i##echo funtoo-current > .control/remote/build<br />
# ##i##echo x86-32bit > .control/remote/arch_desc<br />
# ##i##echo pentium4 > .control/remote/subarch<br />
</console><br />
<br />
<!--T:62--><br />
The steps we follow are similar to those we performed for a ''local build'' to set up our pentium4 directory for local build. However, note the differences. We didn't download a stage, because we are going to use the pentium4 stage to build a new Core_2 32bit stage. We also didn't create the <tt>.control/version/stage{1,3}</tt> files because Metro will create them for us after it successfully builds a new stage1 and stage3. We are still using a <tt>stage3</tt> seed strategy, but we've set the build strategy to <tt>remote</tt>, which means that we're going to use a seed stage that's not from this particular subdirectory. Where are we going to get it from? The <tt>.control/remote</tt> directory contains this information, and lets Metro know that it should look for its seed stage3 in the <tt>/home/mirror/funtoo/funtoo-current/x86-32bit/pentium4</tt> directory. Which one will it grab? You guessed it -- the most recently built ''stage3'' (since our seed strategy was set to <tt>stage3</tt>) that has the version stamp of <tt>2010-12-24</tt>, as recorded in <tt>/home/mirror/funtoo-current/x86-32bit/pentium4/.control/version/stage</tt>. Now you can see how all those control files come together to direct Metro to do the right thing.<br />
<br />
<!--T:63--><br />
{{Note|<code>arch_desc</code> should be set to one of: <code>x86-32bit</code>, <code>x86-64bit</code> or <code>pure64</code> for PC-compatible systems. You must use a 32-bit build as a seed for other 32-bit builds, and a 64-bit build as a seed for other 64-bit builds.}}<br />
<br />
== Step 2: Building the Core_2 32bit stages == <!--T:64--><br />
<br />
<!--T:65--><br />
Now, you could start building your new Core_2 32bit stage1/2/3 (plus openvz and vserver templates) by typing the following:<br />
<br />
<!--T:66--><br />
<console><br />
# ##i##/root/metro/scripts/ezbuild.sh funtoo-current x86-32bit core2_32<br />
</console><br />
<br />
<!--T:67--><br />
In that case, the produced stages are placed in the <tt>/home/mirror/funtoo/funtoo-current/x32-bit/core2_32/YYYY-MM-DD</tt> subdirectory.<br />
<br />
== Step 3: The Next Build == <!--T:68--><br />
<br />
<!--T:69--><br />
At this point, you now have a new Core_2 32bit stage3, built using a "remote" pentium4 stage3. Once the first remote build completes successfully, metro will automatically change {{c|.control/strategy/build}} to be {{c|local}} instead of {{c|remote}}, so it will use the most recently-built Core_2 32bit stage3 as a seed for any new Core_2 32bit builds from now on.<br />
<br />
= Build your own tailored stage3 = <!--T:70--><br />
<br />
<!--T:71--><br />
Metro can be easily configured for building custom stage3 by including additional packages. You can find following directory {{c|/etc/builds/packages}} in your copy of metro repository and a corresponding {{c|arch}} configuration files inside:<br />
{{file|name=/etc/builds/packages/x86-64bit.conf|body=<br />
[section emerge]<br />
<br />
packages: [<br />
sys-kernel/debian-sources<br />
]<br />
}}<br />
Notice a {{c|debian-sources}} ebuild is added for all 64-bit stages. Modify the file to include (or exclude in case Funtoo add additional) packages of your choice. They will be included in your custom stage3 portage's world file.<br />
<br />
= Building Gentoo stages = <!--T:98--><br />
<br />
<!--T:99--><br />
Metro can also build Gentoo stages. After switching to Funtoo profile, see http://www.funtoo.org/Funtoo_Profiles metro require additional steps for this. We have an open bug for this -- it is simply due to the fact that we focus on ensuring Funtoo Linux builds and building Gentoo is a lower priority. Historical note: Funtoo Linux originally started as a fork of Gentoo Linux so that metro could reliably build Gentoo stages.<br />
http://www.funtoo.org/Funtoo_Profiles<br />
<br />
= Advanced Features = <!--T:100--><br />
<br />
<!--T:101--><br />
Metro also includes a number of advanced features that can be used to automate builds and set up distributed build servers. These features require you to {{c|emerge sqlalchemy}}, as SQLite is used as a dependency and also {{c|emerge dev-python/lxml}} as this is needed for index file generation.<br />
<br />
== Repository Management == <!--T:102--><br />
<br />
<!--T:103--><br />
Metro includes a script in the {{c|scripts}} directory called {{c|buildrepo}}. Buildrepo serves as the heart of Metro's advanced repository management features.<br />
<br />
=== Initial Setup === <!--T:104--><br />
<br />
<!--T:105--><br />
To use {{c|buildrepo}}, you will first need to create a {{f|.buildbot}} configuration file. Here is the file I use on my AMD Jaguar build server:<br />
<br />
<!--T:106--><br />
{{file|name=/root/.buildbot|lang=python|body=<br />
builds = (<br />
"funtoo-current",<br />
"funtoo-current-hardened",<br />
)<br />
<br />
<!--T:107--><br />
arches = (<br />
"x86-64bit",<br />
"pure64"<br />
)<br />
<br />
<!--T:108--><br />
subarches = (<br />
"amd64-jaguar",<br />
"amd64-jaguar-pure64",<br />
)<br />
<br />
<!--T:109--><br />
def map_build(build, subarch, full, full_date):<br />
# arguments refer to last build...<br />
if full == True:<br />
buildtype = ( "freshen", )<br />
else:<br />
buildtype = ("full", )<br />
# return value can be a string like "full+openvz" or a sequence type like [ "freshen", "openvz" ]<br />
return buildtype<br />
}}<br />
<br />
<!--T:110--><br />
This file is actually a python source file that defines the tuples {{c|builds}}, {{c|arches}} and {{c|subarches}}. These variables tell {{c|buildrepo}} which builds, arches and subarches it should manage. A {{c|map_build()}} function is also defined which {{c|buildbot}} uses to determine what kind of build to perform. The arguments passed to the function are based on the last successful build. The function can read these arguments and return a string to define the type of the next build. In the above example, the {{c|map_build()}} function will cause the next build after a freshen build to be a full build, and the next build after a full build to be a freshen build, so that the build will alternate between full and freshen.<br />
<br />
== Automated Builds == <!--T:111--><br />
<br />
<!--T:112--><br />
Once the {{c|.buildbot}} file has been created, the {{c|buildrepo}} and {{c|buildbot.sh}} tools are ready to use. Here's how they work. These tools are designed to keep your repository ({{c|path/mirror}} in {{f|/root/.metro}} up-to-date by inspecting your repository and looking for stages that are out-of-date. <br />
<br />
<!--T:113--><br />
To list the next build that will be performed, do this -- this is from my ARM build server:<br />
<br />
<!--T:114--><br />
{{console|body=<br />
# ##i##./buildrepo nextbuild<br />
build=funtoo-current<br />
arch_desc=arm-32bit<br />
subarch=armv7a_hardfp<br />
fulldate=2015-02-08<br />
nextdate=2015-02-20<br />
failcount=0<br />
target=full<br />
extras=''<br />
}}<br />
<br />
<!--T:115--><br />
If no output is displayed, then all your builds are up-to-date.<br />
<br />
<!--T:116--><br />
To actually run the next build, run {{c|buildbot.sh}}:<br />
<br />
<!--T:117--><br />
{{console|body=<br />
# ##i##./buildbot.sh<br />
}}<br />
<br />
<!--T:118--><br />
If you're thinking that {{c|buildbot.sh}} would be a good candidate for a cron job, you've got the right idea!<br />
<br />
=== List Builds === <!--T:119--><br />
<br />
<!--T:120--><br />
To get a quick look at our repository, let's run the {{c|buildrepo fails}} command:<br />
<br />
<!--T:121--><br />
{{console|body=<br />
# ##i##./buildrepo fails<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current/x86-64bit/amd64-jaguar<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current/pure64/amd64-jaguar-pure64<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current-hardened/x86-64bit/amd64-jaguar<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current-hardened/pure64/amd64-jaguar-pure64<br />
}}<br />
<br />
<!--T:122--><br />
On my AMD Jaguar build server, on Feb 20, 2015, this lists all the builds that {{c|buildrepo}} has been configured to manage. The first number on each line is a '''failcount''', which is the number of consecutive times that the build has failed. A zero value indicates that everything's okay. The failcount is an important feature of the advanced repository management features. Here are a number of behaviors that are implemented based on failcount:<br />
<br />
<!--T:123--><br />
* If {{c|buildbot.sh}} tries to build a stage and the build fails, the failcount is incremented.<br />
* If the build succeeds for a particular build, the failcount is reset to zero.<br />
* Builds with the lowest failcount are prioritized by {{buildrepo}} to build next, to steer towards builds that are more likely to complete successfully.<br />
* Once the failcount reaches 3 for a particular build, it is removed from the build rotation.<br />
<br />
=== Resetting Failcount === <!--T:124--><br />
<br />
<!--T:125--><br />
If a build has issues, the failcount for a build will reach 3, at which point it will be pulled out of build rotation. To clear failcount, so that these builds are attempted again -- possibly fixed by new updates to the Portage tree -- use {{c|buildrepo zap}}:<br />
<br />
<!--T:126--><br />
{{console|body=<br />
# /root/metro/scripts/buildrepo zap<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv7a_hardfp/.control/.failcount...<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv6j_hardfp/.control/.failcount...<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv5te/.control/.failcount...<br />
}}<br />
<br />
== Repository Maintenance == <!--T:127--><br />
<br />
<!--T:128--><br />
A couple of repository maintenance tools are provided:<br />
<br />
<!--T:129--><br />
* {{c|buildrepo digestgen}} will generate hash files for the archives in your repository, and clean up stale hashes. <br />
* {{c|buildrepo index.xml}} will create an index.xml file at the root of your repository, listing all builds available.<br />
* {{c|buildrepo clean}} will output a shell script that will remove old stages. No more than the three most recent stage builds for each build/arch/subarch are kept.<br />
<br />
== Distributed Repositories == <!--T:130--><br />
<br />
<!--T:131--><br />
In many situation, you will have a number of build servers, and each will build a subset of your master repository, and then upload builds to the master repository. This is an area of Metro that is being actively developed. For now, automated upload functionality is not enabled, but is expected to be implemented in the relatively near future. However, it is possible to have your master repository differentiate between subarches that are built locally, and thus should be part of that system's {{c|buildbot}} build rotation, and those that are stored locally and built remotely. These builds should be cleaned when {{c|buildrepo clean}} is run, but should not enter the local build rotation. To set this up, modify {{f|/root/.buildbot}} and use the {{c|subarches}} and {{c|all_subarches}} variables:<br />
<br />
<!--T:132--><br />
{{file|name=/root/.buildbot|desc=Excerpt of .buildbot config for master repository|body=<br />
# subarches we are building locally:<br />
<br />
<!--T:133--><br />
subarches = ( <br />
"pentium4",<br />
"athlon-xp",<br />
"corei7",<br />
"corei7-pure64",<br />
"generic_32", <br />
"i686", <br />
"amd64-k8",<br />
"amd64-k8-pure64",<br />
"core2_64",<br />
"core2_64-pure64",<br />
"generic_64",<br />
"generic_64-pure64",<br />
) <br />
<br />
# Things we need to clean, even if we may not be building:<br />
<br />
all_subarches = subarches + (<br />
"atom_32",<br />
"atom_64",<br />
"atom_64-pure64",<br />
"amd64-k10",<br />
"amd64-k10-pure64",<br />
"amd64-bulldozer",<br />
"amd64-bulldozer-pure64",<br />
"amd64-steamroller",<br />
"amd64-steamroller-pure64",<br />
"amd64-piledriver",<br />
"amd64-piledriver-pure64",<br />
"amd64-jaguar",<br />
"amd64-jaguar-pure64",<br />
"intel64-haswell",<br />
"intel64-haswell-pure64",<br />
"intel64-ivybridge-pure64",<br />
"intel64-ivybridge",<br />
"armv7a_hardfp",<br />
"armv6j_hardfp",<br />
"armv5te"<br />
) <br />
}}<br />
== Using binary cache ==<br />
Metro has built-in feature which allows to use binary packages cache rather then building same list of packages from sources. For example, core packages, such as @system are updated at slower pace and it makes sense to enable binary cache to make stage building blazing fast. However, the real disadvantage with using binary cache could be a core package update that due to internal ABI changes require rebuilding of numerous packages from sources. Good example is {{c|sys-libs/ncurses-5}} to {{c|sys-libs/ncurses-6}} major update. This is the case when you would need to disable binary cache and use regular ebuild installation from sources. To enable binary cache, in your metro git repository copy, edit the {{c|common.conf}} <br />
{{file|name=/etc/builds/common.conf|desc=Excerpt of default common.conf|body=<br />
[section metro]<br />
<br />
options:<br />
options/stage:<br />
target: gentoo<br />
}}<br />
and set {{c|cache/package}}<br />
{{file|name=/etc/builds/common.conf|desc=Excerpt of common.conf with binary cache enabled|body=<br />
[section metro]<br />
<br />
options:<br />
options/stage: cache/package<br />
target: gentoo<br />
}}<br />
<br />
During stage build metro will save package cache in {{c|/var/tmp/metro/cache/package-cache}}. With any next builds this binary package cache will be used.<br />
<!--T:134--><br />
[[Category:HOWTO]]<br />
[[Category:Metro]]<br />
__TOC__<br />
</translate><br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Funtoo:Metro&diff=22392Funtoo:Metro2018-08-27T19:04:08Z<p>Oleg: </p>
<hr />
<div><languages/><br />
<translate><br />
<!--T:1--><br />
{{#layout:doc}}{{#widget:AddThis}}Metro is the build system for Funtoo Linux and [[Gentoo Linux]] stages. It automates the bootstrapping process.<br />
<br />
<!--T:2--><br />
This tutorial will take you through installing, setting up and running Metro.<br />
<br />
<!--T:3--><br />
Other [[:Category:Metro|Metro Documentation]] is available.<br />
<br />
= Preface = <!--T:4--> <br />
<br />
{{Note|If you are unfamiliar with how metro works it is recommended that you continue reading this section so you can become familiar with it. If you are already familiar with it, you may wish to use the new [[Metro AutoSetup|autosetup]] script which uses a curses based menu and allows for quickly setting up and running builds base on your choices without requiring any manual steps. Please see the [[Metro AutoSetup]] page for more details}}<br />
<br />
== How Metro Works == <!--T:5--> <br />
<br />
<!--T:6--><br />
Metro is the Funtoo Linux automated build system, and is used to build Funtoo Linux stage tarballs.<br />
<br />
<!--T:7--><br />
Metro cannot create a stage tarball out of thin air. To build a new stage tarball, Metro must use an existing, older stage tarball called a "seed" stage. This seed stage typically is used as the ''build environment'' for creating the stage we want.<br />
<br />
<!--T:8--><br />
Metro can use two kinds of seed stages. Traditionally, Metro has used a stage3 as a seed stage. This stage3 is then used to build a new stage1, which in turn is used to build a new stage2, and then a new stage3. This is generally the most reliable way to build [[Gentoo Linux]] or Funtoo Linux, so it's the recommended approach.<br />
<br />
== Seeds and Build Isolation == <!--T:9--><br />
<br />
<!--T:10--><br />
Another important concept to mention here is something called ''build isolation''. Because Metro creates an isolated build environment, and the build environment is explicitly defined using existing, tangible entities -- a seed stage and a portage snapshot -- you will get consistent, repeatable results. In other words, the same seed stage, portage snapshot and build instructions will generate an essentially identical result, even if you perform the build a month later on someone else's workstation.<br />
<br />
== Local Build == <!--T:11--> <br />
<br />
<!--T:12--><br />
Say you wanted to build a new <tt>pentium4</tt> stage3 tarball. The recommended method of doing this would be to grab an existing <tt>pentium4</tt> stage3 tarball to use as your seed stage. Metro will be told to use this existing <tt>pentium4</tt> stage3 to build a new stage1 for the same <tt>pentium4</tt>. For this process, the generic <tt>pentium4</tt> stage3 would provide the ''build environment'' for creating our new stage1. Then, the new stage1 would serve as the build environment for creating the new <tt>pentium4</tt> stage2. And the new <tt>pentium4</tt> stage2 would serve as the build environment for creating the new <tt>pentium4</tt> stage3.<br />
<br />
<!--T:13--><br />
In the Metro terminology this is called a '''local build''', which means a stage3 of a given architecture is used to seed a brand new build of the same architecture. Incidentally this will be the first exercise we are going to perform in this tutorial.<br />
<br />
<!--T:14--><br />
A week later, you may want to build a brand new <tt>pentium4</tt> stage3 tarball. Rather than starting from the original <tt>pentium4</tt> stage3 again, you'd probably configure Metro to use the most-recently-built <tt>pentium4</tt> stage3 as the seed. Metro has built-in functionality to make this easy, allowing it to easily find and track the most recent stage3 seed available.<br />
<br />
== Remote Build == <!--T:15--> <br />
<br />
<!--T:16--><br />
Metro can also perform '''remote build''', where a stage3 of a different, but binary compatible, architecture is used as a seed to build a different architecture stage3. Consequentiality the second exercise we are going to perform in this tutorial will be to build a <tt>core2 32bit</tt> stage3 tarball from the <tt>pentium4</tt> stage3 tarball we have just built.<br />
<br />
<!--T:17--><br />
TODO: add caveats about what archs can be seeded and what can be not (maybe a table?)<br />
<br />
== Tailored Build == <!--T:18--> <br />
<br />
<!--T:19--><br />
Last, it's also worthy noting that both in <tt>local</tt> and <tt>remote builds</tt>, Metro can be configured to add and/or remove individual packages to the final tarball.<br />
Let's say you can't live without <tt>app-misc/screen</tt>, at the end of this tutorial, we will show how to have your tailored stage3 to include it.<br />
<br />
== Installing Metro == <!--T:20--><br />
<br />
<!--T:21--><br />
'''The recommended and supported method''' is to use the Git repository of Metro. <br />
<br />
<!--T:22--><br />
Ensure that {{Package|dev-vcs/git}}, {{Package|dev-python/requests}} and {{Package|dev-python/boto}} (optional; required for EC2 support) are installed on your system. <br />
<br />
<!--T:23--><br />
{{console|body=<br />
# ##i##emerge dev-vcs/git dev-python/requests dev-python/boto <br />
}}<br />
<br />
<!--T:24--><br />
Next, clone the master git repository as follows:<br />
<br />
<!--T:25--><br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone git://github.com/funtoo/metro.git<br />
# ##i##cp /root/metro/metro.conf ~/.metro<br />
}}<br />
<br />
<!--T:26--><br />
You will now have a directory called {{c|/root/metro}} that contains all the Metro source code.<br />
<br />
<!--T:27--><br />
=== Setting up ego===<br />
Now, we will set the {{c|ego}}, administration tool of Funtoo/Linux. The way it is used with metro is independent from {{c|app-admin/ego}} installed on your box. Setup is easy as follows:<br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone https://github.com/funtoo/ego,git<br />
}}<br />
This way you will have {{c|/root/ego}} directory with {{c|ego}} binary that is then used by metro.<br />
<br />
<!--T:28--><br />
Metro is now installed. It's time to customize it for your local system.<br />
<br />
= Configuring Metro = <!--T:28--><br />
<br />
<!--T:29--><br />
{{Note|Metro is not currently able to build Gentoo stages. See {{Bug|FL-901}}.}}<br />
<br />
<!--T:30--><br />
[[User:Drobbins|Daniel Robbins]] maintains Metro, so it comes pre-configured to successfully build Funtoo Linux releases. Before reading further, you might want to customize some basic settings like the number of concurrent jobs to fit your hardware's capabilities or the directory to use for produced stage archives. This is accomplished by editing {{c|~/.metro}} which is the Metro's master configuration file.<br />
<br />
<!--T:31--><br />
Please note that {{c|path/install}} must point to where metro was installed. Point {{c|path/distfiles}} to where your distfiles reside. Also set {{c|path/mirror/owner}} and {{c|path/mirror/group}} to the owner and group of all the files that will be written to the build repository directory, which by default (as per the configuration file) is at {{c|/home/mirror/funtoo}}. The cache directory normally resides inside the temp directory -- this can be modified as desired. The cache directory can end up holding many cached .tbz2 packages, and eat up a lot of storage. You may want to place the temp directory on faster storage, for faster compile times, and place the cache directory on slower, but more plentiful storage.<br />
<br />
<!--T:32--><br />
{{file|name=.metro|desc=Metro configuration|body=<br />
# Main metro configuration file - these settings need to be tailored to your install:<br />
<br />
<!--T:33--><br />
[section path]<br />
install: /root/metro<br />
tmp: /var/tmp/metro<br />
cache: $[path/tmp]/cache<br />
distfiles: /var/src/distfiles<br />
work: $[path/tmp]/work/$[target/build]/$[target/name]<br />
<br />
<!--T:34--><br />
[section path/mirror]<br />
<br />
<!--T:35--><br />
: /home/mirror/funtoo<br />
owner: root<br />
group: repomgr<br />
dirmode: 775<br />
<br />
<!--T:36--><br />
[section portage]<br />
<br />
<!--T:37--><br />
MAKEOPTS: auto <br />
<br />
<!--T:38--><br />
[section emerge]<br />
<br />
<!--T:39--><br />
options: --jobs=4 --load-average=4 --keep-going=n<br />
<br />
<!--T:40--><br />
# This line should not be modified:<br />
[collect $[path/install]/etc/master.conf]<br />
}}<br />
<br />
== Arch and Subarch == <!--T:41--><br />
<br />
<!--T:42--><br />
In the following example we are creating a pentium4 stage 3 compiled for x86-32bit binary compatibility. Pentium4 is a subarch of the x86-32bit architecture. Once you have metro installed you may find a full list of each subarch in your {{f|/var/git/meta-repo/kits/core-kit/profiles/funtoo-1.0/linux-gnu/arch/x86-32bit/subarch}} directory:<br />
Example:<br />
{{console|body=<br />
# ##i##ls /var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-32bit/subarch/<br />
amd64-k8+sse3_32 athlon-4 athlon-xp core2_32 i486 k6-2 pentium pentium2 pentiumpro<br />
amd64-k8_32 athlon-mp atom_32 generic_32 i686 k6-3 pentium-m pentium3 prescott<br />
athlon athlon-tbird btver1 geode k6 native_32 pentium-mmx pentium4 xen-pentium4+sse3<br />
}}<br />
<br />
64-bit PC profiles can be found in the {{f|/var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-64bit/subarch/}} directory:<br />
{{console|body=<br />
# ##i##ls /var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-64bit/subarch/<br />
amd64-bulldozer amd64-k8+sse3 btver1_64 generic_64 intel64-nehalem native_64<br />
amd64-jaguar amd64-piledriver core-avx-i intel64-broadwell intel64-sandybridge nocona<br />
amd64-k10 amd64-steamroller core2_64 intel64-haswell intel64-silvermont opteron_64<br />
amd64-k8 atom_64 corei7 intel64-ivybridge intel64-westmere xen-pentium4+sse3_64<br />
}}<br />
<br />
= First stages build (local build) = <!--T:43--><br />
<br />
<!--T:44--><br />
To get this all started, we need to bootstrap the process by downloading an initial seed stage3 to use for building and place it in its proper location in {{f|/home/mirror/funtoo}}, so that Metro can find it. We will also need to create some special &quot;control&quot; files in {{f|/home/mirror/funtoo}}, which will allow Metro to understand how it is supposed to proceed.<br />
<br />
== Step 1: Set up pentium4 repository (local build) == <!--T:45--><br />
<br />
<!--T:46--><br />
Assuming we're following the basic steps outlined in the previous section, and building {{f|funtoo-current}} build for the {{f|pentium4}}, using a generic {{f|pentium4}} stage3 as a seed stage, then here the first set of steps we'd perform:<br />
<br />
<!--T:47--><br />
{{console|body=<br />
# ##i##install -d /home/mirror/funtoo/funtoo-current/x86-32bit/pentium4<br />
# ##i##install -d /home/mirror/funtoo/funtoo-current/snapshots<br />
# ##i##cd /home/mirror/funtoo/funtoo-current/x86-32bit/pentium4<br />
# ##i##install -d 2017-10-01<br />
# ##i##cd 2017-10-01<br />
# ##i##wget -c https://build.funtoo.org/funtoo-current/x86-32bit/pentium4/2017-10-01/stage3-pentium4-funtoo-current-2017-10-01.tar.xz<br />
# ##i##cd ..<br />
# ##i##install -d .control/version<br />
# ##i##echo "2017-10-01" > .control/version/stage3<br />
# ##i##install -d .control/strategy<br />
# ##i##echo local > .control/strategy/build<br />
# ##i##echo stage3 > .control/strategy/seed<br />
}}<br />
<br />
<!--T:48--><br />
OK, let's review the steps above. First, we create the directory {{f|/home/mirror/funtoo/funtoo-current/x86-32bit/pentium4}}, which is where Metro will expect to find {{f|funtoo-current}} pentium4 builds -- it is configured to look here by default. Then we create a specially-named directory to house our seed x86 stage3. Again, by default, Metro expects the directory to be named this way. We enter this directory, and download our seed x86 stage3 from funtoo.org. Note that the {{f|2017-10-01}} version stamp matches. Make sure that your directory name matches the stage3 name too. Everything has been set up to match Metro's default filesystem layout.<br />
<br />
<!--T:49--><br />
Next, we go back to the {{f|/home/mirror/metro/funtoo-current/x86-32bit/pentium4}} directory, and inside it, we create a {{f|.control}} directory. This directory and its subdirectories contain special files that Metro references to determine certain aspects of its behavior. The {{f|.control/version/stage3}} file is used by Metro to track the most recently-built stage3 for this particular build and subarch. Metro will automatically update this file with a new version stamp after it successfully builds a new stage3. But because Metro didn't actually ''build'' this stage3, we need to set up the {{f|.control/version/stage3}} file manually. This will allow Metro to find our downloaded stage3 when we set up our pentium4 build to use it as a seed. Also note that Metro will create a similar {{f|.control/version/stage1}} file after it successfully builds an pentium4 funtoo-current stage1.<br />
<br />
<!--T:50--><br />
We also set up {{f|.control/strategy/build}} and {{f|.control/strategy/seed}} files with values of {{f|local}} and {{f|stage3}} respectively. These files define the building strategy Metro will use when we build pentium4 funtoo-current stages. With a build strategy of {{f|local}}, Metro will source its seed stage from funtoo-current pentium4, the current directory. And with a seed strategy of {{f|stage3}}, Metro will use a stage3 as a seed, and use this seed to build a new stage1, stage2 and stage3.<br />
<br />
== Step 2: Building the pentium4 stages == <!--T:51--><br />
<br />
<!--T:52--><br />
Incidentally, if all you wanted to do at this point was to build a new pentium4 funtoo-current stage1/2/3 (plus openvz and vserver templates). You would begin the process by typing:<br />
<br />
<!--T:53--><br />
<console><br />
# ##i##cd /root/metro<br />
# ##i##scripts/ezbuild.sh funtoo-current x86-32bit pentium4<br />
</console><br />
<br />
<!--T:54--><br />
If you have a slow machine, it could take several hours to be completed because several "heavy" components like gcc or glibc have to be recompiled in each stage. Once a stage has been successfully completed, it is placed in the {{f|"${METRO_MIRROR}/funtoo-current/x32-bit/pentium4/YYYY-MM-DD"}} subdirectory, where {{f|YYYY-MM-DD}} is today's date at the time the {{f|ezbuild.sh}} script was started or the date you put on the ezscript.sh command line.<br />
<br />
= Building for another binary compatible architecture (remote build) = <!--T:55--><br />
<br />
<!--T:56--><br />
As written above, Metro is able to perform '''remote build''' building different architecture stage3 from a binary compatible seeding stage3 (e.g. using a pentium4 stage3 to seed a <tt>Intel Core2 32bits</tt> stage3). <br />
<br />
<!--T:57--><br />
In the Metro terminology this is called a '''remote build''' (a stage 3 of a different, but binary compatible, architecture is used as a seed). <br />
What's not compatible? You can't use a <tt>Sparc</tt> architecture to generate an <tt>x86</tt> or <tt>ARM</tt> based stage and vice-versa. If you use a 32bit stage then you don't want to seed a 64bit build from it. Be sure that you are using a stage from the same architecture that you are trying to seed. Check [http://ftp.osuosl.org/pub/funtoo/funtoo-current/ Funtoo-current FTP Mirror] for a stage that is from the same Architecture that you will be building. <br />
<br />
<!--T:58--><br />
{{Note|Often, one build (ie. funtoo-current) can be used as a seed for another build such as funtoo-stable. However, hardened builds require hardened stages as seeds in order for the build to complete successfully.}}<br />
<br />
== Step 1: Set up Core_2 32bit repository == <!--T:59--><br />
<br />
<!--T:60--><br />
In this example, we're going to use this pentium4 funtoo-current stage3 to seed a new Core_2 32bit funtoo-current build. To get that done, we need to set up the pentium4 build directory as follows:<br />
<br />
<!--T:61--><br />
<console><br />
# ##i## cd /home/mirror/funtoo/funtoo-current/x86-32bit<br />
# ##i##install -d core2_32<br />
# ##i##cd core2_32<br />
# ##i##install -d .control/strategy<br />
# ##i##echo remote > .control/strategy/build<br />
# ##i##echo stage3 > .control/strategy/seed<br />
# ##i##install -d .control/remote<br />
# ##i##echo funtoo-current > .control/remote/build<br />
# ##i##echo x86-32bit > .control/remote/arch_desc<br />
# ##i##echo pentium4 > .control/remote/subarch<br />
</console><br />
<br />
<!--T:62--><br />
The steps we follow are similar to those we performed for a ''local build'' to set up our pentium4 directory for local build. However, note the differences. We didn't download a stage, because we are going to use the pentium4 stage to build a new Core_2 32bit stage. We also didn't create the <tt>.control/version/stage{1,3}</tt> files because Metro will create them for us after it successfully builds a new stage1 and stage3. We are still using a <tt>stage3</tt> seed strategy, but we've set the build strategy to <tt>remote</tt>, which means that we're going to use a seed stage that's not from this particular subdirectory. Where are we going to get it from? The <tt>.control/remote</tt> directory contains this information, and lets Metro know that it should look for its seed stage3 in the <tt>/home/mirror/funtoo/funtoo-current/x86-32bit/pentium4</tt> directory. Which one will it grab? You guessed it -- the most recently built ''stage3'' (since our seed strategy was set to <tt>stage3</tt>) that has the version stamp of <tt>2010-12-24</tt>, as recorded in <tt>/home/mirror/funtoo-current/x86-32bit/pentium4/.control/version/stage</tt>. Now you can see how all those control files come together to direct Metro to do the right thing.<br />
<br />
<!--T:63--><br />
{{Note|<code>arch_desc</code> should be set to one of: <code>x86-32bit</code>, <code>x86-64bit</code> or <code>pure64</code> for PC-compatible systems. You must use a 32-bit build as a seed for other 32-bit builds, and a 64-bit build as a seed for other 64-bit builds.}}<br />
<br />
== Step 2: Building the Core_2 32bit stages == <!--T:64--><br />
<br />
<!--T:65--><br />
Now, you could start building your new Core_2 32bit stage1/2/3 (plus openvz and vserver templates) by typing the following:<br />
<br />
<!--T:66--><br />
<console><br />
# ##i##/root/metro/scripts/ezbuild.sh funtoo-current x86-32bit core2_32<br />
</console><br />
<br />
<!--T:67--><br />
In that case, the produced stages are placed in the <tt>/home/mirror/funtoo/funtoo-current/x32-bit/core2_32/YYYY-MM-DD</tt> subdirectory.<br />
<br />
== Step 3: The Next Build == <!--T:68--><br />
<br />
<!--T:69--><br />
At this point, you now have a new Core_2 32bit stage3, built using a "remote" pentium4 stage3. Once the first remote build completes successfully, metro will automatically change {{c|.control/strategy/build}} to be {{c|local}} instead of {{c|remote}}, so it will use the most recently-built Core_2 32bit stage3 as a seed for any new Core_2 32bit builds from now on.<br />
<br />
= Build your own tailored stage3 = <!--T:70--><br />
<br />
<!--T:71--><br />
Metro can be easily configured for building custom stage3 by including additional packages. You can find following directory {{c|/etc/builds/packages}} in your copy of metro repository and a corresponding {{c|arch}} configuration files inside:<br />
{{file|name=/etc/builds/packages/x86-64bit.conf|body=<br />
[section emerge]<br />
<br />
packages: [<br />
sys-kernel/debian-sources<br />
]<br />
}}<br />
Notice a {{c|debian-sources}} ebuild is added for all 64-bit stages. Modify the file to include (or exclude in case Funtoo add additional) packages of your choice. They will be included in your custom stage3 portage's world file.<br />
<br />
= Building Gentoo stages = <!--T:98--><br />
<br />
<!--T:99--><br />
Metro can also build Gentoo stages. After switching to Funtoo profile, see http://www.funtoo.org/Funtoo_Profiles metro require additional steps for this. We have an open bug for this -- it is simply due to the fact that we focus on ensuring Funtoo Linux builds and building Gentoo is a lower priority. Historical note: Funtoo Linux originally started as a fork of Gentoo Linux so that metro could reliably build Gentoo stages.<br />
http://www.funtoo.org/Funtoo_Profiles<br />
<br />
= Advanced Features = <!--T:100--><br />
<br />
<!--T:101--><br />
Metro also includes a number of advanced features that can be used to automate builds and set up distributed build servers. These features require you to {{c|emerge sqlalchemy}}, as SQLite is used as a dependency and also {{c|emerge dev-python/lxml}} as this is needed for index file generation.<br />
<br />
== Repository Management == <!--T:102--><br />
<br />
<!--T:103--><br />
Metro includes a script in the {{c|scripts}} directory called {{c|buildrepo}}. Buildrepo serves as the heart of Metro's advanced repository management features.<br />
<br />
=== Initial Setup === <!--T:104--><br />
<br />
<!--T:105--><br />
To use {{c|buildrepo}}, you will first need to create a {{f|.buildbot}} configuration file. Here is the file I use on my AMD Jaguar build server:<br />
<br />
<!--T:106--><br />
{{file|name=/root/.buildbot|lang=python|body=<br />
builds = (<br />
"funtoo-current",<br />
"funtoo-current-hardened",<br />
)<br />
<br />
<!--T:107--><br />
arches = (<br />
"x86-64bit",<br />
"pure64"<br />
)<br />
<br />
<!--T:108--><br />
subarches = (<br />
"amd64-jaguar",<br />
"amd64-jaguar-pure64",<br />
)<br />
<br />
<!--T:109--><br />
def map_build(build, subarch, full, full_date):<br />
# arguments refer to last build...<br />
if full == True:<br />
buildtype = ( "freshen", )<br />
else:<br />
buildtype = ("full", )<br />
# return value can be a string like "full+openvz" or a sequence type like [ "freshen", "openvz" ]<br />
return buildtype<br />
}}<br />
<br />
<!--T:110--><br />
This file is actually a python source file that defines the tuples {{c|builds}}, {{c|arches}} and {{c|subarches}}. These variables tell {{c|buildrepo}} which builds, arches and subarches it should manage. A {{c|map_build()}} function is also defined which {{c|buildbot}} uses to determine what kind of build to perform. The arguments passed to the function are based on the last successful build. The function can read these arguments and return a string to define the type of the next build. In the above example, the {{c|map_build()}} function will cause the next build after a freshen build to be a full build, and the next build after a full build to be a freshen build, so that the build will alternate between full and freshen.<br />
<br />
== Automated Builds == <!--T:111--><br />
<br />
<!--T:112--><br />
Once the {{c|.buildbot}} file has been created, the {{c|buildrepo}} and {{c|buildbot.sh}} tools are ready to use. Here's how they work. These tools are designed to keep your repository ({{c|path/mirror}} in {{f|/root/.metro}} up-to-date by inspecting your repository and looking for stages that are out-of-date. <br />
<br />
<!--T:113--><br />
To list the next build that will be performed, do this -- this is from my ARM build server:<br />
<br />
<!--T:114--><br />
{{console|body=<br />
# ##i##./buildrepo nextbuild<br />
build=funtoo-current<br />
arch_desc=arm-32bit<br />
subarch=armv7a_hardfp<br />
fulldate=2015-02-08<br />
nextdate=2015-02-20<br />
failcount=0<br />
target=full<br />
extras=''<br />
}}<br />
<br />
<!--T:115--><br />
If no output is displayed, then all your builds are up-to-date.<br />
<br />
<!--T:116--><br />
To actually run the next build, run {{c|buildbot.sh}}:<br />
<br />
<!--T:117--><br />
{{console|body=<br />
# ##i##./buildbot.sh<br />
}}<br />
<br />
<!--T:118--><br />
If you're thinking that {{c|buildbot.sh}} would be a good candidate for a cron job, you've got the right idea!<br />
<br />
=== List Builds === <!--T:119--><br />
<br />
<!--T:120--><br />
To get a quick look at our repository, let's run the {{c|buildrepo fails}} command:<br />
<br />
<!--T:121--><br />
{{console|body=<br />
# ##i##./buildrepo fails<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current/x86-64bit/amd64-jaguar<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current/pure64/amd64-jaguar-pure64<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current-hardened/x86-64bit/amd64-jaguar<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current-hardened/pure64/amd64-jaguar-pure64<br />
}}<br />
<br />
<!--T:122--><br />
On my AMD Jaguar build server, on Feb 20, 2015, this lists all the builds that {{c|buildrepo}} has been configured to manage. The first number on each line is a '''failcount''', which is the number of consecutive times that the build has failed. A zero value indicates that everything's okay. The failcount is an important feature of the advanced repository management features. Here are a number of behaviors that are implemented based on failcount:<br />
<br />
<!--T:123--><br />
* If {{c|buildbot.sh}} tries to build a stage and the build fails, the failcount is incremented.<br />
* If the build succeeds for a particular build, the failcount is reset to zero.<br />
* Builds with the lowest failcount are prioritized by {{buildrepo}} to build next, to steer towards builds that are more likely to complete successfully.<br />
* Once the failcount reaches 3 for a particular build, it is removed from the build rotation.<br />
<br />
=== Resetting Failcount === <!--T:124--><br />
<br />
<!--T:125--><br />
If a build has issues, the failcount for a build will reach 3, at which point it will be pulled out of build rotation. To clear failcount, so that these builds are attempted again -- possibly fixed by new updates to the Portage tree -- use {{c|buildrepo zap}}:<br />
<br />
<!--T:126--><br />
{{console|body=<br />
# /root/metro/scripts/buildrepo zap<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv7a_hardfp/.control/.failcount...<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv6j_hardfp/.control/.failcount...<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv5te/.control/.failcount...<br />
}}<br />
<br />
== Repository Maintenance == <!--T:127--><br />
<br />
<!--T:128--><br />
A couple of repository maintenance tools are provided:<br />
<br />
<!--T:129--><br />
* {{c|buildrepo digestgen}} will generate hash files for the archives in your repository, and clean up stale hashes. <br />
* {{c|buildrepo index.xml}} will create an index.xml file at the root of your repository, listing all builds available.<br />
* {{c|buildrepo clean}} will output a shell script that will remove old stages. No more than the three most recent stage builds for each build/arch/subarch are kept.<br />
<br />
== Distributed Repositories == <!--T:130--><br />
<br />
<!--T:131--><br />
In many situation, you will have a number of build servers, and each will build a subset of your master repository, and then upload builds to the master repository. This is an area of Metro that is being actively developed. For now, automated upload functionality is not enabled, but is expected to be implemented in the relatively near future. However, it is possible to have your master repository differentiate between subarches that are built locally, and thus should be part of that system's {{c|buildbot}} build rotation, and those that are stored locally and built remotely. These builds should be cleaned when {{c|buildrepo clean}} is run, but should not enter the local build rotation. To set this up, modify {{f|/root/.buildbot}} and use the {{c|subarches}} and {{c|all_subarches}} variables:<br />
<br />
<!--T:132--><br />
{{file|name=/root/.buildbot|desc=Excerpt of .buildbot config for master repository|body=<br />
# subarches we are building locally:<br />
<br />
<!--T:133--><br />
subarches = ( <br />
"pentium4",<br />
"athlon-xp",<br />
"corei7",<br />
"corei7-pure64",<br />
"generic_32", <br />
"i686", <br />
"amd64-k8",<br />
"amd64-k8-pure64",<br />
"core2_64",<br />
"core2_64-pure64",<br />
"generic_64",<br />
"generic_64-pure64",<br />
) <br />
<br />
# Things we need to clean, even if we may not be building:<br />
<br />
all_subarches = subarches + (<br />
"atom_32",<br />
"atom_64",<br />
"atom_64-pure64",<br />
"amd64-k10",<br />
"amd64-k10-pure64",<br />
"amd64-bulldozer",<br />
"amd64-bulldozer-pure64",<br />
"amd64-steamroller",<br />
"amd64-steamroller-pure64",<br />
"amd64-piledriver",<br />
"amd64-piledriver-pure64",<br />
"amd64-jaguar",<br />
"amd64-jaguar-pure64",<br />
"intel64-haswell",<br />
"intel64-haswell-pure64",<br />
"intel64-ivybridge-pure64",<br />
"intel64-ivybridge",<br />
"armv7a_hardfp",<br />
"armv6j_hardfp",<br />
"armv5te"<br />
) <br />
}}<br />
== Using binary cache ==<br />
Metro has built-in feature which allows to use binary packages cache rather then building same list of packages from sources. For example, core packages, such as @system are updated at slower pace and it makes sense to enable binary cache to make stage building blazing fast. However, the real disadvantage with using binary cache could be a core package update that due to internal ABI changes require rebuilding of numerous packages from sources. Good example is {{c|sys-libs/ncurses-5}} to {{c|sys-libs/ncurses-6}} major update. This is the case when you would need to disable binary cache and use regular ebuild installation from sources. To enable binary cache, in your metro git repository copy, edit the {{c|common.conf}} <br />
{{file|name=/etc/builds/common.conf|desc=Excerpt of default common.conf|body=<br />
[section metro]<br />
<br />
options:<br />
options/stage:<br />
target: gentoo<br />
}}<br />
and set {{c|cache/package}}<br />
{{file|name=/etc/builds/common.conf|desc=Excerpt of common.conf with binary cache enabled|body=<br />
[section metro]<br />
<br />
options:<br />
options/stage: cache/package<br />
target: gentoo<br />
}}<br />
<br />
During stage build metro will save package cache in {{c|/var/tmp/metro/cache/package-cache}}. With any next builds this binary package cache will be used.<br />
<!--T:134--><br />
[[Category:HOWTO]]<br />
[[Category:Metro]]<br />
__TOC__<br />
</translate><br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Funtoo:Metro&diff=22391Funtoo:Metro2018-08-27T19:03:31Z<p>Oleg: </p>
<hr />
<div><languages/><br />
<translate><br />
<!--T:1--><br />
{{#layout:doc}}{{#widget:AddThis}}Metro is the build system for Funtoo Linux and [[Gentoo Linux]] stages. It automates the bootstrapping process.<br />
<br />
<!--T:2--><br />
This tutorial will take you through installing, setting up and running Metro.<br />
<br />
<!--T:3--><br />
Other [[:Category:Metro|Metro Documentation]] is available.<br />
<br />
= Preface = <!--T:4--> <br />
<br />
{{Note|If you are unfamiliar with how metro works it is recommended that you continue reading this section so you can become familiar with it. If you are already familiar with it, you may wish to use the new [[Metro AutoSetup|autosetup]] script which uses a curses based menu and allows for quickly setting up and running builds base on your choices without requiring any manual steps. Please see the [[Metro AutoSetup]] page for more details}}<br />
<br />
== How Metro Works == <!--T:5--> <br />
<br />
<!--T:6--><br />
Metro is the Funtoo Linux automated build system, and is used to build Funtoo Linux stage tarballs.<br />
<br />
<!--T:7--><br />
Metro cannot create a stage tarball out of thin air. To build a new stage tarball, Metro must use an existing, older stage tarball called a "seed" stage. This seed stage typically is used as the ''build environment'' for creating the stage we want.<br />
<br />
<!--T:8--><br />
Metro can use two kinds of seed stages. Traditionally, Metro has used a stage3 as a seed stage. This stage3 is then used to build a new stage1, which in turn is used to build a new stage2, and then a new stage3. This is generally the most reliable way to build [[Gentoo Linux]] or Funtoo Linux, so it's the recommended approach.<br />
<br />
== Seeds and Build Isolation == <!--T:9--><br />
<br />
<!--T:10--><br />
Another important concept to mention here is something called ''build isolation''. Because Metro creates an isolated build environment, and the build environment is explicitly defined using existing, tangible entities -- a seed stage and a portage snapshot -- you will get consistent, repeatable results. In other words, the same seed stage, portage snapshot and build instructions will generate an essentially identical result, even if you perform the build a month later on someone else's workstation.<br />
<br />
== Local Build == <!--T:11--> <br />
<br />
<!--T:12--><br />
Say you wanted to build a new <tt>pentium4</tt> stage3 tarball. The recommended method of doing this would be to grab an existing <tt>pentium4</tt> stage3 tarball to use as your seed stage. Metro will be told to use this existing <tt>pentium4</tt> stage3 to build a new stage1 for the same <tt>pentium4</tt>. For this process, the generic <tt>pentium4</tt> stage3 would provide the ''build environment'' for creating our new stage1. Then, the new stage1 would serve as the build environment for creating the new <tt>pentium4</tt> stage2. And the new <tt>pentium4</tt> stage2 would serve as the build environment for creating the new <tt>pentium4</tt> stage3.<br />
<br />
<!--T:13--><br />
In the Metro terminology this is called a '''local build''', which means a stage3 of a given architecture is used to seed a brand new build of the same architecture. Incidentally this will be the first exercise we are going to perform in this tutorial.<br />
<br />
<!--T:14--><br />
A week later, you may want to build a brand new <tt>pentium4</tt> stage3 tarball. Rather than starting from the original <tt>pentium4</tt> stage3 again, you'd probably configure Metro to use the most-recently-built <tt>pentium4</tt> stage3 as the seed. Metro has built-in functionality to make this easy, allowing it to easily find and track the most recent stage3 seed available.<br />
<br />
== Remote Build == <!--T:15--> <br />
<br />
<!--T:16--><br />
Metro can also perform '''remote build''', where a stage3 of a different, but binary compatible, architecture is used as a seed to build a different architecture stage3. Consequentiality the second exercise we are going to perform in this tutorial will be to build a <tt>core2 32bit</tt> stage3 tarball from the <tt>pentium4</tt> stage3 tarball we have just built.<br />
<br />
<!--T:17--><br />
TODO: add caveats about what archs can be seeded and what can be not (maybe a table?)<br />
<br />
== Tailored Build == <!--T:18--> <br />
<br />
<!--T:19--><br />
Last, it's also worthy noting that both in <tt>local</tt> and <tt>remote builds</tt>, Metro can be configured to add and/or remove individual packages to the final tarball.<br />
Let's say you can't live without <tt>app-misc/screen</tt>, at the end of this tutorial, we will show how to have your tailored stage3 to include it.<br />
<br />
== Installing Metro == <!--T:20--><br />
<br />
<!--T:21--><br />
'''The recommended and supported method''' is to use the Git repository of Metro. <br />
<br />
<!--T:22--><br />
Ensure that {{Package|dev-vcs/git}}, {{Package|dev-python/requests}} and {{Package|dev-python/boto}} (optional; required for EC2 support) are installed on your system. <br />
<br />
<!--T:23--><br />
{{console|body=<br />
# ##i##emerge dev-vcs/git dev-python/requests dev-python/boto <br />
}}<br />
<br />
<!--T:24--><br />
Next, clone the master git repository as follows:<br />
<br />
<!--T:25--><br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone git://github.com/funtoo/metro.git<br />
# ##i##cp /root/metro/metro.conf ~/.metro<br />
}}<br />
<br />
<!--T:26--><br />
You will now have a directory called {{c|/root/metro}} that contains all the Metro source code.<br />
<br />
<!--T:27--><br />
=== Setting up ego===<br />
Now, we will set the {{c|ego}}, administration tool of Funtoo/Linux. The way it is used with metro is independent from {{c|app-admin/ego}} installed on your box. Setup is easy as follows:<br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone https://github.com/funtoo/ego,git<br />
}}<br />
This way you will have {{c|/root/ego}} directory with {{c|ego}} binary that is then used by metro.<br />
<br />
<!--T:28--><br />
Metro is now installed. It's time to customize it for your local system.<br />
<br />
= Configuring Metro = <!--T:28--><br />
<br />
<!--T:29--><br />
{{Note|Metro is not currently able to build Gentoo stages. See {{Bug|FL-901}}.}}<br />
<br />
<!--T:30--><br />
[[User:Drobbins|Daniel Robbins]] maintains Metro, so it comes pre-configured to successfully build Funtoo Linux releases. Before reading further, you might want to customize some basic settings like the number of concurrent jobs to fit your hardware's capabilities or the directory to use for produced stage archives. This is accomplished by editing {{c|~/.metro}} which is the Metro's master configuration file.<br />
<br />
<!--T:31--><br />
Please note that {{c|path/install}} must point to where metro was installed. Point {{c|path/distfiles}} to where your distfiles reside. Also set {{c|path/mirror/owner}} and {{c|path/mirror/group}} to the owner and group of all the files that will be written to the build repository directory, which by default (as per the configuration file) is at {{c|/home/mirror/funtoo}}. The cache directory normally resides inside the temp directory -- this can be modified as desired. The cache directory can end up holding many cached .tbz2 packages, and eat up a lot of storage. You may want to place the temp directory on faster storage, for faster compile times, and place the cache directory on slower, but more plentiful storage.<br />
<br />
<!--T:32--><br />
{{file|name=.metro|desc=Metro configuration|body=<br />
# Main metro configuration file - these settings need to be tailored to your install:<br />
<br />
<!--T:33--><br />
[section path]<br />
install: /root/metro<br />
tmp: /var/tmp/metro<br />
cache: $[path/tmp]/cache<br />
distfiles: /var/src/distfiles<br />
work: $[path/tmp]/work/$[target/build]/$[target/name]<br />
<br />
<!--T:34--><br />
[section path/mirror]<br />
<br />
<!--T:35--><br />
: /home/mirror/funtoo<br />
owner: root<br />
group: repomgr<br />
dirmode: 775<br />
<br />
<!--T:36--><br />
[section portage]<br />
<br />
<!--T:37--><br />
MAKEOPTS: auto <br />
<br />
<!--T:38--><br />
[section emerge]<br />
<br />
<!--T:39--><br />
options: --jobs=4 --load-average=4 --keep-going=n<br />
<br />
<!--T:40--><br />
# This line should not be modified:<br />
[collect $[path/install]/etc/master.conf]<br />
}}<br />
<br />
== Arch and Subarch == <!--T:41--><br />
<br />
<!--T:42--><br />
In the following example we are creating a pentium4 stage 3 compiled for x86-32bit binary compatibility. Pentium4 is a subarch of the x86-32bit architecture. Once you have metro installed you may find a full list of each subarch in your {{f|/var/git/meta-repo/kits/core-kit/profiles/funtoo-1.0/linux-gnu/arch/x86-32bit/subarch}} directory:<br />
Example:<br />
{{console|body=<br />
# ##i##ls /var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-32bit/subarch/<br />
amd64-k8+sse3_32 athlon-4 athlon-xp core2_32 i486 k6-2 pentium pentium2 pentiumpro<br />
amd64-k8_32 athlon-mp atom_32 generic_32 i686 k6-3 pentium-m pentium3 prescott<br />
athlon athlon-tbird btver1 geode k6 native_32 pentium-mmx pentium4 xen-pentium4+sse3<br />
}}<br />
<br />
64-bit PC profiles can be found in the {{f|/var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-64bit/subarch/}} directory:<br />
{{console|body=<br />
# ##i##ls /var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-64bit/subarch/<br />
amd64-bulldozer amd64-k8+sse3 btver1_64 generic_64 intel64-nehalem native_64<br />
amd64-jaguar amd64-piledriver core-avx-i intel64-broadwell intel64-sandybridge nocona<br />
amd64-k10 amd64-steamroller core2_64 intel64-haswell intel64-silvermont opteron_64<br />
amd64-k8 atom_64 corei7 intel64-ivybridge intel64-westmere xen-pentium4+sse3_64<br />
}}<br />
<br />
= First stages build (local build) = <!--T:43--><br />
<br />
<!--T:44--><br />
To get this all started, we need to bootstrap the process by downloading an initial seed stage3 to use for building and place it in its proper location in {{f|/home/mirror/funtoo}}, so that Metro can find it. We will also need to create some special &quot;control&quot; files in {{f|/home/mirror/funtoo}}, which will allow Metro to understand how it is supposed to proceed.<br />
<br />
== Step 1: Set up pentium4 repository (local build) == <!--T:45--><br />
<br />
<!--T:46--><br />
Assuming we're following the basic steps outlined in the previous section, and building {{f|funtoo-current}} build for the {{f|pentium4}}, using a generic {{f|pentium4}} stage3 as a seed stage, then here the first set of steps we'd perform:<br />
<br />
<!--T:47--><br />
<console><br />
# ##i##install -d /home/mirror/funtoo/funtoo-current/x86-32bit/pentium4<br />
# ##i##install -d /home/mirror/funtoo/funtoo-current/snapshots<br />
# ##i##cd /home/mirror/funtoo/funtoo-current/x86-32bit/pentium4<br />
# ##i##install -d 2017-10-01<br />
# ##i##cd 2017-10-01<br />
# ##i##wget -c https://build.funtoo.org/funtoo-current/x86-32bit/pentium4/2017-10-01/stage3-pentium4-funtoo-current-2017-10-01.tar.xz<br />
# ##i##cd ..<br />
# ##i##install -d .control/version<br />
# ##i##echo "2017-10-01" > .control/version/stage3<br />
# ##i##install -d .control/strategy<br />
# ##i##echo local > .control/strategy/build<br />
# ##i##echo stage3 > .control/strategy/seed<br />
</console><br />
<br />
<!--T:48--><br />
OK, let's review the steps above. First, we create the directory {{f|/home/mirror/funtoo/funtoo-current/x86-32bit/pentium4}}, which is where Metro will expect to find {{f|funtoo-current}} pentium4 builds -- it is configured to look here by default. Then we create a specially-named directory to house our seed x86 stage3. Again, by default, Metro expects the directory to be named this way. We enter this directory, and download our seed x86 stage3 from funtoo.org. Note that the {{f|2017-10-01}} version stamp matches. Make sure that your directory name matches the stage3 name too. Everything has been set up to match Metro's default filesystem layout.<br />
<br />
<!--T:49--><br />
Next, we go back to the {{f|/home/mirror/metro/funtoo-current/x86-32bit/pentium4}} directory, and inside it, we create a {{f|.control}} directory. This directory and its subdirectories contain special files that Metro references to determine certain aspects of its behavior. The {{f|.control/version/stage3}} file is used by Metro to track the most recently-built stage3 for this particular build and subarch. Metro will automatically update this file with a new version stamp after it successfully builds a new stage3. But because Metro didn't actually ''build'' this stage3, we need to set up the {{f|.control/version/stage3}} file manually. This will allow Metro to find our downloaded stage3 when we set up our pentium4 build to use it as a seed. Also note that Metro will create a similar {{f|.control/version/stage1}} file after it successfully builds an pentium4 funtoo-current stage1.<br />
<br />
<!--T:50--><br />
We also set up {{f|.control/strategy/build}} and {{f|.control/strategy/seed}} files with values of {{f|local}} and {{f|stage3}} respectively. These files define the building strategy Metro will use when we build pentium4 funtoo-current stages. With a build strategy of {{f|local}}, Metro will source its seed stage from funtoo-current pentium4, the current directory. And with a seed strategy of {{f|stage3}}, Metro will use a stage3 as a seed, and use this seed to build a new stage1, stage2 and stage3.<br />
<br />
== Step 2: Building the pentium4 stages == <!--T:51--><br />
<br />
<!--T:52--><br />
Incidentally, if all you wanted to do at this point was to build a new pentium4 funtoo-current stage1/2/3 (plus openvz and vserver templates). You would begin the process by typing:<br />
<br />
<!--T:53--><br />
<console><br />
# ##i##cd /root/metro<br />
# ##i##scripts/ezbuild.sh funtoo-current x86-32bit pentium4<br />
</console><br />
<br />
<!--T:54--><br />
If you have a slow machine, it could take several hours to be completed because several "heavy" components like gcc or glibc have to be recompiled in each stage. Once a stage has been successfully completed, it is placed in the {{f|"${METRO_MIRROR}/funtoo-current/x32-bit/pentium4/YYYY-MM-DD"}} subdirectory, where {{f|YYYY-MM-DD}} is today's date at the time the {{f|ezbuild.sh}} script was started or the date you put on the ezscript.sh command line.<br />
<br />
= Building for another binary compatible architecture (remote build) = <!--T:55--><br />
<br />
<!--T:56--><br />
As written above, Metro is able to perform '''remote build''' building different architecture stage3 from a binary compatible seeding stage3 (e.g. using a pentium4 stage3 to seed a <tt>Intel Core2 32bits</tt> stage3). <br />
<br />
<!--T:57--><br />
In the Metro terminology this is called a '''remote build''' (a stage 3 of a different, but binary compatible, architecture is used as a seed). <br />
What's not compatible? You can't use a <tt>Sparc</tt> architecture to generate an <tt>x86</tt> or <tt>ARM</tt> based stage and vice-versa. If you use a 32bit stage then you don't want to seed a 64bit build from it. Be sure that you are using a stage from the same architecture that you are trying to seed. Check [http://ftp.osuosl.org/pub/funtoo/funtoo-current/ Funtoo-current FTP Mirror] for a stage that is from the same Architecture that you will be building. <br />
<br />
<!--T:58--><br />
{{Note|Often, one build (ie. funtoo-current) can be used as a seed for another build such as funtoo-stable. However, hardened builds require hardened stages as seeds in order for the build to complete successfully.}}<br />
<br />
== Step 1: Set up Core_2 32bit repository == <!--T:59--><br />
<br />
<!--T:60--><br />
In this example, we're going to use this pentium4 funtoo-current stage3 to seed a new Core_2 32bit funtoo-current build. To get that done, we need to set up the pentium4 build directory as follows:<br />
<br />
<!--T:61--><br />
<console><br />
# ##i## cd /home/mirror/funtoo/funtoo-current/x86-32bit<br />
# ##i##install -d core2_32<br />
# ##i##cd core2_32<br />
# ##i##install -d .control/strategy<br />
# ##i##echo remote > .control/strategy/build<br />
# ##i##echo stage3 > .control/strategy/seed<br />
# ##i##install -d .control/remote<br />
# ##i##echo funtoo-current > .control/remote/build<br />
# ##i##echo x86-32bit > .control/remote/arch_desc<br />
# ##i##echo pentium4 > .control/remote/subarch<br />
</console><br />
<br />
<!--T:62--><br />
The steps we follow are similar to those we performed for a ''local build'' to set up our pentium4 directory for local build. However, note the differences. We didn't download a stage, because we are going to use the pentium4 stage to build a new Core_2 32bit stage. We also didn't create the <tt>.control/version/stage{1,3}</tt> files because Metro will create them for us after it successfully builds a new stage1 and stage3. We are still using a <tt>stage3</tt> seed strategy, but we've set the build strategy to <tt>remote</tt>, which means that we're going to use a seed stage that's not from this particular subdirectory. Where are we going to get it from? The <tt>.control/remote</tt> directory contains this information, and lets Metro know that it should look for its seed stage3 in the <tt>/home/mirror/funtoo/funtoo-current/x86-32bit/pentium4</tt> directory. Which one will it grab? You guessed it -- the most recently built ''stage3'' (since our seed strategy was set to <tt>stage3</tt>) that has the version stamp of <tt>2010-12-24</tt>, as recorded in <tt>/home/mirror/funtoo-current/x86-32bit/pentium4/.control/version/stage</tt>. Now you can see how all those control files come together to direct Metro to do the right thing.<br />
<br />
<!--T:63--><br />
{{Note|<code>arch_desc</code> should be set to one of: <code>x86-32bit</code>, <code>x86-64bit</code> or <code>pure64</code> for PC-compatible systems. You must use a 32-bit build as a seed for other 32-bit builds, and a 64-bit build as a seed for other 64-bit builds.}}<br />
<br />
== Step 2: Building the Core_2 32bit stages == <!--T:64--><br />
<br />
<!--T:65--><br />
Now, you could start building your new Core_2 32bit stage1/2/3 (plus openvz and vserver templates) by typing the following:<br />
<br />
<!--T:66--><br />
<console><br />
# ##i##/root/metro/scripts/ezbuild.sh funtoo-current x86-32bit core2_32<br />
</console><br />
<br />
<!--T:67--><br />
In that case, the produced stages are placed in the <tt>/home/mirror/funtoo/funtoo-current/x32-bit/core2_32/YYYY-MM-DD</tt> subdirectory.<br />
<br />
== Step 3: The Next Build == <!--T:68--><br />
<br />
<!--T:69--><br />
At this point, you now have a new Core_2 32bit stage3, built using a "remote" pentium4 stage3. Once the first remote build completes successfully, metro will automatically change {{c|.control/strategy/build}} to be {{c|local}} instead of {{c|remote}}, so it will use the most recently-built Core_2 32bit stage3 as a seed for any new Core_2 32bit builds from now on.<br />
<br />
= Build your own tailored stage3 = <!--T:70--><br />
<br />
<!--T:71--><br />
Metro can be easily configured for building custom stage3 by including additional packages. You can find following directory {{c|/etc/builds/packages}} in your copy of metro repository and a corresponding {{c|arch}} configuration files inside:<br />
{{file|name=/etc/builds/packages/x86-64bit.conf|body=<br />
[section emerge]<br />
<br />
packages: [<br />
sys-kernel/debian-sources<br />
]<br />
}}<br />
Notice a {{c|debian-sources}} ebuild is added for all 64-bit stages. Modify the file to include (or exclude in case Funtoo add additional) packages of your choice. They will be included in your custom stage3 portage's world file.<br />
<br />
= Building Gentoo stages = <!--T:98--><br />
<br />
<!--T:99--><br />
Metro can also build Gentoo stages. After switching to Funtoo profile, see http://www.funtoo.org/Funtoo_Profiles metro require additional steps for this. We have an open bug for this -- it is simply due to the fact that we focus on ensuring Funtoo Linux builds and building Gentoo is a lower priority. Historical note: Funtoo Linux originally started as a fork of Gentoo Linux so that metro could reliably build Gentoo stages.<br />
http://www.funtoo.org/Funtoo_Profiles<br />
<br />
= Advanced Features = <!--T:100--><br />
<br />
<!--T:101--><br />
Metro also includes a number of advanced features that can be used to automate builds and set up distributed build servers. These features require you to {{c|emerge sqlalchemy}}, as SQLite is used as a dependency and also {{c|emerge dev-python/lxml}} as this is needed for index file generation.<br />
<br />
== Repository Management == <!--T:102--><br />
<br />
<!--T:103--><br />
Metro includes a script in the {{c|scripts}} directory called {{c|buildrepo}}. Buildrepo serves as the heart of Metro's advanced repository management features.<br />
<br />
=== Initial Setup === <!--T:104--><br />
<br />
<!--T:105--><br />
To use {{c|buildrepo}}, you will first need to create a {{f|.buildbot}} configuration file. Here is the file I use on my AMD Jaguar build server:<br />
<br />
<!--T:106--><br />
{{file|name=/root/.buildbot|lang=python|body=<br />
builds = (<br />
"funtoo-current",<br />
"funtoo-current-hardened",<br />
)<br />
<br />
<!--T:107--><br />
arches = (<br />
"x86-64bit",<br />
"pure64"<br />
)<br />
<br />
<!--T:108--><br />
subarches = (<br />
"amd64-jaguar",<br />
"amd64-jaguar-pure64",<br />
)<br />
<br />
<!--T:109--><br />
def map_build(build, subarch, full, full_date):<br />
# arguments refer to last build...<br />
if full == True:<br />
buildtype = ( "freshen", )<br />
else:<br />
buildtype = ("full", )<br />
# return value can be a string like "full+openvz" or a sequence type like [ "freshen", "openvz" ]<br />
return buildtype<br />
}}<br />
<br />
<!--T:110--><br />
This file is actually a python source file that defines the tuples {{c|builds}}, {{c|arches}} and {{c|subarches}}. These variables tell {{c|buildrepo}} which builds, arches and subarches it should manage. A {{c|map_build()}} function is also defined which {{c|buildbot}} uses to determine what kind of build to perform. The arguments passed to the function are based on the last successful build. The function can read these arguments and return a string to define the type of the next build. In the above example, the {{c|map_build()}} function will cause the next build after a freshen build to be a full build, and the next build after a full build to be a freshen build, so that the build will alternate between full and freshen.<br />
<br />
== Automated Builds == <!--T:111--><br />
<br />
<!--T:112--><br />
Once the {{c|.buildbot}} file has been created, the {{c|buildrepo}} and {{c|buildbot.sh}} tools are ready to use. Here's how they work. These tools are designed to keep your repository ({{c|path/mirror}} in {{f|/root/.metro}} up-to-date by inspecting your repository and looking for stages that are out-of-date. <br />
<br />
<!--T:113--><br />
To list the next build that will be performed, do this -- this is from my ARM build server:<br />
<br />
<!--T:114--><br />
{{console|body=<br />
# ##i##./buildrepo nextbuild<br />
build=funtoo-current<br />
arch_desc=arm-32bit<br />
subarch=armv7a_hardfp<br />
fulldate=2015-02-08<br />
nextdate=2015-02-20<br />
failcount=0<br />
target=full<br />
extras=''<br />
}}<br />
<br />
<!--T:115--><br />
If no output is displayed, then all your builds are up-to-date.<br />
<br />
<!--T:116--><br />
To actually run the next build, run {{c|buildbot.sh}}:<br />
<br />
<!--T:117--><br />
{{console|body=<br />
# ##i##./buildbot.sh<br />
}}<br />
<br />
<!--T:118--><br />
If you're thinking that {{c|buildbot.sh}} would be a good candidate for a cron job, you've got the right idea!<br />
<br />
=== List Builds === <!--T:119--><br />
<br />
<!--T:120--><br />
To get a quick look at our repository, let's run the {{c|buildrepo fails}} command:<br />
<br />
<!--T:121--><br />
{{console|body=<br />
# ##i##./buildrepo fails<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current/x86-64bit/amd64-jaguar<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current/pure64/amd64-jaguar-pure64<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current-hardened/x86-64bit/amd64-jaguar<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current-hardened/pure64/amd64-jaguar-pure64<br />
}}<br />
<br />
<!--T:122--><br />
On my AMD Jaguar build server, on Feb 20, 2015, this lists all the builds that {{c|buildrepo}} has been configured to manage. The first number on each line is a '''failcount''', which is the number of consecutive times that the build has failed. A zero value indicates that everything's okay. The failcount is an important feature of the advanced repository management features. Here are a number of behaviors that are implemented based on failcount:<br />
<br />
<!--T:123--><br />
* If {{c|buildbot.sh}} tries to build a stage and the build fails, the failcount is incremented.<br />
* If the build succeeds for a particular build, the failcount is reset to zero.<br />
* Builds with the lowest failcount are prioritized by {{buildrepo}} to build next, to steer towards builds that are more likely to complete successfully.<br />
* Once the failcount reaches 3 for a particular build, it is removed from the build rotation.<br />
<br />
=== Resetting Failcount === <!--T:124--><br />
<br />
<!--T:125--><br />
If a build has issues, the failcount for a build will reach 3, at which point it will be pulled out of build rotation. To clear failcount, so that these builds are attempted again -- possibly fixed by new updates to the Portage tree -- use {{c|buildrepo zap}}:<br />
<br />
<!--T:126--><br />
{{console|body=<br />
# /root/metro/scripts/buildrepo zap<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv7a_hardfp/.control/.failcount...<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv6j_hardfp/.control/.failcount...<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv5te/.control/.failcount...<br />
}}<br />
<br />
== Repository Maintenance == <!--T:127--><br />
<br />
<!--T:128--><br />
A couple of repository maintenance tools are provided:<br />
<br />
<!--T:129--><br />
* {{c|buildrepo digestgen}} will generate hash files for the archives in your repository, and clean up stale hashes. <br />
* {{c|buildrepo index.xml}} will create an index.xml file at the root of your repository, listing all builds available.<br />
* {{c|buildrepo clean}} will output a shell script that will remove old stages. No more than the three most recent stage builds for each build/arch/subarch are kept.<br />
<br />
== Distributed Repositories == <!--T:130--><br />
<br />
<!--T:131--><br />
In many situation, you will have a number of build servers, and each will build a subset of your master repository, and then upload builds to the master repository. This is an area of Metro that is being actively developed. For now, automated upload functionality is not enabled, but is expected to be implemented in the relatively near future. However, it is possible to have your master repository differentiate between subarches that are built locally, and thus should be part of that system's {{c|buildbot}} build rotation, and those that are stored locally and built remotely. These builds should be cleaned when {{c|buildrepo clean}} is run, but should not enter the local build rotation. To set this up, modify {{f|/root/.buildbot}} and use the {{c|subarches}} and {{c|all_subarches}} variables:<br />
<br />
<!--T:132--><br />
{{file|name=/root/.buildbot|desc=Excerpt of .buildbot config for master repository|body=<br />
# subarches we are building locally:<br />
<br />
<!--T:133--><br />
subarches = ( <br />
"pentium4",<br />
"athlon-xp",<br />
"corei7",<br />
"corei7-pure64",<br />
"generic_32", <br />
"i686", <br />
"amd64-k8",<br />
"amd64-k8-pure64",<br />
"core2_64",<br />
"core2_64-pure64",<br />
"generic_64",<br />
"generic_64-pure64",<br />
) <br />
<br />
# Things we need to clean, even if we may not be building:<br />
<br />
all_subarches = subarches + (<br />
"atom_32",<br />
"atom_64",<br />
"atom_64-pure64",<br />
"amd64-k10",<br />
"amd64-k10-pure64",<br />
"amd64-bulldozer",<br />
"amd64-bulldozer-pure64",<br />
"amd64-steamroller",<br />
"amd64-steamroller-pure64",<br />
"amd64-piledriver",<br />
"amd64-piledriver-pure64",<br />
"amd64-jaguar",<br />
"amd64-jaguar-pure64",<br />
"intel64-haswell",<br />
"intel64-haswell-pure64",<br />
"intel64-ivybridge-pure64",<br />
"intel64-ivybridge",<br />
"armv7a_hardfp",<br />
"armv6j_hardfp",<br />
"armv5te"<br />
) <br />
}}<br />
== Using binary cache ==<br />
Metro has built-in feature which allows to use binary packages cache rather then building same list of packages from sources. For example, core packages, such as @system are updated at slower pace and it makes sense to enable binary cache to make stage building blazing fast. However, the real disadvantage with using binary cache could be a core package update that due to internal ABI changes require rebuilding of numerous packages from sources. Good example is {{c|sys-libs/ncurses-5}} to {{c|sys-libs/ncurses-6}} major update. This is the case when you would need to disable binary cache and use regular ebuild installation from sources. To enable binary cache, in your metro git repository copy, edit the {{c|common.conf}} <br />
{{file|name=/etc/builds/common.conf|desc=Excerpt of default common.conf|body=<br />
[section metro]<br />
<br />
options:<br />
options/stage:<br />
target: gentoo<br />
}}<br />
and set {{c|cache/package}}<br />
{{file|name=/etc/builds/common.conf|desc=Excerpt of common.conf with binary cache enabled|body=<br />
[section metro]<br />
<br />
options:<br />
options/stage: cache/package<br />
target: gentoo<br />
}}<br />
<br />
During stage build metro will save package cache in {{c|/var/tmp/metro/cache/package-cache}}. With any next builds this binary package cache will be used.<br />
<!--T:134--><br />
[[Category:HOWTO]]<br />
[[Category:Metro]]<br />
__TOC__<br />
</translate><br />
[[Category:Official Documentation]]</div>Oleghttps://www.funtoo.org/index.php?title=Funtoo:Metro&diff=22390Funtoo:Metro2018-08-27T19:03:02Z<p>Oleg: </p>
<hr />
<div><languages/><br />
<translate><br />
<!--T:1--><br />
{{#layout:doc}}{{#widget:AddThis}}Metro is the build system for Funtoo Linux and [[Gentoo Linux]] stages. It automates the bootstrapping process.<br />
<br />
<!--T:2--><br />
This tutorial will take you through installing, setting up and running Metro.<br />
<br />
<!--T:3--><br />
Other [[:Category:Metro|Metro Documentation]] is available.<br />
<br />
= Preface = <!--T:4--> <br />
<br />
{{Note|If you are unfamiliar with how metro works it is recommended that you continue reading this section so you can become familiar with it. If you are already familiar with it, you may wish to use the new [[Metro AutoSetup|autosetup]] script which uses a curses based menu and allows for quickly setting up and running builds base on your choices without requiring any manual steps. Please see the [[Metro AutoSetup]] page for more details}}<br />
<br />
== How Metro Works == <!--T:5--> <br />
<br />
<!--T:6--><br />
Metro is the Funtoo Linux automated build system, and is used to build Funtoo Linux stage tarballs.<br />
<br />
<!--T:7--><br />
Metro cannot create a stage tarball out of thin air. To build a new stage tarball, Metro must use an existing, older stage tarball called a "seed" stage. This seed stage typically is used as the ''build environment'' for creating the stage we want.<br />
<br />
<!--T:8--><br />
Metro can use two kinds of seed stages. Traditionally, Metro has used a stage3 as a seed stage. This stage3 is then used to build a new stage1, which in turn is used to build a new stage2, and then a new stage3. This is generally the most reliable way to build [[Gentoo Linux]] or Funtoo Linux, so it's the recommended approach.<br />
<br />
== Seeds and Build Isolation == <!--T:9--><br />
<br />
<!--T:10--><br />
Another important concept to mention here is something called ''build isolation''. Because Metro creates an isolated build environment, and the build environment is explicitly defined using existing, tangible entities -- a seed stage and a portage snapshot -- you will get consistent, repeatable results. In other words, the same seed stage, portage snapshot and build instructions will generate an essentially identical result, even if you perform the build a month later on someone else's workstation.<br />
<br />
== Local Build == <!--T:11--> <br />
<br />
<!--T:12--><br />
Say you wanted to build a new <tt>pentium4</tt> stage3 tarball. The recommended method of doing this would be to grab an existing <tt>pentium4</tt> stage3 tarball to use as your seed stage. Metro will be told to use this existing <tt>pentium4</tt> stage3 to build a new stage1 for the same <tt>pentium4</tt>. For this process, the generic <tt>pentium4</tt> stage3 would provide the ''build environment'' for creating our new stage1. Then, the new stage1 would serve as the build environment for creating the new <tt>pentium4</tt> stage2. And the new <tt>pentium4</tt> stage2 would serve as the build environment for creating the new <tt>pentium4</tt> stage3.<br />
<br />
<!--T:13--><br />
In the Metro terminology this is called a '''local build''', which means a stage3 of a given architecture is used to seed a brand new build of the same architecture. Incidentally this will be the first exercise we are going to perform in this tutorial.<br />
<br />
<!--T:14--><br />
A week later, you may want to build a brand new <tt>pentium4</tt> stage3 tarball. Rather than starting from the original <tt>pentium4</tt> stage3 again, you'd probably configure Metro to use the most-recently-built <tt>pentium4</tt> stage3 as the seed. Metro has built-in functionality to make this easy, allowing it to easily find and track the most recent stage3 seed available.<br />
<br />
== Remote Build == <!--T:15--> <br />
<br />
<!--T:16--><br />
Metro can also perform '''remote build''', where a stage3 of a different, but binary compatible, architecture is used as a seed to build a different architecture stage3. Consequentiality the second exercise we are going to perform in this tutorial will be to build a <tt>core2 32bit</tt> stage3 tarball from the <tt>pentium4</tt> stage3 tarball we have just built.<br />
<br />
<!--T:17--><br />
TODO: add caveats about what archs can be seeded and what can be not (maybe a table?)<br />
<br />
== Tailored Build == <!--T:18--> <br />
<br />
<!--T:19--><br />
Last, it's also worthy noting that both in <tt>local</tt> and <tt>remote builds</tt>, Metro can be configured to add and/or remove individual packages to the final tarball.<br />
Let's say you can't live without <tt>app-misc/screen</tt>, at the end of this tutorial, we will show how to have your tailored stage3 to include it.<br />
<br />
== Installing Metro == <!--T:20--><br />
<br />
<!--T:21--><br />
'''The recommended and supported method''' is to use the Git repository of Metro. <br />
<br />
<!--T:22--><br />
Ensure that {{Package|dev-vcs/git}}, {{Package|dev-python/requests}} and {{Package|dev-python/boto}} (optional; required for EC2 support) are installed on your system. <br />
<br />
<!--T:23--><br />
{{console|body=<br />
# ##i##emerge dev-vcs/git dev-python/requests dev-python/boto <br />
}}<br />
<br />
<!--T:24--><br />
Next, clone the master git repository as follows:<br />
<br />
<!--T:25--><br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone git://github.com/funtoo/metro.git<br />
# ##i##cp /root/metro/metro.conf ~/.metro<br />
}}<br />
<br />
<!--T:26--><br />
You will now have a directory called {{c|/root/metro}} that contains all the Metro source code.<br />
<br />
<!--T:27--><br />
=== Setting up ego===<br />
Now, we will set the {{c|ego}}, administration tool of Funtoo/Linux. The way it is used with metro is independent from {{c|app-admin/ego}} installed on your box. Setup is easy as follows:<br />
{{console|body=<br />
# ##i##cd /root<br />
# ##i##git clone https://github.com/funtoo/ego,git<br />
}}<br />
This way you will have {{c|/root/ego}} directory with {{c|ego}} binary that is then used by metro.<br />
<br />
<!--T:28--><br />
Metro is now installed. It's time to customize it for your local system.<br />
<br />
= Configuring Metro = <!--T:28--><br />
<br />
<!--T:29--><br />
{{Note|Metro is not currently able to build Gentoo stages. See {{Bug|FL-901}}.}}<br />
<br />
<!--T:30--><br />
[[User:Drobbins|Daniel Robbins]] maintains Metro, so it comes pre-configured to successfully build Funtoo Linux releases. Before reading further, you might want to customize some basic settings like the number of concurrent jobs to fit your hardware's capabilities or the directory to use for produced stage archives. This is accomplished by editing {{c|~/.metro}} which is the Metro's master configuration file.<br />
<br />
<!--T:31--><br />
Please note that {{c|path/install}} must point to where metro was installed. Point {{c|path/distfiles}} to where your distfiles reside. Also set {{c|path/mirror/owner}} and {{c|path/mirror/group}} to the owner and group of all the files that will be written to the build repository directory, which by default (as per the configuration file) is at {{c|/home/mirror/funtoo}}. The cache directory normally resides inside the temp directory -- this can be modified as desired. The cache directory can end up holding many cached .tbz2 packages, and eat up a lot of storage. You may want to place the temp directory on faster storage, for faster compile times, and place the cache directory on slower, but more plentiful storage.<br />
<br />
<!--T:32--><br />
{{file|name=.metro|desc=Metro configuration|body=<br />
# Main metro configuration file - these settings need to be tailored to your install:<br />
<br />
<!--T:33--><br />
[section path]<br />
install: /root/metro<br />
tmp: /var/tmp/metro<br />
cache: $[path/tmp]/cache<br />
distfiles: /var/src/distfiles<br />
work: $[path/tmp]/work/$[target/build]/$[target/name]<br />
<br />
<!--T:34--><br />
[section path/mirror]<br />
<br />
<!--T:35--><br />
: /home/mirror/funtoo<br />
owner: root<br />
group: repomgr<br />
dirmode: 775<br />
<br />
<!--T:36--><br />
[section portage]<br />
<br />
<!--T:37--><br />
MAKEOPTS: auto <br />
<br />
<!--T:38--><br />
[section emerge]<br />
<br />
<!--T:39--><br />
options: --jobs=4 --load-average=4 --keep-going=n<br />
<br />
<!--T:40--><br />
# This line should not be modified:<br />
[collect $[path/install]/etc/master.conf]<br />
}}<br />
<br />
== Arch and Subarch == <!--T:41--><br />
<br />
<!--T:42--><br />
In the following example we are creating a pentium4 stage 3 compiled for x86-32bit binary compatibility. Pentium4 is a subarch of the x86-32bit architecture. Once you have metro installed you may find a full list of each subarch in your {{f|/var/git/meta-repo/kits/core-kit/profiles/funtoo-1.0/linux-gnu/arch/x86-32bit/subarch}} directory:<br />
Example:<br />
<console><br />
# ##i##ls /var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-32bit/subarch/<br />
amd64-k8+sse3_32 athlon-4 athlon-xp core2_32 i486 k6-2 pentium pentium2 pentiumpro<br />
amd64-k8_32 athlon-mp atom_32 generic_32 i686 k6-3 pentium-m pentium3 prescott<br />
athlon athlon-tbird btver1 geode k6 native_32 pentium-mmx pentium4 xen-pentium4+sse3<br />
</console><br />
<br />
64-bit PC profiles can be found in the {{f|/var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-64bit/subarch/}} directory:<br />
{{console|body=<br />
# ##i##ls /var/git/meta-repo/kits/core-kit/profiles/funtoo/1.0/linux-gnu/arch/x86-64bit/subarch/<br />
amd64-bulldozer amd64-k8+sse3 btver1_64 generic_64 intel64-nehalem native_64<br />
amd64-jaguar amd64-piledriver core-avx-i intel64-broadwell intel64-sandybridge nocona<br />
amd64-k10 amd64-steamroller core2_64 intel64-haswell intel64-silvermont opteron_64<br />
amd64-k8 atom_64 corei7 intel64-ivybridge intel64-westmere xen-pentium4+sse3_64<br />
}}<br />
<br />
= First stages build (local build) = <!--T:43--><br />
<br />
<!--T:44--><br />
To get this all started, we need to bootstrap the process by downloading an initial seed stage3 to use for building and place it in its proper location in {{f|/home/mirror/funtoo}}, so that Metro can find it. We will also need to create some special &quot;control&quot; files in {{f|/home/mirror/funtoo}}, which will allow Metro to understand how it is supposed to proceed.<br />
<br />
== Step 1: Set up pentium4 repository (local build) == <!--T:45--><br />
<br />
<!--T:46--><br />
Assuming we're following the basic steps outlined in the previous section, and building {{f|funtoo-current}} build for the {{f|pentium4}}, using a generic {{f|pentium4}} stage3 as a seed stage, then here the first set of steps we'd perform:<br />
<br />
<!--T:47--><br />
<console><br />
# ##i##install -d /home/mirror/funtoo/funtoo-current/x86-32bit/pentium4<br />
# ##i##install -d /home/mirror/funtoo/funtoo-current/snapshots<br />
# ##i##cd /home/mirror/funtoo/funtoo-current/x86-32bit/pentium4<br />
# ##i##install -d 2017-10-01<br />
# ##i##cd 2017-10-01<br />
# ##i##wget -c https://build.funtoo.org/funtoo-current/x86-32bit/pentium4/2017-10-01/stage3-pentium4-funtoo-current-2017-10-01.tar.xz<br />
# ##i##cd ..<br />
# ##i##install -d .control/version<br />
# ##i##echo "2017-10-01" > .control/version/stage3<br />
# ##i##install -d .control/strategy<br />
# ##i##echo local > .control/strategy/build<br />
# ##i##echo stage3 > .control/strategy/seed<br />
</console><br />
<br />
<!--T:48--><br />
OK, let's review the steps above. First, we create the directory {{f|/home/mirror/funtoo/funtoo-current/x86-32bit/pentium4}}, which is where Metro will expect to find {{f|funtoo-current}} pentium4 builds -- it is configured to look here by default. Then we create a specially-named directory to house our seed x86 stage3. Again, by default, Metro expects the directory to be named this way. We enter this directory, and download our seed x86 stage3 from funtoo.org. Note that the {{f|2017-10-01}} version stamp matches. Make sure that your directory name matches the stage3 name too. Everything has been set up to match Metro's default filesystem layout.<br />
<br />
<!--T:49--><br />
Next, we go back to the {{f|/home/mirror/metro/funtoo-current/x86-32bit/pentium4}} directory, and inside it, we create a {{f|.control}} directory. This directory and its subdirectories contain special files that Metro references to determine certain aspects of its behavior. The {{f|.control/version/stage3}} file is used by Metro to track the most recently-built stage3 for this particular build and subarch. Metro will automatically update this file with a new version stamp after it successfully builds a new stage3. But because Metro didn't actually ''build'' this stage3, we need to set up the {{f|.control/version/stage3}} file manually. This will allow Metro to find our downloaded stage3 when we set up our pentium4 build to use it as a seed. Also note that Metro will create a similar {{f|.control/version/stage1}} file after it successfully builds an pentium4 funtoo-current stage1.<br />
<br />
<!--T:50--><br />
We also set up {{f|.control/strategy/build}} and {{f|.control/strategy/seed}} files with values of {{f|local}} and {{f|stage3}} respectively. These files define the building strategy Metro will use when we build pentium4 funtoo-current stages. With a build strategy of {{f|local}}, Metro will source its seed stage from funtoo-current pentium4, the current directory. And with a seed strategy of {{f|stage3}}, Metro will use a stage3 as a seed, and use this seed to build a new stage1, stage2 and stage3.<br />
<br />
== Step 2: Building the pentium4 stages == <!--T:51--><br />
<br />
<!--T:52--><br />
Incidentally, if all you wanted to do at this point was to build a new pentium4 funtoo-current stage1/2/3 (plus openvz and vserver templates). You would begin the process by typing:<br />
<br />
<!--T:53--><br />
<console><br />
# ##i##cd /root/metro<br />
# ##i##scripts/ezbuild.sh funtoo-current x86-32bit pentium4<br />
</console><br />
<br />
<!--T:54--><br />
If you have a slow machine, it could take several hours to be completed because several "heavy" components like gcc or glibc have to be recompiled in each stage. Once a stage has been successfully completed, it is placed in the {{f|"${METRO_MIRROR}/funtoo-current/x32-bit/pentium4/YYYY-MM-DD"}} subdirectory, where {{f|YYYY-MM-DD}} is today's date at the time the {{f|ezbuild.sh}} script was started or the date you put on the ezscript.sh command line.<br />
<br />
= Building for another binary compatible architecture (remote build) = <!--T:55--><br />
<br />
<!--T:56--><br />
As written above, Metro is able to perform '''remote build''' building different architecture stage3 from a binary compatible seeding stage3 (e.g. using a pentium4 stage3 to seed a <tt>Intel Core2 32bits</tt> stage3). <br />
<br />
<!--T:57--><br />
In the Metro terminology this is called a '''remote build''' (a stage 3 of a different, but binary compatible, architecture is used as a seed). <br />
What's not compatible? You can't use a <tt>Sparc</tt> architecture to generate an <tt>x86</tt> or <tt>ARM</tt> based stage and vice-versa. If you use a 32bit stage then you don't want to seed a 64bit build from it. Be sure that you are using a stage from the same architecture that you are trying to seed. Check [http://ftp.osuosl.org/pub/funtoo/funtoo-current/ Funtoo-current FTP Mirror] for a stage that is from the same Architecture that you will be building. <br />
<br />
<!--T:58--><br />
{{Note|Often, one build (ie. funtoo-current) can be used as a seed for another build such as funtoo-stable. However, hardened builds require hardened stages as seeds in order for the build to complete successfully.}}<br />
<br />
== Step 1: Set up Core_2 32bit repository == <!--T:59--><br />
<br />
<!--T:60--><br />
In this example, we're going to use this pentium4 funtoo-current stage3 to seed a new Core_2 32bit funtoo-current build. To get that done, we need to set up the pentium4 build directory as follows:<br />
<br />
<!--T:61--><br />
<console><br />
# ##i## cd /home/mirror/funtoo/funtoo-current/x86-32bit<br />
# ##i##install -d core2_32<br />
# ##i##cd core2_32<br />
# ##i##install -d .control/strategy<br />
# ##i##echo remote > .control/strategy/build<br />
# ##i##echo stage3 > .control/strategy/seed<br />
# ##i##install -d .control/remote<br />
# ##i##echo funtoo-current > .control/remote/build<br />
# ##i##echo x86-32bit > .control/remote/arch_desc<br />
# ##i##echo pentium4 > .control/remote/subarch<br />
</console><br />
<br />
<!--T:62--><br />
The steps we follow are similar to those we performed for a ''local build'' to set up our pentium4 directory for local build. However, note the differences. We didn't download a stage, because we are going to use the pentium4 stage to build a new Core_2 32bit stage. We also didn't create the <tt>.control/version/stage{1,3}</tt> files because Metro will create them for us after it successfully builds a new stage1 and stage3. We are still using a <tt>stage3</tt> seed strategy, but we've set the build strategy to <tt>remote</tt>, which means that we're going to use a seed stage that's not from this particular subdirectory. Where are we going to get it from? The <tt>.control/remote</tt> directory contains this information, and lets Metro know that it should look for its seed stage3 in the <tt>/home/mirror/funtoo/funtoo-current/x86-32bit/pentium4</tt> directory. Which one will it grab? You guessed it -- the most recently built ''stage3'' (since our seed strategy was set to <tt>stage3</tt>) that has the version stamp of <tt>2010-12-24</tt>, as recorded in <tt>/home/mirror/funtoo-current/x86-32bit/pentium4/.control/version/stage</tt>. Now you can see how all those control files come together to direct Metro to do the right thing.<br />
<br />
<!--T:63--><br />
{{Note|<code>arch_desc</code> should be set to one of: <code>x86-32bit</code>, <code>x86-64bit</code> or <code>pure64</code> for PC-compatible systems. You must use a 32-bit build as a seed for other 32-bit builds, and a 64-bit build as a seed for other 64-bit builds.}}<br />
<br />
== Step 2: Building the Core_2 32bit stages == <!--T:64--><br />
<br />
<!--T:65--><br />
Now, you could start building your new Core_2 32bit stage1/2/3 (plus openvz and vserver templates) by typing the following:<br />
<br />
<!--T:66--><br />
<console><br />
# ##i##/root/metro/scripts/ezbuild.sh funtoo-current x86-32bit core2_32<br />
</console><br />
<br />
<!--T:67--><br />
In that case, the produced stages are placed in the <tt>/home/mirror/funtoo/funtoo-current/x32-bit/core2_32/YYYY-MM-DD</tt> subdirectory.<br />
<br />
== Step 3: The Next Build == <!--T:68--><br />
<br />
<!--T:69--><br />
At this point, you now have a new Core_2 32bit stage3, built using a "remote" pentium4 stage3. Once the first remote build completes successfully, metro will automatically change {{c|.control/strategy/build}} to be {{c|local}} instead of {{c|remote}}, so it will use the most recently-built Core_2 32bit stage3 as a seed for any new Core_2 32bit builds from now on.<br />
<br />
= Build your own tailored stage3 = <!--T:70--><br />
<br />
<!--T:71--><br />
Metro can be easily configured for building custom stage3 by including additional packages. You can find following directory {{c|/etc/builds/packages}} in your copy of metro repository and a corresponding {{c|arch}} configuration files inside:<br />
{{file|name=/etc/builds/packages/x86-64bit.conf|body=<br />
[section emerge]<br />
<br />
packages: [<br />
sys-kernel/debian-sources<br />
]<br />
}}<br />
Notice a {{c|debian-sources}} ebuild is added for all 64-bit stages. Modify the file to include (or exclude in case Funtoo add additional) packages of your choice. They will be included in your custom stage3 portage's world file.<br />
<br />
= Building Gentoo stages = <!--T:98--><br />
<br />
<!--T:99--><br />
Metro can also build Gentoo stages. After switching to Funtoo profile, see http://www.funtoo.org/Funtoo_Profiles metro require additional steps for this. We have an open bug for this -- it is simply due to the fact that we focus on ensuring Funtoo Linux builds and building Gentoo is a lower priority. Historical note: Funtoo Linux originally started as a fork of Gentoo Linux so that metro could reliably build Gentoo stages.<br />
http://www.funtoo.org/Funtoo_Profiles<br />
<br />
= Advanced Features = <!--T:100--><br />
<br />
<!--T:101--><br />
Metro also includes a number of advanced features that can be used to automate builds and set up distributed build servers. These features require you to {{c|emerge sqlalchemy}}, as SQLite is used as a dependency and also {{c|emerge dev-python/lxml}} as this is needed for index file generation.<br />
<br />
== Repository Management == <!--T:102--><br />
<br />
<!--T:103--><br />
Metro includes a script in the {{c|scripts}} directory called {{c|buildrepo}}. Buildrepo serves as the heart of Metro's advanced repository management features.<br />
<br />
=== Initial Setup === <!--T:104--><br />
<br />
<!--T:105--><br />
To use {{c|buildrepo}}, you will first need to create a {{f|.buildbot}} configuration file. Here is the file I use on my AMD Jaguar build server:<br />
<br />
<!--T:106--><br />
{{file|name=/root/.buildbot|lang=python|body=<br />
builds = (<br />
"funtoo-current",<br />
"funtoo-current-hardened",<br />
)<br />
<br />
<!--T:107--><br />
arches = (<br />
"x86-64bit",<br />
"pure64"<br />
)<br />
<br />
<!--T:108--><br />
subarches = (<br />
"amd64-jaguar",<br />
"amd64-jaguar-pure64",<br />
)<br />
<br />
<!--T:109--><br />
def map_build(build, subarch, full, full_date):<br />
# arguments refer to last build...<br />
if full == True:<br />
buildtype = ( "freshen", )<br />
else:<br />
buildtype = ("full", )<br />
# return value can be a string like "full+openvz" or a sequence type like [ "freshen", "openvz" ]<br />
return buildtype<br />
}}<br />
<br />
<!--T:110--><br />
This file is actually a python source file that defines the tuples {{c|builds}}, {{c|arches}} and {{c|subarches}}. These variables tell {{c|buildrepo}} which builds, arches and subarches it should manage. A {{c|map_build()}} function is also defined which {{c|buildbot}} uses to determine what kind of build to perform. The arguments passed to the function are based on the last successful build. The function can read these arguments and return a string to define the type of the next build. In the above example, the {{c|map_build()}} function will cause the next build after a freshen build to be a full build, and the next build after a full build to be a freshen build, so that the build will alternate between full and freshen.<br />
<br />
== Automated Builds == <!--T:111--><br />
<br />
<!--T:112--><br />
Once the {{c|.buildbot}} file has been created, the {{c|buildrepo}} and {{c|buildbot.sh}} tools are ready to use. Here's how they work. These tools are designed to keep your repository ({{c|path/mirror}} in {{f|/root/.metro}} up-to-date by inspecting your repository and looking for stages that are out-of-date. <br />
<br />
<!--T:113--><br />
To list the next build that will be performed, do this -- this is from my ARM build server:<br />
<br />
<!--T:114--><br />
{{console|body=<br />
# ##i##./buildrepo nextbuild<br />
build=funtoo-current<br />
arch_desc=arm-32bit<br />
subarch=armv7a_hardfp<br />
fulldate=2015-02-08<br />
nextdate=2015-02-20<br />
failcount=0<br />
target=full<br />
extras=''<br />
}}<br />
<br />
<!--T:115--><br />
If no output is displayed, then all your builds are up-to-date.<br />
<br />
<!--T:116--><br />
To actually run the next build, run {{c|buildbot.sh}}:<br />
<br />
<!--T:117--><br />
{{console|body=<br />
# ##i##./buildbot.sh<br />
}}<br />
<br />
<!--T:118--><br />
If you're thinking that {{c|buildbot.sh}} would be a good candidate for a cron job, you've got the right idea!<br />
<br />
=== List Builds === <!--T:119--><br />
<br />
<!--T:120--><br />
To get a quick look at our repository, let's run the {{c|buildrepo fails}} command:<br />
<br />
<!--T:121--><br />
{{console|body=<br />
# ##i##./buildrepo fails<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current/x86-64bit/amd64-jaguar<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current/pure64/amd64-jaguar-pure64<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current-hardened/x86-64bit/amd64-jaguar<br />
0 2015-02-18 /home/mirror/funtoo/funtoo-current-hardened/pure64/amd64-jaguar-pure64<br />
}}<br />
<br />
<!--T:122--><br />
On my AMD Jaguar build server, on Feb 20, 2015, this lists all the builds that {{c|buildrepo}} has been configured to manage. The first number on each line is a '''failcount''', which is the number of consecutive times that the build has failed. A zero value indicates that everything's okay. The failcount is an important feature of the advanced repository management features. Here are a number of behaviors that are implemented based on failcount:<br />
<br />
<!--T:123--><br />
* If {{c|buildbot.sh}} tries to build a stage and the build fails, the failcount is incremented.<br />
* If the build succeeds for a particular build, the failcount is reset to zero.<br />
* Builds with the lowest failcount are prioritized by {{buildrepo}} to build next, to steer towards builds that are more likely to complete successfully.<br />
* Once the failcount reaches 3 for a particular build, it is removed from the build rotation.<br />
<br />
=== Resetting Failcount === <!--T:124--><br />
<br />
<!--T:125--><br />
If a build has issues, the failcount for a build will reach 3, at which point it will be pulled out of build rotation. To clear failcount, so that these builds are attempted again -- possibly fixed by new updates to the Portage tree -- use {{c|buildrepo zap}}:<br />
<br />
<!--T:126--><br />
{{console|body=<br />
# /root/metro/scripts/buildrepo zap<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv7a_hardfp/.control/.failcount...<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv6j_hardfp/.control/.failcount...<br />
Removing /mnt/data/funtoo/funtoo-current/arm-32bit/armv5te/.control/.failcount...<br />
}}<br />
<br />
== Repository Maintenance == <!--T:127--><br />
<br />
<!--T:128--><br />
A couple of repository maintenance tools are provided:<br />
<br />
<!--T:129--><br />
* {{c|buildrepo digestgen}} will generate hash files for the archives in your repository, and clean up stale hashes. <br />
* {{c|buildrepo index.xml}} will create an index.xml file at the root of your repository, listing all builds available.<br />
* {{c|buildrepo clean}} will output a shell script that will remove old stages. No more than the three most recent stage builds for each build/arch/subarch are kept.<br />
<br />
== Distributed Repositories == <!--T:130--><br />
<br />
<!--T:131--><br />
In many situation, you will have a number of build servers, and each will build a subset of your master repository, and then upload builds to the master repository. This is an area of Metro that is being actively developed. For now, automated upload functionality is not enabled, but is expected to be implemented in the relatively near future. However, it is possible to have your master repository differentiate between subarches that are built locally, and thus should be part of that system's {{c|buildbot}} build rotation, and those that are stored locally and built remotely. These builds should be cleaned when {{c|buildrepo clean}} is run, but should not enter the local build rotation. To set this up, modify {{f|/root/.buildbot}} and use the {{c|subarches}} and {{c|all_subarches}} variables:<br />
<br />
<!--T:132--><br />
{{file|name=/root/.buildbot|desc=Excerpt of .buildbot config for master repository|body=<br />
# subarches we are building locally:<br />
<br />
<!--T:133--><br />
subarches = ( <br />
"pentium4",<br />
"athlon-xp",<br />
"corei7",<br />
"corei7-pure64",<br />
"generic_32", <br />
"i686", <br />
"amd64-k8",<br />
"amd64-k8-pure64",<br />
"core2_64",<br />
"core2_64-pure64",<br />
"generic_64",<br />
"generic_64-pure64",<br />
) <br />
<br />
# Things we need to clean, even if we may not be building:<br />
<br />
all_subarches = subarches + (<br />
"atom_32",<br />
"atom_64",<br />
"atom_64-pure64",<br />
"amd64-k10",<br />
"amd64-k10-pure64",<br />
"amd64-bulldozer",<br />
"amd64-bulldozer-pure64",<br />
"amd64-steamroller",<br />
"amd64-steamroller-pure64",<br />
"amd64-piledriver",<br />
"amd64-piledriver-pure64",<br />
"amd64-jaguar",<br />
"amd64-jaguar-pure64",<br />
"intel64-haswell",<br />
"intel64-haswell-pure64",<br />
"intel64-ivybridge-pure64",<br />
"intel64-ivybridge",<br />
"armv7a_hardfp",<br />
"armv6j_hardfp",<br />
"armv5te"<br />
) <br />
}}<br />
== Using binary cache ==<br />
Metro has built-in feature which allows to use binary packages cache rather then building same list of packages from sources. For example, core packages, such as @system are updated at slower pace and it makes sense to enable binary cache to make stage building blazing fast. However, the real disadvantage with using binary cache could be a core package update that due to internal ABI changes require rebuilding of numerous packages from sources. Good example is {{c|sys-libs/ncurses-5}} to {{c|sys-libs/ncurses-6}} major update. This is the case when you would need to disable binary cache and use regular ebuild installation from sources. To enable binary cache, in your metro git repository copy, edit the {{c|common.conf}} <br />
{{file|name=/etc/builds/common.conf|desc=Excerpt of default common.conf|body=<br />
[section metro]<br />
<br />
options:<br />
options/stage:<br />
target: gentoo<br />
}}<br />
and set {{c|cache/package}}<br />
{{file|name=/etc/builds/common.conf|desc=Excerpt of common.conf with binary cache enabled|body=<br />
[section metro]<br />
<br />
options:<br />
options/stage: cache/package<br />
target: gentoo<br />
}}<br />
<br />
During stage build metro will save package cache in {{c|/var/tmp/metro/cache/package-cache}}. With any next builds this binary package cache will be used.<br />
<!--T:134--><br />
[[Category:HOWTO]]<br />
[[Category:Metro]]<br />
__TOC__<br />
</translate><br />
[[Category:Official Documentation]]</div>Oleg