Difference between revisions of "FLOP:Release Engineering"

m (removed all references of chinchilla)
 
(13 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
{{FLOP
 
{{FLOP
|Summary=This is a proposal to implement a strong release engineering infrastructure for Funtoo Linux. Funtoo currently is _only_ a rolling-release distro and does not have the _option_ to also be non-rolling. In order to create a more stable Funtoo system, this proposal will be offering a few things that we can do to make that happen.
+
|Created on=2013/07/28
|Author=Fearedbliss
+
|Author=Jonathan Vasquez
|Maintainer=Fearedbliss
+
|Maintainer=Jonathan Vasquez
 
}}
 
}}
== Introduction ==
+
=== Introduction ===
  
This is a proposal to implement a strong release engineering infrastructure for Funtoo Linux. Funtoo currently is _only_ a rolling-release distro and does not have the _option_ to also be non-rolling. In order to create a more stable Funtoo system, this proposal will be offering a few things that we can do to make that happen.
+
This is a proposal to implement a strong release engineering infrastructure for Funtoo Linux. Funtoo is currently only a rolling-release distro and does not have the option to have another pace. In order to create a more stable, maintainable, and predictable Funtoo, this proposal will be offering a few things that can be done to make that happen.
  
The below proposals will not change Funtoo from a rolling-release system to a non-rolling release system. It will simply add the option to be non-rolling. Funtoo will also not be a binary distro. It will still be a source based distro but it will also have the ability to use binaries for a few select packages (Basically the Gentoo Reference Platform will be restarted and improved).
+
This proposal will not change Funtoo from being a rolling release distro, but it will instead simply add the option to also run at another pace.
  
I believe that this will also make running Funtoo for users that want further stability and/or users that are running Funtoo as a server in an enterprise environment/or otherwise, more feasible.
+
Funtoo will also not become a binary distro and will remain primarily a source based one. However, binaries do provide various advantages that allow users to have faster deployments or easier disaster recovery. For this reason Funtoo will make available binary packages for the most time consuming, and most commonly used applications.
 +
 
 +
This will make Funtoo a much more stable, maintainable, and predictable distro.
  
 
The following things are proposed:
 
The following things are proposed:
 +
* Semi-Rolling Releases
 +
* A Complete OS
 +
* Funtoo Binary Platform
  
* Semi-rolling release model for Funtoo-'''RELEASE''' and Funtoo-'''STABLE''' (Funtoo-'''CURRENT''' will stay rolling release)'''
+
=== Semi-Rolling Releases ===
* A Complete Core OS'''
+
* A set of monitored applications that will be checked for stability and consistency'''
+
  
== Semi-rolling Time Based Releases ==
+
The semi-rolling release model is a hybrid between a rolling release and a non-rolling release. This means that instead of bring new packages in all the time (rolling release), and instead of just completely freezing everything and bringing new packages/features every X months, we can have a middle ground where we can quickly and easily branch our Portage Git Tree, and focus on stabilizing it. Once we stabilize it, people can use it without having to worry about major version upgrades. The user can then use this branch until a later branch is created, and can easily upgrade to the new branch by switching their profile to the new version.
  
The semi-rolling release model is a hybrid between a rolling release and just a regular release. This means that instead of bring new packages in all the time (rolling release), and instead of just completely freezing everything and bringing new packages/features every X months, we can have a middle ground where we can branch the Funtoo-'''CURRENT''' git branch and then focus on stabilizing that tree. Once we stabilize it, people can use it without having to worry about major version upgrades. The user can then use this branch until another branch later in the future is created. The user can then easily upgrade to the new branch by switching their SYNC variable.
+
==== New Funtoo Profiles for Releases ====
  
So essentially it is a slow rolling-release (or semi-rolling or rolling-release with speed bumps).
+
Since we want to provide users the ability to easily enter and exit into a semi-rolling state, new profiles will be added periodically.
  
Names for Funtoo's 3 git branches:
+
The first and main branch for development is the “current” branch. This is the same branch that everyone is using and that is the traditional rolling release branch.
* Funtoo-'''CURRENT''' (Latest Developments - HEAD. This is the normal funtoo-current tree.)
+
  
* Funtoo-'''RELEASE''' (Just a release in a specific point in time - a branch of Funtoo-'''CURRENT''' is created, frozen, and stabilize.)
+
The second branch is the “stable” branch. This is the same stable branch that is available today, also using a rolling release approach and that uses ‘arch’ and other masks in order to provide stability.
  
* Funtoo-'''STABLE''' (This is the same as a '''RELEASE''' but it is supported for a longer period of time).
+
The new branches are considered semi-rolling branches. These branches configure your system to follow the branch selected. This funtoo branch will not introduce any major version changes and will only include tree fixes (bugs or ebuild) and security updates.
  
=== Which branch is for what person? ===
+
Example:
The Funtoo-'''CURRENT''' branch is for people who want to be on the bleeding edge all the time. You will get the latest updates, and here is where all the development happens. Your system might not be fully stable all the time, and things might fail to compile. This is literally the traditional Gentoo rolling-release model. If you are used to using Gentoo/Funtoo, and want to continue using your system the way it has always been, this is the branch for you.
+
  
The Funtoo-'''RELEASE''' branch is for people who want to be rolling and receive new features but be more stable than Funtoo-'''CURRENT''' by using a frozen portage tree at a specific point in time that is audited for stability.
+
Let's say we have a release called Funtoo 14.1 and that it was released in January 2014. In order to use this release you can select this it from the profile module:
  
The Funtoo-'''STABLE''' branch is for people who prefer stability and don't need the latest features that are in the '''RELEASE''' branches. This branch is frozen for a longer period of time.
+
<console>
 +
# ##i##eselect profile set-build 5
 +
# ##i##eselect profile list
 +
Currently available arch profiles:
 +
[1] funtoo/1.0/linux-gnu/arch/x86-64bit *
 +
[2] funtoo/1.0/linux-gnu/arch/pure64
 +
Currently available build profiles:
 +
[3] funtoo/1.0/linux-gnu/build/stable
 +
[4] funtoo/1.0/linux-gnu/build/current
 +
[5] funtoo/1.0/linux-gnu/build/14.1 *
 +
Currently available flavor profiles:
 +
[6] funtoo/1.0/linux-gnu/flavor/minimal
 +
[7] funtoo/1.0/linux-gnu/flavor/core *
 +
[8] funtoo/1.0/linux-gnu/flavor/desktop
 +
[9] funtoo/1.0/linux-gnu/flavor/workstation
 +
[10] funtoo/1.0/linux-gnu/flavor/hardened
 +
</console>
  
=== Example of 4 month RELEASE cycle, and a 2 year STABLE cycle ===
+
Once a new semi-rolling branch is released, you can change your profile to point to it. Semi-rolling releases are made to freeze the distro for the short term, and upgrading to the next semi-rolling release is recommended. New semi-rolling branches act as "upgrade packs" since you will just be getting a new set of applications that won't be changing drastically for another 4 months. There will most likely be a “Long Term” semi-rolling releases as well for people that do not like to update every 4 months.
For the branch developments: Let's picture a 4 month RELEASE cycle (semi-rolling), and a 2 years STABLE cycle. If we expand this starting from January 2013, we would get the following releases over two years:
+
  
<pre>
+
In the event that you do not update for various releases, a full system reinstall is recommended since a lot of things change over time including toolchain updates that have cascading effects.
Funtoo-13.1-STABLE  (January 2013)
+
Funtoo-13.2-RELEASE (May 2013)
+
Funtoo-13.3-RELEASE (September 2013)
+
Funtoo-14.1-RELEASE (January 2014)
+
Funtoo-14.2-RELEASE (May 2014)
+
Funtoo-14.3-RELEASE (September 2014)
+
Funtoo-15.1-STABLE  (January 2015)
+
</pre>
+
  
This gives you 2 STABLE releases in two years and 5 RELEASEs in between. That's a total of 7 releases. STABLE releases only get bugfixes and security updates. RELEASE are for people that want to get the latest bleeding edge stuff, but still want to be stable within RELEASEs.
+
==== Example of 4 Month Release Cycle ====
  
For people that are used to the normal Gentoo/Funtoo stuff, you could just stay on the Funtoo-CURRENT branch.
+
* Funtoo 14.1 (January 2014)
 +
* Funtoo 14.2 (May 2014)
 +
* Funtoo 14.3 (September 2014)
 +
* Funtoo 15.1 (January 2015)
  
=== Example of 3 month RELEASE cycle, and a 2 year STABLE cycle ===
+
==== Which branch is for what person? ====
<pre>
+
The “current” branch is for people who want to be on the bleeding edge all the time. You will get the latest updates, and here is where all the development happens. Your system might not be fully stable all the time, and things might fail to compile. This is the traditional Funtoo rolling release model. If you want to continue using your system the way it has always been, this is the branch for you.
Funtoo-13.1-STABLE  (January 2013)
+
Funtoo-13.2-RELEASE (April 2013)
+
Funtoo-13.3-RELEASE (July 2013)
+
Funtoo-13.4-RELEASE (October 2013)
+
Funtoo-14.1-RELEASE (January 2014)
+
Funtoo-14.2-RELEASE (April 2014)
+
Funtoo-14.3-RELEASE (July 2014)
+
Funtoo-14.4-RELEASE (October 2014)
+
Funtoo-15.1-STABLE  (January 2015)
+
</pre>
+
  
If we have have a 3 month (semi-rolling) cycle, we would end up with 2 STABLE releases and 7 RELEASEs within two years. This is a total of 9 releases.
+
The “stable” branch is for people who still want to be using the rolling release model but want to depend on the traditional method of ebuild’s hiding newer versions based on “~”.
  
=== What will these branches contain? What will they focus on? ===
+
The new frozen branches are for people who don’t want a lot of updates but would rather have a more stable version of the “current” tree that is audited for stability.
Funtoo-'''RELEASE''' and Funtoo-'''STABLE''' branches will only focus on the stability of specific packages that we are deciding to maintain at a specific point in time. All outside packages can be installed and maintained by outside package mantainers.
+
  
== A Complete Core OS ==
+
=== A Complete OS ===
An operating system is not just a stage3 tarball. The stage3 will not boot by itself, but rather needs the user to compile a kernel and install a bootloader. We should have a well tested default kernel that is tested for stability. This will speed up deployments and will provide predictability for kernel modules, and other applications that rely on a kernel. We will also need to provide a way for the system to boot this kernel. Thus a default bootloader should be provided.
+
  
'''Core OS:'''
+
An operating system is not just a stage3 tarball. The stage3 is incomplete and requires the user to compile their own kernel and bootloader before being able to use their system. We should have a stage which includes a well tested kernel, bootloader, and other utilities necessary for an user to deploy their system. This will speed up deployments and will provide predictability for kernel modules, and other applications that rely on a kernel.
* stage3 (Minimal @system applications needed for a functional Funtoo base system)
+
* kernel (bliss-kernel can become the base of funtoo-kernel - or another kernel you think is good)
+
* bootloader (a default bootloader to provide a seamless, fast, and easy deployment experience)
+
  
== Monitored Set of Applications ==
+
The fundamental and primary contents of a stage3 is to have a full @system, and nothing more. A stage4 would be the next iteration which includes the stage3, a kernel, bootloader, and other utilities necessary for an user to deploy their system.
In order for us to make a release stable, we will need to monitor a set of applications that we believe are essential for people that want to install servers and desktops. All of the monitored applications should work fine, they should be able to compile with no bugs (If the user is compiling), and they will also have binaries available. (The binaries will be compiled with the default and recommended USE flags that Funtoo developers believe give a functional binary).  
+
  
Proposed set of packages:
+
=== Funtoo Binary Platform ===
  
'''Core Applications:'''
+
The Funtoo Binary Platform is intended to provide binaries for the most time consuming and most commonly used applications in the Funtoo Community. There are applications that will not be provided in the the FBP. Examples of these applications are applications that require a kernel for compilation, applications that require explicit license acceptance, or applications that cannot be distributed due to certain patents/copyright issues.
* Critical packages of stage3 that provide the Funtoo base system.
+
  
'''Server Applications:'''
+
Example of some applications that can be in the FBP can be found at the link below:
{| class="wikitable"
+
|-
+
! Name !! Port
+
|-
+
| Apache || www-servers/apache
+
|-
+
| Nginx || www-servers/nginx
+
|-
+
| MariaDB || dev-db/mariadb
+
|-
+
| MySQL || dev-db/mysql
+
|-
+
| PostgreSQL || dev-db/postgre-server
+
|-
+
| SQLite || dev-db/sqlite
+
|-
+
| PHP || dev-lang/php
+
|-
+
| Python || dev-lang/python
+
|-
+
| Ruby || dev-lang/ruby
+
|-
+
| Perl || dev-lang/perl
+
|-
+
| DRBD || sys-cluster/drbd
+
|-
+
| Puppet || app-admin/puppet
+
|-
+
| Heartbeat || sys-cluster/heartbeat
+
|-
+
| Pacemaker || sys-cluster/pacemaker
+
|-
+
| Corosync || sys-cluster/corosync
+
|-
+
| phpmyadmin || dev-db/phpmyadmin
+
|-
+
| Fail2Ban || net-analyzer/fail2ban
+
|-
+
| nmap || net-analyzer/nmap
+
|-
+
| traceroute || net-analyzer/traceroute
+
|-
+
| Samba || net-fs/samba
+
|-
+
| NTP || net-misc/ntp
+
|-
+
| Dovecot || net-mail/dovecot
+
|}
+
  
 
+
https://docs.google.com/spreadsheet/ccc?key=0Aj4WbXLCgD5KdGd2Mm1VYlBzanVHMXVQU0RTN1RtX3c&usp=sharing
'''Desktop Applications:'''
+
{| class="wikitable"
+
|-
+
! Name !! Port
+
|-
+
| GNOME || gnome-base/gnome
+
|-
+
| KDE || kde-base/kde-meta
+
|-
+
| XFCE || xfce-base/xfce4-meta
+
|-
+
| Awesome || x11-wm/awesome
+
|-
+
| Ratpoison || x11-wm/ratpoison
+
|-
+
| Xmonad || x11-wm/xmonad
+
|-
+
| Openbox || x11-wm/openbox
+
|-
+
| Fluxbox || x11-wm/fluxbox
+
|-
+
| Firefox || www-client/firefox
+
|-
+
| Google Chrome || www-client/google-chrome
+
|-
+
| Chromium || www-client/chromium
+
|-
+
| Chrome Plugins for Chromium || www-plugins/chrome-binary-plugins
+
|-
+
| Google Talk Plugin || www-plugins/google-talkplugin
+
|-
+
| Adobe Flash Player || www-plugins/adobe-flash
+
|-
+
| OpenJDK || dev-java/icedtea
+
|-
+
| ISO Master || app-cdr/isomaster
+
|-
+
| LibreOffice || app-office/libreoffice
+
|-
+
| GIMP || media-gfx/gimp
+
|-
+
| VLC || media-video/vlc
+
|-
+
| Filezilla || net-ftp/filezilla
+
|-
+
| Pidgin || net-im/pidgin
+
|-
+
| Hexchat || net-irc/hexchat
+
|}
+
 
+
'''Command Line & Tools Applications:'''
+
{| class="wikitable"
+
|-
+
! Name !! Port
+
|-
+
| genlop || app-portage/genlop
+
|-
+
| gentoolkit || app-portage/gentoolkit
+
|-
+
| Dash || app-shells/dash
+
|-
+
| GNU Screen || app-misc/screen
+
|-
+
| hddtemp || app-admin/hddtemp
+
|-
+
| logrotate || app-admin/logrotate
+
|-
+
| pwgen || app-admin/pwgen
+
|-
+
| syslog-ng || app-admin/syslog-ng
+
|-
+
| sysstat || app-admin/sysstat
+
|-
+
| Parallel Bzip2 || app-arch/pbzip2
+
|-
+
| Parallel GZ || app-arch/pigz
+
|-
+
| Parallel XZ || app-arch/pxz
+
|-
+
| vim || app-editors/vim
+
|-
+
| nano || app-editors/nano
+
|-
+
| Telnet || net-misc/telnet-bsd
+
|-
+
| Ethtool || sys-apps/ethtool
+
|-
+
| GPT fdisk || sys-apps/gptfdisk
+
|-
+
| smartmon || sys-apps/smartmontools
+
|-
+
| ccache || dev-util/ccache
+
|-
+
| Mutt || mail-client/mutt
+
|-
+
| htop || sys-process/htop
+
|-
+
| lsof || sys-process/lsof
+
|-
+
| vixie-cron || sys-process/vixie-cron
+
|}
+
 
+
Of course this is just a list of applications that I've deemed important for server and desktop users. More applications should be added so that we can filter mostly used and important applications, from other more fringe applications.
+
 
+
== Other ==
+
 
+
/etc/gentoo-release -> /etc/funtoo-release  (rename this file)
+
 
+
should contain information for the currently installed release:
+
 
+
Example:
+
<pre>
+
Funtoo-13.1-STABLE  (January 2013)
+
Funtoo-13.2-RELEASE (May 2013)
+
Funtoo-13.3-RELEASE (September 2013)
+
Funtoo-14.1-RELEASE (January 2014)
+
Funtoo-14.2-RELEASE (May 2014)
+
Funtoo-14.3-RELEASE (September 2014)
+
Funtoo-15.1-STABLE  (January 2015)
+
</pre>
+
  
 
[[Category:Internals]]
 
[[Category:Internals]]
 
[[Category:FLOP]]
 
[[Category:FLOP]]
 
{{FLOPFooter}}
 
{{FLOPFooter}}

Latest revision as of 01:20, March 3, 2014

Created on
2013/07/28
Original Author(s)
Jonathan Vasquez
Current Maintainer(s)
Jonathan Vasquez

Funtoo Linux Optimization Proposal: Release Engineering

Introduction

This is a proposal to implement a strong release engineering infrastructure for Funtoo Linux. Funtoo is currently only a rolling-release distro and does not have the option to have another pace. In order to create a more stable, maintainable, and predictable Funtoo, this proposal will be offering a few things that can be done to make that happen.

This proposal will not change Funtoo from being a rolling release distro, but it will instead simply add the option to also run at another pace.

Funtoo will also not become a binary distro and will remain primarily a source based one. However, binaries do provide various advantages that allow users to have faster deployments or easier disaster recovery. For this reason Funtoo will make available binary packages for the most time consuming, and most commonly used applications.

This will make Funtoo a much more stable, maintainable, and predictable distro.

The following things are proposed:

  • Semi-Rolling Releases
  • A Complete OS
  • Funtoo Binary Platform

Semi-Rolling Releases

The semi-rolling release model is a hybrid between a rolling release and a non-rolling release. This means that instead of bring new packages in all the time (rolling release), and instead of just completely freezing everything and bringing new packages/features every X months, we can have a middle ground where we can quickly and easily branch our Portage Git Tree, and focus on stabilizing it. Once we stabilize it, people can use it without having to worry about major version upgrades. The user can then use this branch until a later branch is created, and can easily upgrade to the new branch by switching their profile to the new version.

New Funtoo Profiles for Releases

Since we want to provide users the ability to easily enter and exit into a semi-rolling state, new profiles will be added periodically.

The first and main branch for development is the “current” branch. This is the same branch that everyone is using and that is the traditional rolling release branch.

The second branch is the “stable” branch. This is the same stable branch that is available today, also using a rolling release approach and that uses ‘arch’ and other masks in order to provide stability.

The new branches are considered semi-rolling branches. These branches configure your system to follow the branch selected. This funtoo branch will not introduce any major version changes and will only include tree fixes (bugs or ebuild) and security updates.

Example:

Let's say we have a release called Funtoo 14.1 and that it was released in January 2014. In order to use this release you can select this it from the profile module:

# eselect profile set-build 5
# eselect profile list
Currently available arch profiles:
[1] funtoo/1.0/linux-gnu/arch/x86-64bit *
[2] funtoo/1.0/linux-gnu/arch/pure64
Currently available build profiles:
[3] funtoo/1.0/linux-gnu/build/stable
[4] funtoo/1.0/linux-gnu/build/current
[5] funtoo/1.0/linux-gnu/build/14.1 *
Currently available flavor profiles:
[6] funtoo/1.0/linux-gnu/flavor/minimal
[7] funtoo/1.0/linux-gnu/flavor/core *
[8] funtoo/1.0/linux-gnu/flavor/desktop
[9] funtoo/1.0/linux-gnu/flavor/workstation
[10] funtoo/1.0/linux-gnu/flavor/hardened

Once a new semi-rolling branch is released, you can change your profile to point to it. Semi-rolling releases are made to freeze the distro for the short term, and upgrading to the next semi-rolling release is recommended. New semi-rolling branches act as "upgrade packs" since you will just be getting a new set of applications that won't be changing drastically for another 4 months. There will most likely be a “Long Term” semi-rolling releases as well for people that do not like to update every 4 months.

In the event that you do not update for various releases, a full system reinstall is recommended since a lot of things change over time including toolchain updates that have cascading effects.

Example of 4 Month Release Cycle

  • Funtoo 14.1 (January 2014)
  • Funtoo 14.2 (May 2014)
  • Funtoo 14.3 (September 2014)
  • Funtoo 15.1 (January 2015)

Which branch is for what person?

The “current” branch is for people who want to be on the bleeding edge all the time. You will get the latest updates, and here is where all the development happens. Your system might not be fully stable all the time, and things might fail to compile. This is the traditional Funtoo rolling release model. If you want to continue using your system the way it has always been, this is the branch for you.

The “stable” branch is for people who still want to be using the rolling release model but want to depend on the traditional method of ebuild’s hiding newer versions based on “~”.

The new frozen branches are for people who don’t want a lot of updates but would rather have a more stable version of the “current” tree that is audited for stability.

A Complete OS

An operating system is not just a stage3 tarball. The stage3 is incomplete and requires the user to compile their own kernel and bootloader before being able to use their system. We should have a stage which includes a well tested kernel, bootloader, and other utilities necessary for an user to deploy their system. This will speed up deployments and will provide predictability for kernel modules, and other applications that rely on a kernel.

The fundamental and primary contents of a stage3 is to have a full @system, and nothing more. A stage4 would be the next iteration which includes the stage3, a kernel, bootloader, and other utilities necessary for an user to deploy their system.

Funtoo Binary Platform

The Funtoo Binary Platform is intended to provide binaries for the most time consuming and most commonly used applications in the Funtoo Community. There are applications that will not be provided in the the FBP. Examples of these applications are applications that require a kernel for compilation, applications that require explicit license acceptance, or applications that cannot be distributed due to certain patents/copyright issues.

Example of some applications that can be in the FBP can be found at the link below:

https://docs.google.com/spreadsheet/ccc?key=0Aj4WbXLCgD5KdGd2Mm1VYlBzanVHMXVQU0RTN1RtX3c&usp=sharing