Difference between revisions of "Forking An Ebuild"

(Funtoo-ification: added helper function)
(Forking an Ebuild: added convenience alias to get gentoo ebuilds)
 
Line 18: Line 18:
  
 
# Find <tt>sys-foo/bar</tt> in you regular Portage tree. Make sure you have run <tt>emerge --sync</tt> recently to ensure it is up-to-date. If you want to fork from very recent changes that are not yet in our tree, you may need to grab the most recent Gentoo Portage tree to serve as your source for <tt>sys-foo/bar</tt> (this typically isn't necessary.)
 
# Find <tt>sys-foo/bar</tt> in you regular Portage tree. Make sure you have run <tt>emerge --sync</tt> recently to ensure it is up-to-date. If you want to fork from very recent changes that are not yet in our tree, you may need to grab the most recent Gentoo Portage tree to serve as your source for <tt>sys-foo/bar</tt> (this typically isn't necessary.)
 +
<console>
 +
# alias to recursively grab latest from Gentoo Portage tree WITHOUT history
 +
# usage: getgen gentoo-x86/dev-db/mongodb
 +
alias getgen="cvs -d :pserver:anonymous@anoncvs.gentoo.org:/var/cvsroot export -D$(date '+%Y-%m-%d')"
 +
</console>
 
# Copy the <tt>sys-foo/bar</tt> directory in its entirety to <tt>funtoo-overlay/sys-foo/bar</tt>.
 
# Copy the <tt>sys-foo/bar</tt> directory in its entirety to <tt>funtoo-overlay/sys-foo/bar</tt>.
 
# Make any necessary modifications to <tt>funtoo-overlay/sys-foo/bar</tt>.
 
# Make any necessary modifications to <tt>funtoo-overlay/sys-foo/bar</tt>.

Latest revision as of 06:56, October 20, 2014

Often, a Funtoo developer needs to fork an upstream ebuild. This is necessary when we want to apply fixes to it. This page will explain the concepts of forking and how this works in the context of Funtoo.

Portage Tree Generation

Funtoo Linux generates its Portage tree using a special script that essentially takes a Gentoo tree as its starting point, and then applies various modifications to it. The modifications involve adding packages from various overlays, including our Overlay:Funtoo-overlay. Some packages added are brand new, while other packages are our special forked versions that replace existing packages.

In the vast majority of cases, when we fork a package, we take full responsibility for all ebuilds associated with that package, meaning that we have a full copy of the sys-foo/bar directory in one of our overlays.

If you're interested in seeing the actual script that does all these things, take a look at the following files:

http://git.funtoo.org/funtoo-overlay/tree/funtoo/scripts/current-update.sh
cronned script that calls merge.py.
http://git.funtoo.org/funtoo-overlay/tree/funtoo/scripts/merge.py
python script that does the heavy lifting of combining Gentoo tree with various overlays, including our flora and funtoo-overlay. When we want to change what overlays we merge, what packages we exclude as a matter of policy (such as stale packages in some overlays), we make changes to this file.
http://git.funtoo.org/funtoo-overlay/tree/funtoo/scripts/merge_utils.py
python module that contains classes and methods that implement the merging functionality.

Forking an Ebuild

In general, we fork ebuilds from Gentoo that we want to modify in some way. Before you fork an ebuild, it's important to understand that in general we fork entire packages, not just a single ebuild. This means that if you want to make some changes to sys-foo/bar, you are going to fork all sys-foo/bar ebuilds, and then Funtoo will be responsible for continuing to maintain these ebuilds until the package is unforked. Here are the steps we would use to fork sys-foo/bar:

  1. Find sys-foo/bar in you regular Portage tree. Make sure you have run emerge --sync recently to ensure it is up-to-date. If you want to fork from very recent changes that are not yet in our tree, you may need to grab the most recent Gentoo Portage tree to serve as your source for sys-foo/bar (this typically isn't necessary.)
# alias to recursively grab latest from Gentoo Portage tree WITHOUT history
# usage: getgen gentoo-x86/dev-db/mongodb
alias getgen="cvs -d :pserver:anonymous@anoncvs.gentoo.org:/var/cvsroot export -D$(date '+%Y-%m-%d')"
  1. Copy the sys-foo/bar directory in its entirety to funtoo-overlay/sys-foo/bar.
  2. Make any necessary modifications to funtoo-overlay/sys-foo/bar.
  3. Perform some funtoo-ification steps prior to commit.
  4. Add and commit the changes to funtoo-overlay.
  5. Push changes to funtoo-overlay.

At this point, the forked sys-foo/bar package will be part of funtoo-overlay. The next time our unified Portage tree is generated by merge.py (the one that users have in their /usr/portage and is updated via emerge --sync), your forked ebuild will be used in place of the Gentoo ebuild. Why is this? It is because our merge.py script has been defined with a policy that any ebuilds in funtoo-overlay will replace any existing Gentoo ebuilds if they exist. The mechanism of replacement is that our sys-foo/bar directory will be used in place of Gentoo's sys-foo/bar directory. So this is how the forking process works.

Funtoo-ification

When we fork a package from Gentoo, we perform the following tweaks to the package directory before committing:

  1. Removal of ChangeLog.
  2. Run ebuild foo-1.0.ebuild digest before committing. This will cause the Manifest file to be regenerated. Gentoo has a lot more entries in this file than we do, since we use mini-Manfiests that only include DIST listings (for distfiles only.) We want to commit our mini-Manifest (still called Manifest, just with less entries in it) rather than the one that came from Gentoo.
  3. Edit the top of each ebuild, and remove all Copyright and $Header: lines at the top of the file. We have a LICENSE.txt and COPYRIGHT.txt file in the root of our Portage tree, which is easier to maintain than keeping all the years up-to-date in each ebuild. Also, the $Header: line is there for the CVS version control system in Gentoo which Funtoo does not use. The only comment that should remain on the top of the ebuild is the one stating that it is distributed under the GPLv2..
# If you find yourself doing this often, place this function in your .bashrc, .zshrc, etc
funtooize() {
    if [ -z "$1" ]; then
        search_path='.'
    else
        search_path=$1
    fi

    find $search_path -type f -exec sed -i -e '/^# Copyright\|^# \$Header/d' {} +
    find $search_path -type f -name "ChangeLog*" -delete
    find $search_path -type f -name '*.ebuild' -exec ebuild {} manifest \;
}

Here are a few additional changes that you are allowed to make to any forked ebuilds:

  1. Line length greater than 80 characters. Gentoo enforces an 80-character line length limit. We don't.
  2. KEYWORDS of * and ~*. Gentoo does not allow these shortcuts. We do. They allow you to say "all arches" and "all unstable arches" in a concise way. Gentoo doesn't allow these shortcuts because it's Gentoo's policy to have each arch team manually approve each package. We do not have this policy so we can use the shortcuts.
  3. Use of 4-python EAPI. We allow the use of this EAPI for enhanced python functionality.