Difference between revisions of "Portage (Funtoo)"

Revision as of 05:14, 23 December 2010

Introduction

Portage is the official package manager of Funtoo Linux. Daniel Robbins maintains a slightly different version from upstream Gentoo Linux, with support for mini-manifests.

The recommended way to update your system

# emerge --sync
# emerge -auDN world --jobs=3

Portage Specifications, Changes, News

The latest progress and changes related to EAPI/PMS and Portage can often be found in the Gentoo Council meeting logs, which are listed on the Gentoo Council page.

The latest PMS specification to be approved by the Gentoo Council is available: eapi-3-approved-2010-01-18. The PMS specification is an attempt to codify the inner workings of Portage, and is authored by Ciaran McCreesh and Stephen Bennett as a Gentoo-hosted project.

Emerge Options

--with-bdeps=y

This option tells Portage to consider DEPEND for already-installed packages (that are not required to be considered) when updating your system. So when used with -uDN @world, the --with-bdeps=y option will cause any updated packages listed in DEPEND @world set's ebuilds -- not just those packages being installed/updated -- to be upgraded. However, if any updates are made to a package's DEPEND, it will not cause the specific package(s) containing the DEPEND to be rebuilt, so that they were explicitly built against the packages that were just upgraded. To Daniel Robbins, this seems like a half-step rather than the full upgrade step. It would be preferable if there were an option to cause any parent packages to be rebuilt if anything in DEPEND were upgraded. The reason for this is that if there is a change in DEPEND, then there is the potential for the parent ebuild to also be affected by these changes in any number of ways. So rebuilding against the most recently-upgraded DEPEND is generally safest.

Ebuild Variable Definitions

This section builds on information available in the The Gentoo Developer Guide and EAPI specifications. We will add information to this section to provide further clarification and details for Funtoo ebuild development.

Profile Variables

ARCH

ARCH defines the name for the underlying system architecture. As of late December 2010, Gentoo Linux supports defines the following architectures in various system profiles:

alpha amd64 amd64-fbsd arm cobalt fulong-2e hppa hppa-hpux ia64 ia64-hpux ip22 ip27 ip28 ip30 ip32 ip32r10k m68k m68k-mint mips mips-irix mips64 mips64el mipsel ppc ppc-aix ppc-macos ppc-openbsd ppc64 ppc64-macos s390 sgi sh sparc sparc-fbsd sparc-solaris sparc64 sparc64-fbsd sparc64-solaris x64-macos x64-openbsd x64-solaris x86 x86-cygwin x86-fbsd x86-interix x86-macos x86-netbsd x86-openbsd x86-solaris x86-winnt xbox yeeloong

This list was generated using the following command:

# cd /usr/portage/profiles
# grep -r ARCH= * | cut -f2 -d"=" | sed -e 's/"//g' | sort -u 

Path Variables

PORTDIR

PORTDIR will be set to the path of the Portage directory that contains the ebuild that is currently executing, such as /usr/portage.

PORTAGE_TMPDIR

PORTAGE_TMPDIR defines the path to the location that Portage should use for all its temporary files, and defaults to /var/tmp.

DISTDIR

DISTDIR defines the path to the location of all distfiles (downloaded source code/artifacts), and defaults to /usr/portage/distfiles.

PKGDIR

PKGDIR defines the path to where Portage will store all built binary packages, and defaults to /usr/portage/packages.

RPMDIR

RPMDIR defines the path to where Portage will store all built binary RPM packages, and defaults to /usr/portage/rpm.

ROOT

ROOT defines the root of the filesystem to which packages should be installed, and may be set on the command-line. If not defined, Portage will set ROOT to / by the time your ebuild executes. When writing pkg_* ebuild functions, all your ebuild logic must respect the setting of the ROOT variable and compensate for situations when ROOT is not /. ROOT is used by stage1 builds to install to a sub-directory. ROOT=/tmp/stage1root emerge -e system is a command that might be run by a stage1 build. After src_install executes, Portage will always install the resultant files placed in $D to the filesystem defined by ROOT.

Prefix Variables

Portage supports the ability to act as a package manager for commercial operating systems such as MacOS X, or to install ebuilds on non-Gentoo systems, or to sub-directories on Gentoo or non-Gentoo systems. When used in this capacity, ebuilds are not installed to the root filesystem but are instead installed to a subdirectory, called a "prefix". Portage's prefix variables, available in EAPI 3, support this capability, and are designed to work with ebuilds that are maintained in a separate prefix overlay.

As part of the EAPI 3 prefix support, all ebuild helpers such as econf and einstall use $ED instead of $D. See below for full variable definitions.

EPREFIX

This variable is typically set in /etc/make.conf or make.globals, and defines the prefix that should be used by ebuilds. Prefix-aware ebuilds can reference this variable and use it appropriately. if EPREFIX is not defined, then EPREFIX will default to no value.

EROOT

EROOT is a more concise way of specifying $ROOT$EPREFIX, and is intended to be used in pkg_* ebuild functions.

ED

ED is a more concise way of specifying $D$EPREFIX, and is intended to be used in src_* ebuild functions.

For more information on Portage's prefix support, see the Gentoo Prefix Techdocs.

Path Variables For Your Use

WORKDIR

WORKDIR defines the temporary filesystem location that is used for extracting source code/artifacts, preparation, configuration and compilation, and will be defined for you by the time your ebuild executes, and its value must not be modified. By default, src_unpack will extract all source code/artifacts within WORKDIR. If you need to manually extract any archives, this must all be done somewhere within WORKDIR, T or D, with WORKDIR being the preferred location for doing this. WORKDIR is automatically cleaned up by Portage upon completion.

S

S defaults to $WORKDIR/$P, but may be redefined in your ebuild to be some other sub-directory of WORKDIR. Typically, S may be redefined to reflect the real directory name that is contained within the source tarball when it differs from the default of $WORKDIR/$P. When src_prepare, src_configure, src_compile, src_test and src_install run, they will attempt to set the current working directory to S if it exists, falling back to WORKDIR if S does not exist.

T

A temporary file storage location that is available to you during your ebuild's execution. T is automatically cleaned up by Portage upon completion.

D

D defines the filesystem tree to which the src_install function should install all files. All files defined here will be installed by Portage to the destination defined by ROOT.

FILESDIR

FILESDIR is shorthand for $PORTDIR/$CATEGORY/$PN/files -- the files/ sub-directory in the Portage tree for your particular ebuild. Typically used in src_prepare for applying patches, or by src_install for installing supplementary files such as initialization scripts that were committed to the Portage tree alongside your ebuild.

Metadata Variables

EAPI

Periodically, the Gentoo Linux project defines new functionality and behavior related to how ebuilds are handled by Portage. These changes may require you to write your ebuilds slightly differently, or may allow you to take advantage of new capabilities in Portage. In your ebuild, you can specify the EAPI you are targeted by setting the EAPI variable as the first line in the ebuild. If no EAPI is specified, an EAPI of 0 ("classic" behavior) is assumed. To view a complete definition of EAPIs 1 through 3, please see The Gentoo Developer Guide's EAPI page. Note that the information on EAPI 4 is incomplete, and more detailed information can be found on in-progress EAPI 4 document maintained by Zac Medico. At some point, it may be possible to set an EAPI on a repository or overlay-wide basis, but for now, it is set by each individual ebuild.

DEPEND

DEPEND is used to specify an ebuild's build dependencies, and is a list of all ebuilds (and their corresponding versions/slots) that are required to be installed to a bare stage3 (or @system set + dependencies) in order for this package to build (compile, etc.). If one or more of these ebuilds are not installed, then it must cause the building of this package to fail; otherwise it is not a true build dependency. Do not list any dependencies for packages that are included in the underlying bare stage3 unless specific versions of the core packages (such as gcc, or glibc) are required, since these packages are expected to be available on all systems. All dependencies listed in DEPEND are satisfied prior to this package being built.

RDEPEND

RDEPEND is used to specify an ebuild's runtime dependencies, and is a list of all ebuilds (and their corresponding versions/slots) that are required to be installed to a bare stage3 (or @system set + dependencies) in order for this package to run (function). If one or more of these ebuilds are not installed, then it must cause the execution of this package to fail; otherwise it is not a true runtime dependency. Do not list any dependencies for packages that are included in the bare stage3 unless specific versions of the core packages (such as glibc) are required, since these packages are expected to be available on all systems. The dependencies listed in RDEPEND may or not be installed prior to this package being built.

DEPEND and RDEPEND

Note that if a dependency needs to be available both before the package is built, and also after the package is built, then the dependency must be listed in both DEPEND and RDEPEND, as it is both a build and runtime dependency. This is common with many packages, including virtually all shared libraries, which must be installed prior to compilation for headers and linking purposes but must continue to be available after the package is installed so that it can function properly.

(R)DEPEND if installed

It is often useful for an ebuild to conditionally depend on a specific version of another package, if it happens to be installed. Take sys-fs/lvm2 and sys-fs/udev, for example. sys-fs/lvm2 may not always be installed, but sys-fs/udev is a required ebuild. It would be nice if we could specify in the sys-fs/udev ebuild that a specific or higher version of lvm2 is required to work with this particular version of udev, or specify in the lvm2 ebuild that a specific version or higher of udev is required to work with this version of lvm2. While this is not currently possible in Portage, it is possible to use the blocker functionality in Portage to define that earlier versions are not compatible, which essentially accomplishes the same thing. To specify that another package should not be installed alongside your ebuild, use the double-exclamation from of a blocker: "!!<sys-fs/udev-160" might be an example. This will prevent both ebuilds from being installed on the system at the same time. The double-exclamation blocker is supported in EAPI 2 and above.

IUSE

IUSE specifies the USE variables that are recognized by this ebuild. Note that changing IUSE in existing ebuilds should be handled carefully, as if another ebuild depends on a particular USE variable in IUSE being enabled in your ebuild, and you remove it, then Portage will not be able to satisfy this dependency. Similarly, if another ebuild depends on a particular USE variable in IUSE being disabled in your ebuild, and you remove it, then Portage will no longer be able to satisfy this dependency either. For this reason, it can become difficult to remove USE variables from IUSE once other ebuilds explicitly depend upon them being enabled or disabled.

SRC_URI

SRC_URI contains a list of all remote files that are used by this ebuild during the unpack/prepare/compile/install phases. Starting with EAPI 2, it is possible to rename source tarballs, which comes in handy when pulling down tagged sources from GitHub, using the following trick. This approach also makes it quite easy to release source tarballs for ebuilds, as all you need to do is tag the release and then push the tags to GitHub with a git push tags command:

GITHUB_USER="foo"
SRC_URI="https://github.com/${GITHUB_USER}/${PN}/tarball/${PV} -> ${PN}-git-${PV}.tgz"
 
src_prepare() {
  cd "${WORKDIR}"/${GITHUB_USER}-${PN}-*
  S="$(pwd)"
}

KEYWORDS

KEYWORDS is a general keywording system for ebuilds, but is now used exclusively by the Gentoo architecture teams to define whether a particular ebuild is marked as stable, unstable or masked for a particular system architecture. In a user's /etc/make.conf, ACCEPT_KEYWORDS variable is defined which specifies what specific keywords should be unmasked for this system by default. This value defaults to the stable tree of the current system architecture. On Funtoo Linux systems, ACCEPT_KEYWORDS is set to one of four values:

amd64

64-bit PC-compatible stable tree

~amd64

64-bit PC-compatible unstable tree

x86

32-bit PC-compatible stable tree

~x86

32-bit PC-compatible unstable tree

An ebuild uses the KEYWORDS variable to control what tree or trees the ebuild is part of, and will contain at most one setting for each system architecture. The KEYWORD settings are defined with the following meaning:

amd64

ebuild unmasked in 64-bit PC-compatible stable and unstable trees

~amd64

ebuild unmasked in 64-bit PC-compatible stable tree only

-amd64

ebuild masked for all 64-bit PC-compatible trees

x86

ebuild unmasked in 32-bit PC-compatible stable and unstable trees

~x86

ebuild unmasked in 32-bit PC-compatible stable tree only

-x86

ebuild masked for all 32-bit PC-compatible trees

An important thing to note is that an x86 or amd64 setting in the ebuild's KEYWORDS will unmask the ebuild for both unstable and stable tree systems -- that is, any ebuild made available to stable is also made available to unstable.

Additional wildcard keywords are supported:

*

unmask for all trees. This variable can be modified by appending negative keywords: "* -ia64" means "all keywords except (~)ia64

-*

mask for all trees. This variable can be modified by appending positive keywords: "-* amd64 x86" means "no keywords except for x86 and amd64 stable trees

Gentoo Linux KEYWORDS Policies

Due to the exclusive use of KEYWORDS by the Gentoo Linux architecture teams, a number of unfortunate policy-related checks have been integrated into the Portage source code. One is banning the use of KEYWORDS="*" to enable KEYWORDS for all available architectures, since it is Gentoo Linux policy for each architecture team to independently verify an ebuild one architecture at a time - this rule currently applies to all ebuilds, including architecture-independent ebuilds. These policy checks will result in warnings being displayed by Portage when using certain combinations of wildcard keywords. This also means that virtually all ebuilds originating from Gentoo Linux will specify an exhaustive, explicit list of all architecture trees for which this package has been unmasked, which will look something like this:

KEYWORDS="~alpha ~amd64 ~arm ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64 ~s390 ~sh ~sparc ~sparc-fbsd ~x86 ~x86-fbsd"

Multiple ABI Support

Portage contains support for multiple ABIs (Application Binary Interfaces) co-existing on the same system. This functionality has been extensively documented and has been moved to its own page: Multiple ABI Support.

Ebuild Functions

An ebuild developer has the ability to define Ebuild Functions that define steps to perform during a particular part of the ebuild lifecycle. These steps are now documented on their own page: Ebuild Functions .

Funtoo Portage Development

The Funtoo Core Team is currently working on a general-purpose plug-in system so that the GNU info file regeneration, news update display, and scanning of files that need updating in /etc can be pulled out of the official Portage emerge code. In addition, this plug-in system will allow other types of things to be hooked into various phases of emerge. This will allow various new plug-ins to be developed and used on systems, such as periodic security checks, etc.

Funtoo Features/Changes

emerge --sync from git

If a git-based Portage tree is already in place, emerge --sync will run "git pull" to update the underlying Portage tree.

Sed Wrapper Symlink and PATH fix

The Funtoo version of Portage has replaced the BSD-only sed wrapper with a symlink. This will eventually be deprecated. The sed wrapper was a way to provide BSD systems with a "sed" command that could be used inside ebuilds that worked similarly to GNU sed. A PATH fix has been applied so that /bin/sed will be detected first anyway.

mini-manifest

Funtoo's Portage supports a special mode of operation where Manifests only contain digests for distfiles and not for files in the Portage tree. This is to eliminate redundant digests since git already provides SHA1 digests. This feature is currently enabled by adding "mini-manifest" to FEATURES in /etc/make.conf but the intention is to eventually move it to a repo-specific option. Funtoo provides a special "mini-manifest" tree that is smaller than the full-size Portage tree, and is intended to be used with the mini-manifest feature.

preserve bindist through USE filtering

Normally, anything not in an ebuild's IUSE is stripped from the USE passed to ebuild.sh. This patch allows "bindist" to not be stripped, so it can be used as a means to disable pre-merge sanity checks that may exist in pkg_setup() and pkg_pretend() but will not otherwise affect the resultant build. If "bindist" will affect some functionality in the package, then it should be added to an ebuild's IUSE. This patch allows pkg_setup() and pkg_pretend() to look for "bindist", which indicates that the ebuild is being built for release, typically in an automated fashion, and thus runtime sanity checks that might otherwise run can be optionally skipped. This check is used by the udev-160-r1.ebuild to see if we should fail if we are merging udev on a system where the kernel will not support it. In Metro, this is not a big deal, but on a real production system, merging the udev on an incompatible system will render the kernel inoperable.

safetydance FEATURE

A new FEATURE setting is used by Funtoo's udev ebuild called "safetydance" which can be used to manually bypass sanity checks. This is an alternative to the "bindist" approach above. udev-160 in Funtoo Linux supports both approaches and Metro sets "safetydance" by default.

GLEP 55 removal

Some code to support GLEP 55 has been removed.

new metadata format (experimental)

Some tweaks to ebuild.sh have been made so that it is easier to support new metadata formats in the future.

xz-utils auto-dependency

There are several ebuilds in the Gentoo Portage repository that use .xz files but do not explicitly depend on xz-utils. A workaround has been added to ebuild.sh to add this dependency to metadata automatically if a .xz file exists in SRC_URI. This change is not yet in the official Portage sources but is being used on the Funtoo side when generating our git-based Portage trees.

Commands

emerge

high-level dependency-based package merge/unmerge tool

ebuild

lower-level package build tool

<issues/>