Portage API

From Funtoo
Revision as of 03:18, July 8, 2020 by D4g33z (talk | contribs) (→‎Manipulating and Comparing Package Strings)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

This page is the beginning of the missing documentation on the Portage Python API.


Portage has always had a Python API. This page represents an initial effort to document the API that is available for querying packages, etc. The API described on the page was created by Daniel Robbins and is still used internally by Portage. While Portage's API has evolved and grown over the years as various abstractions have been added to Portage's design, these fundamental API calls remain at the heart of Portage's functionality, and can be used by third-party Python applications to query and interact with the package database.

Portage DBAPI

The API that exists in Portage to query this is called the DBAPI. The "porttree" is the nickname used to describe the Portage tree, which is typically stored in /usr/portage. In these examples, we will use Portage's python DBAPI to query the porttree using the interactive python interpreter.

w520 drobbins # python
Python 3.3.5 (default, Sep 21 2015, 23:01:43) 
[GCC 4.9.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import portage
>>> portage.root
>>> p = portage.db[portage.root]["porttree"].dbapi

"porttree" contains every ebuild and "vartree" the installed ones. "bintree" is also available

p now contains a reference to the DBAPI associated with the Portage tree associated with the system installed at portage.root, which is typically /usr/portage. We can now perform various queries using this variable.

>>> p.cp_list("sys-apps/portage")
['sys-apps/portage-2.3.6-r9', 'sys-apps/portage-2.3.8']

Above, cp_list takes what is called a "catpkg", which is a reference to a particular ebuild in a repository in "category/packagename" format. We can also get lists of specific available packages, using other functions:

>>> p.match("sys-apps/portage")
['sys-apps/portage-2.3.6-r9', 'sys-apps/portage-2.3.8']
>>> p.match("=sys-apps/portage-2*")
['sys-apps/portage-2.3.6-r9', 'sys-apps/portage-2.3.8']

While at first the match() method looks no different than the cp_list() method, you can see that it accepts any valid dependency atom, which is any individual dependency reference to a particular package. The match() method is actually a shortcut for the more sophisticated xmatch() method -- it matches all visible (non-masked) packages that satisfy the dependency.

Using xmatch(), more sophisticated matching can be performed:

>>> p.xmatch("match-visible", "=sys-apps/portage-2*")
['sys-apps/portage-2.3.6-r9', 'sys-apps/portage-2.3.8']

As you can see, xmatch() with the "match_visible" gives us the identical results as match(). In fact, match() is a shortcut for this particular xmatch() call. However, xmatch() has other matching methods, detailed below:

match modeDescriptionReturn Type
bestmatch-visibleFind the best (highest) visible (unmasked) match for the dependency.single string
match-allFind all matches, masked or unmasked.list of strings
match-visibleFind all visible (unmasked) matches for the dependency.list of strings
minimum-allFind the lowest match for the dependency, ignoring masks.single string
minimum-visibleFind the lowest match for the dependency, respecting masks.single string

Querying Package Metadata

Portage is designed to extract and cache a certain set of data from each ebuild, called metadata, and this metadata is queryable and used for dependency calculations, fetching sources, etc. Querying of metadata for a particular ebuild can be performed using the aux_get() DBAPI method:

>>> p.aux_get("sys-apps/portage-2.3.8", ["EAPI", "SRC_URI"])
['5-progress', 'https://www.github.com/funtoo/portage-funtoo/tarball/funtoo-2.3.8 -> portage-funtoo-2.3.8.tar.gz']

The parameters of the aux_get() method are, first, the name of a package in "catpkg-v" format, which is a reference to a specific version of a package in "category/packagename-version(-revision)" format. The second argument is a list or tuple of metadata to return. A list is returned containing the queried metadata in the order specified in the first argument. If necessary, Portage will evaluate the contents of the ebuild to extract and cache this metadata. For porttrees, the following metadata is available (and this may vary by EAPI):

DEFINED_PHASESString containing all phases (see Ebuild Functions) defined in the ebuild, delimited by spaces.
DEPENDBuild dependencies
EAPIEbuild API version
HDEPENDHost Build Dependencies (EAPI 5-hdepend only)
INHERITEDA complete list of eclasses used by this ebuild, directly or indirectly (via eclasses inheriting other eclasses)
IUSEIUSE variable from ebuild
KEYWORDSKEYWORDS (masking) setting
LICENSELICENSE variable from ebuild
PDEPENDPDEPEND variable from ebuild
PROPERTIESPROPERTIES variable from ebuild
PROVIDEPROVIDE variable from ebuild
RDEPENDRuntime dependencies
REQUIRED_USEREQUIRED_USE variable from ebuild
repositoryRepository name ("gentoo" in main Gentoo or Funtoo Portage tree)
RESTRICTRESTRICT variable from ebuild
SLOTSLOT variable from ebuild

Manipulating and Comparing Package Strings

Portage's API contains a number of functions that are used to manipulate, split and compare Portage's various types of package strings, and these functions are defined in /usr/lib/python3.7/site-packages/portage/versions.py. Here is a list of the various functions available:

Function nameDescriptionExampleResult
catpkgsplit(cpv_string)Takes a complete catpkg-version string, and splits into category, package name, package version, package revision. Will return None if argument string is invalid -- such as missing a version, or category.
('foo-bar', 'oni', '1.0', 'r1')
catsplit(catpkg_string)Takes a string in catpkg or catpkg-version format, and splits into category and "other" part (list of length 2) at the initial "/" separator. Equivalent to string.split(catpkg_string, "/", maxsplit=1). No validation of the package or package-version part is performed.
['foo-bar', 'oni-1.0']
['foo-bar', 'oni-2asdfz00']
pkgsplit(pv_or_cpv_string)Takes a complete catpkg-version string or a pkg-version string, and splits into catpkg (or package, if pkg-version specified), package version, package revision. Will return None if argument string is invalid -- such as missing a version, or category.
('foo-bar/oni', '1.0', 'r1')
('oni', '1.0', 'r1')
ververify(version_string)Takes a version string (can include revision,) and uses regexes to determine if it is valid and supported by Portage. Returns True or False.
vercmp(version_string_1, version_string_2)Compares two version strings and determines which string is greater. If first string is a higher version, then a positive number is returned. If the second string is a higher version, a negative number is returned. If the versions are considered to be equal, then zero is returned.
portage.vercmp("1.0-r1", "1.0-r2")
pkgcmp(pkgsplit_1, pkgsplit_2)Compares two packages in pkgsplit() format. If first package is a higher version, then a positive number is returned. If the second package is a higher version, a negative number is returned. If the versions are considered to be equal, then zero is returned. Note that the package or catpkg atoms used for each pkgsplit call must match exactly (although the versions can of course differ) or the packages will be considered to be different and incomparable and None will be returned.
pkgcmp(pkgsplit("oni-1.0-r1"), pkgsplit("oni-1.0-r2"))
pkgcmp(pkgsplit("oni-1.0-r1"), pkgsplit("boni-1.0-r2"))