Multiple ABI Support

From Funtoo
This is the approved revision of this page, as well as being the most recent.
Jump to: navigation, search

Summary

Support for multiple ABIs has been partially integrated into Portage and the Gentoo gcc wrapper (part of sys-devel/gcc-config) and has been enabled in some system profiles. Extended Multiple ABI is available as part of multilib.eclass. This page documents this functionality so it can be more easily understood, used and improved.

Core ABI Support

Portage and some system profiles currently contain a very small set of changes so that multiple ABIs can be targeted by Portage. In addition, Gentoo's gcc "wrapper" (part of sys-devel/gcc-config) has a key feature in it to enable this multiple ABI support -- this functionality is detailed later in this section. This implements the core minimal multiple ABI functionality.

In practice, this capability, combined with the gcc-wrapper's ABI support, can come in handy for directing Portage to build libraries for the non-default ABI, using the appropriate directories so that the resultant .tbz2 file can be manually installed on your system without overwriting the libraries for the default ABI. An example of this approach is documented later in this section. This support is also useful when it is necessary for the ebuild to direct the system to create 32-bit versions of some applications that may not be compatible with 64-bit multilib systems, and supports the ability for the ebuild to reliably compile both 32-bit and 64-bit versions of certain applications when necessary.

Core ABI: econf()

Portage's econf() function has support for automatically specifying the proper --libdir= setting to configure based on settings that are found in the system profile. The intended purpose of automatically setting --libdir= is designed so that 64-bit libraries will be installed into /usr/lib64 on multilib systems, and 32-bit libraries will be installed into /usr/lib32 on multilib systems, rather than simply installing these libraries into /usr/lib.

While this functionality has questionable value for most normal ebuilds (since a library installed into /usr/lib will work just as well as one installed into /usr/lib64 on an amd64 multilib system,) this functionality likely comes in handy when building the app-emulation/emul-linux-x86-* binary library bundles to support 32-bit applications, as it allows the ABI to be set in the environment, and in combination with the gcc wrapper will cause 32-bit libraries to be built and installed to /usr/lib32.

Core ABI: Gcc Wrapper

It is important to note that Gentoo Linux uses a gcc wrapper as a front-end for all calls to gcc, and this wrapper is part of the sys-devel/gcc-config ebuild. One of the features of the wrapper is that it determines whether the ABI variable has been defined in the environment, and if it has, the wrapper will automatically ensure that the CFLAGS variable that the compiler sees is actually the CFLAGS_$ABI variable, which originates from the system profile. On amd64 multilib systems, this means that a call to ABI="x86" gcc will result in an extra -m32 option being passed to gcc to force it to produce 32-bit code. The -m32 option will generally not appear in the build log, which may cause some confusion for developers. This is a critical component of the multiple ABI support in Gentoo and Funtoo Linux.

Core ABI: Working Together

The algorithm used by econf() works as follows: A system profile (or the user, via the environment) sets the ABI variable to a value like amd64 or x86. A corresponding ABI-specific variable named LIBDIR_$ABI (e.g. LIBDIR_x86) will be used from the system profile, and will be set to either lib32, lib64 or lib. This will be used to define the target --libdir setting used by econf(). This will allow the system profile to control exactly where libraries are installed when building them for a particular ABI, as long as the ebuild author uses econf() (part of core Portage) or get_libdir() (part of multilib.eclass.)

In addition, the ABI variable set in the environment will cause the gcc-wrapper to adjust the CFLAGS variable, by using the CFLAGS_$ABI variable to tell the multilib-aware gcc to target the alternate ABI. With ABI="x86" on amd64 multilib systems, this will cause -m32 to be appended to the CFLAGS variable, which will instruct gcc to produce 32-bit code.

Core ABI: Demonstration

To test the multiple ABI functionality on an amd64 multilib system, you can execute the following command:

# ABI="x86" emerge --buildpkgonly sys-libs/zlib

If you compare the libz shared library in the resultant .tbz2 package ot the one installed in /lib64, you'll note that the one in the .tbz2 is 32-bit while the one in /lib64 is 64-bit:

ninja1 lib32 # file libz.so.1.2.5
libz.so.1.2.5: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, stripped
ninja1 lib32 # file /lib/libz.so.1.2.5
/lib/libz.so.1.2.5: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, stripped

It is important to note that Gentoo's gcc wrapper (part of sys-devel/gcc-config) instructs gcc to produce 32-bit code by silently passing (not visible in the build output, due to the wrapper design)a -m32 option to all compiler calls.

Note that you should not install the 32-bit zlib .tbz2 package on a 64-bit multilib system, as it will replace the critical 64-bit zlib binaries on your system. Portage's /var/db/pkg database does not allow side-by-side installations of packages that were built against different ABIs. However, this Portage functionality can be used to build 32-bit libraries when needed, which can be installed via manual extraction of the resultant .tbz2 file to the root filesystem.

Extended ABI Support

Beyond the core functionality, the multilib.eclass (which is inherited as part of the ubiquitous eutils.eclass) contains a more significant set of code to support multiple ABIs, which appears to be designed to be eventually merged into the Portage core. This means that latent multiple ABI support is available in Portage and can be used without inheriting multilib.eclass for "regular" ebuilds, but ebuilds that need more control over the ABI configuration can inherit multilib.eclass for access to a significant number of helper functions. If you want to use multilib.eclass, first familiarize yourself with the changes in Portage that exist that support multilib.eclass functionality, which are documented in this section. Then you will have a much easier time understanding multilib.eclass.

ABI Profile Variables

The following variables are supported on a limited number of architectures - namely, those that have different ABIs available. These include Sparc, PowerPC and PC-compatible x86/amd64 architectures.

ABI
Defines the name of the current ABI for which packages should be built. This variable is recognized by the gcc wrapper - see "Core gcc wrapper ABI support", below.
DEFAULT_ABI
Defines the name of the default ABI for which packages should be built.
MULTILIB_ABIS
This consists of a space-separated string of one or more ABIs that are supported on the current system. Amd64 multilib systems will have this set to amd64 x86

ABI-Specific Profile Variables

These variables have a suffix (represented below by *) that is set based on the ABI that the particular setting is for. For example, LIBDIR_amd64 would set the library directory name for the amd64 ABI. The rationale for this naming convention is that it allows settings for multiple architectures to be defined together in a single file, and multiple ABI settings to exist on a system that may support multiple ABIs.

LIBDIR_*
CHOST_*
CDEFINE_*
CFLAGS_*
Note: this variable is used by the gcc wrapper when ABI is defined in the environment.
LDFLAGS_*
ASFLAGS_*

multilib.eclass

Note that a number of these functions can probably be replaced with enhanced profile settings, as all they do is spit out canned values based on the setting of one variable or another. They are also prime candidates for inclusion into the Portage core, possibly with some reworking or deprecation so that as many of these as possible are replaced with "dead" variables rather than "live" code.

has_multilib_profile()
This is a boolean function that returns 0 (true) if multiple ABIs are defined in the MULTILIB_ABIS variable; otherwise 1 (false).
get_libdir()
Returns the "lib" directory name to use, based on the current setting of ABI. For example, on amd64 multilib systems, this will typically return lib64, and is typically used in src_configure() like this: ./configure --libdir=/usr/$(get_libdir).
get_modname()
Used by some ebuilds that generate dynamically-loadable modules, called "bundles" on MacOS X. ELF (used by Linux) makes no differentiation between the handling of shared libraries and loadable modules (or their file extension, which is ".so",) but Mach (MacOS X) does. MacOS X bundles cannot be linked against, but can be dynamically loaded using the dyld API. Apple also recommends that they have an extension of ".bundle" rather than ".so". This function will return ".bundle" for Darwin (Mach) systems, and ".so" for everything else. For more information, see MacOS X Guide For Unix Geeks.
get_libname()
Used by a handful of ebuilds to determine the proper suffix for shared libraries on the current system. This function has various hard-coded values depending on the value of CHOST. For example, Darwin systems will get an echoed value of ".dylib" while Linux systems will get a value of ".so". Accepts an optional version argument that will be properly appended to the output.
multilib_env()
Used by toolchain.eclass, gnatbuild.eclass and the glibc ebuild - sets up environment variables for using a cross-compiler. Accepts a single argument - the target architecture in GNU format.
multilib_toolchain_setup()
In practice, this function is used exclusively to target a non-default x86 ABI on amd64 multilib systems. It accepts one argument, the name of an ABI, or default as a shorthand for the default system ABI. It will configure environment variables so the x86 (or other) compiler is targeted, and also backs up all modified variables so they can be restored later. It is typically used to allow non-64-bit-compatible code to still be installed on amd64 multilib systems, by adding the following to the top of src_configure():
src_configure() {
  use amd64 && multilib_toolchain_setup x86
  # we're now building a 32-bit app on a 64-bit system, whee!
  econf
}

ABI Support Limitations

Only a handful of applications leverage the more sophisticated functionality available in multilib.eclass, and Gentoo Linux currently uses binary bundles of 32-bit libraries to provide support for 32-bit applications on 64-bit multilib systems, rather than using Portage functionality to build these components from source. One possible explanation for this approach is that Portage currently does not allow applications to be slotted on ABI -- that is, a 32-bit and 64-bit version of sys-libs/zlib cannot co-exist in the Portage /var/db/pkg database, even if they do not overwrite one another on the filesystem when installed. There may be other possible reasons why building 32-bit packages from source remains unfeasible in Gentoo Linux, and will be documented here as they are discovered.