Difference between revisions of "Repository Configuration"

(Migrating existing configurations)
(Internals)
 
(57 intermediate revisions by 3 users not shown)
Line 1: Line 1:
{{Warning|This article is a work-in-progress referring to a future Portage version. It does not apply to the current Funtoo Portage version. Please do not update your configuration yet.}}
+
{{Warning|This article is a work-in-progress referring to functionality in Gentoo's Portage that is being evaluated. It does not apply to the current Funtoo Portage version. Please do not update your configuration yet.}}
 
+
<div style="float: right;">
 +
__TOC__
 +
</div>
 
Starting with Portage-2.3.8, a switch to a new repository configuration framework is complete and users may want to update their configuration files. This document aims to describe the goals for the new framework and how to use it.
 
Starting with Portage-2.3.8, a switch to a new repository configuration framework is complete and users may want to update their configuration files. This document aims to describe the goals for the new framework and how to use it.
  
== Multiple repository layout ==
+
== Repository layout design==
One of the most important changes is the switch from the old ''overlay'' layout to a new cleaner ''repository'' system. The new layout is more flexible and more predictable. For example, repositories can now use resources (eclasses, for example) provided by other repositories.
+
One of the most important changes is the switch from the old ''overlay'' layout to a new cleaner ''repository'' system. The new layout is more flexible and more predictable. For example, repositories can now use resources (eclasses, for example) provided by other repositories. (Editor: How?)
 +
 
 +
The old layout was based on the concept of one ''main tree'' and optionally a number of overlays. The main tree provided base system ebuilds, eclasses, profiles, while overlays mostly were able to provide their own ebuilds. The ebuild provided by overlays overrode the ebuilds in main tree to the extent of making it impossible to install the main tree version. Overlays could also provide eclasses for their own ebuilds and package.* entries that applied to all overlays and to the main tree. The Package Manager is responsible for updating the main tree, while overlays are managed externally.
 +
 
 +
The new layout is based on the concept of one or more configurable repositories. Each repository can either be stand-alone or depend upon other repositories. Users can install more repositories at they will, the repositories providing their own ebuilds, eclasses and profiles as necessary and/or using them from other repositories. Users can explicitly choose the repository they want to install packages from. The Package Manager can update all repositories.
 +
 
 +
== Old vs. New Design ==
 +
 
 +
Let's contrast the old vs. new repository configuration in a bit more detail. In the past, you might configure your system to use a main Portage tree, with a few overlays, as follows:
 +
 
 +
{{file|name=/etc/make.conf|body=
 +
PORTDIR_OVERLAY="/var/src/bleh-overlay /var/src/games-overlay /var/src/extra-overlay"}}
 +
 
 +
Above, you've added three overlays on top of your Portage tree in {{f|/usr/portage}}. The {{c|extra-overlay}} is the top-most overlay, which can override ebuilds and eclasses in all overlays and trees underneath it. As you can see, the old overlay system created a single, monolithic repository by stacking repositories and overlays on top of each other. As the stack became bigger and bigger, the interactions between the various items in the stack could become more complex. For example, maybe you wanted an ebuild from games-overlay, but extra-overlay happened to also contain a modified version of that ebuild. What if the dependencies for bleh-overlay's ebuilds were affected in some bad way by ebuilds in games-overlay, for example? The interactions between the various overlays may not always be obvious, and could cause problems.
  
The old layout was based on the concept of one ''main tree'' and optionally a number of overlays. The main tree provided base system ebuilds, eclasses, profiles, while overlays mostly were able to provide their own ebuilds. The ebuild provided by overlays overrode the ebuilds in main tree to the extend of making it impossible to install the main tree version. Overlays could also provide eclasses for their own ebuilds and package.* entries that applied to all overlays and to the main tree. The Package Manager is responsible for updating the main tree, while overlays are managed externally.
+
With the new repository layout, bleh-overlay, games-overlay and extra-overlay can contain a {{f|metadata/layout.conf}} file which specifies that they "sit on top of" the "gentoo" repository. Or -- said another way -- and using the new terminology for overlays: "their ''master'' is ''gentoo''."
  
The new layout is based on the concept of one or more configurable repositories. Each repository can either be stand-alone or depend upon other repositories. The distribution provides a repository called ''funtoo'' (a drop-in replacement for Gentoo's ''gentoo'' repository). Users can install more repositories at they will, the repositories providing their own ebuilds, eclasses and profiles as necessary and/or using them from other repositories. Users can explicitly choose the repository they want to install packages from. The Package Manager can update all repositories.
+
What this means is that each overlay can inherit things like eclasses from the gentoo repository in {{f|/usr/portage}} without having any interaction with each other. In many ways, this is much improved behavior and massively reduces the possibility of those overlays interacting in strange and unfortunate ways, which was possible with the older "monolithic stack" system implemented via {{c|PORTDIR_OVERLAY}}. Of course, with the new masters system, it is possible for an overlay to have multiple masters, which works similarly to having a stack of overlays underneath -- but in this case, it is intentional and explicitly specified by the {{f|metadata/layout.conf}} file maintained by the overlay's maintainer, rather than a side-effect of how the overlays are hooked into Portage on the local system.
  
 
== Portage configuration ==
 
== Portage configuration ==
 
=== New repository layout ===
 
=== New repository layout ===
The repository configuration should be stored in <code>/etc/portage/repos.conf</code>. It can be either a single file or a directory containing one or more ''.conf'' files.
 
  
The default configuration is installed as <code>/usr/share/portage/config/repos.conf</code>. This file is internal configuration file installed with portage ebuild and should '''not''' be modified. Instead, the configuration in <code>/etc/portage/repos.conf</code> can override the defaults specified there.
+
The repository configuration should be stored in <code>/etc/portage/repos.conf</code>. It can be either a single file or a directory containing one or more ''.conf'' files. This configuration file specifies the various repositories on your system, and also allows you to override settings that are in each repository's/overlay's {{f|metadata/layout.conf}} file.
  
The configuration uses format similar to Windows .ini files. Each section heading (repository name in square brackets) signifies a single repository, followed by one or more key-value option pairs. For example, the following file copies default configuration for Funtoo repository:
+
The default configuration is installed as <code>/usr/share/portage/config/repos.conf</code>:
  
{{file|name=/etc/portage/repos.conf/funtoo.conf|desc=Example configuration override for Funtoo repository to move it to non-standard location|body=
+
{{file|name=/usr/share/portage/config/repos.conf|body=
[funtoo]
+
[DEFAULT]
# moved to non-standard location!
+
main-repo = gentoo
location = /var/db/repos/funtoo
+
 
 +
[gentoo]
 +
location = /usr/portage
 
sync-type = git
 
sync-type = git
sync-uri = git://github.com/funtoo/ports-2015.git
+
sync-uri = git://github.com/funtoo/ports-2012.git
 
auto-sync = yes
 
auto-sync = yes
 
}}
 
}}
  
The most useful repository configuration options are listed below:
+
The configuration uses format similar to Windows .ini files. Each section heading (repository name in square brackets) signifies a single repository, followed by one or more key-value option pairs. This file is internal configuration file installed with portage ebuild and should '''not''' be modified. Instead, the configuration in <code>/etc/portage/repos.conf</code> can override the defaults in this file.
;location: ''Obligatory.'' Specifies the directory where repository is/will be stored. If Portage knows how to sync the repository and the location does not exist, it will be created on next ''emerge --sync''. Otherwise, the directory must exist.
+
;priority: Specifies the priority used for ordering ebuilds from different repositories. If two repositories provide an ebuild with matching versions, the repository with higher priority will be used.
+
;auto-sync: Specifies whether ''emerge --sync'' should update the repository. Defaults to ''yes'' if ''sync-type'' is specified, ''no'' otherwise.
+
;sync-depth: Specifies ''--depth'' for git clone. Used only on initial sync. Defaults to 1. Can be set to 0 to force full clone (not pass ''--depth'' at all).
+
;sync-type: Specifies syncing/update method. Can be one of: ''cvs'', ''git'', ''rsync'', ''svn''.
+
;sync-umask: Specifies the umask used when updating/syncing the repository.
+
;sync-uri: Specifies remote URI from which the repository will be cloned/synced. Can use any syntax valid for a particular syncing method.
+
;sync-user: Specifies the user[:group] used to update/sync the repository. If ''FEATURES=usersync'' is used, defaults to the credentials of directory owner.
+
  
Additionally a <code>[DEFAULT]</code> section may be specified. Options in this section are used as defaults for all repositories.
+
The following settings are valid in the {{c|[DEFAULT]}} section:
 +
 
 +
{{TableStart}}
 +
{{2Col|{{c|main-repo}}|Specifies the "main" repository name.}}
 +
{{2Col|{{c|eclass-overrides}}|Primarily intended for use by eclass developers, this allows one to specify a space-delimited list of repositories (by name) to look for eclasses to satisfy eclass references for ebuilds in this repository, or in this case (since we are in the {{c|[DEFAULT]}} section, ''all'' repositories. Portage will look in the specified override repository and use the eclass if it exists; if not found, it will use the eclass in this repository.}}
 +
{{2Col|{{c|force}}|Space-delimited list of settings that should be respected by repoman.}}
 +
{{TableEnd}}
 +
 
 +
For defining individual repositories, use the {{f|repos.conf}} configuration options listed below. Note that in the configuration file, hyphens are used instead of underscores for configuration variables -- the options below are listed using underscores, because that is how Portage stores these options internally.
 +
 
 +
{{TableStart}}
 +
{{2ColHead|Option|Description}}
 +
{{2Col|{{c|aliases}}|Other names that can be used to refer to this repository elsewhere. Alias names prefixed with a {{c|-}} will cause the alias, if already defined in i.e {{f|layout.conf}}, to be discarded.}}
 +
{{2Col|{{c|auto_sync}}|Specifies whether ''emerge --sync'' should update the repository. Defaults to ''yes'' if ''sync-type'' is specified, ''no'' otherwise.}}
 +
{{2Col|{{c|eclass_overrides}}|Similar to {{c|eclass_overrides}} in the {{c|[DEFAULT]}} section (see above), but applies only to this repository.}}
 +
{{2Col|{{c|force}}|Similar to {{c|force}} in the {{c|[DEFAULT]}} section (see above), but applies only to this repository.}}
 +
{{2Col|{{c|format}}|Currently unused.}}
 +
{{2Col|{{c|location}}|''Obligatory.'' Specifies the directory where repository is/will be stored. If Portage knows how to sync the repository and the location does not exist, it will be created on next ''emerge --sync''. Otherwise, the directory must exist.}}
 +
{{2Col|{{c|masters}}|This setting is typically used by repository maintainers and is set in a repository's {{f|layout.conf}} file. This setting contains a space-separated list of repository names which are considered 'masters' of this repository, or in other words, what this repository is designed to run 'on top of' (like an overlay). That means that when ebuilds are merged from this repository, the masters will be searched for eclasses, and in addition, {{f|profiles/package.*}} files will be inherited as well. Inheritance order is as follows: if repository C specifies masters of "B E", then '''eclasses''' will be searched for first in C, and if not found, then E, and if not found, then B. So, in other words, eclass search order when emerging ebuilds from repository C will be as if B is the primary repository, overlaid by E, and then overlaid by the current repository, C. For '''masking''' inheritance, things are a bit more complicated. If {{c|~}} is the function for applying masks, so that {{c|A~B}} means A's masks are modified by B's masks, then the masking order for repository C would be {{c|(B~C)~(E~C)}}. In other words, for mask inheritance, C's masks are applied to each master's masks, before they are combined. While this may seem overly complex, it allows C's {{c|package.unmask}}, etc. to be applied properly to each master, and prevents B and E from clobbering each other. This can be explored in more detail in {{f|pym/portage/package/ebuild/_config/MaskManager.py}}, starting with line {{c|for repo in repositories.repos_with_profiles():}} }}
 +
{{2Col|{{c|priority}}|Specifies the priority used for ordering ebuilds from different repositories. If two repositories provide an ebuild with matching versions, the repository with higher priority will be used.}}
 +
{{2Col|{{c|sync_cvs_repo}}|When using CVS, the repository name needs to be specified separately from the URI in {{c|sync_uri}}. This is done using this variable.}}
 +
{{2Col|{{c|sync_depth}}|Specifies ''--depth'' for git clone. Used only on initial sync. Defaults to 1. Can be set to 0 to force full clone (not pass ''--depth'' at all).}}
 +
{{2Col|{{c|sync_type}}|Specifies syncing/update method. Can be one of: ''cvs'', ''git'', ''rsync'', ''svn''.}}
 +
{{2Col|{{c|sync_umask}}|Specifies the umask used when updating/syncing the repository.}}
 +
{{2Col|{{c|sync_uri}}|Specifies remote URI from which the repository will be cloned/synced. Can use any syntax valid for a particular syncing method.}}
 +
{{2Col|{{c|sync_user}}|Specifies the user:group used to update/sync the repository. If {{c|FEATURES{{=}}"usersync"}} is used, then defaults to the credentials of the directory owner.}}
 +
{{TableEnd}}
 +
 
 +
Additionally a {{c|[DEFAULT]}} section may be specified. Options in this section are used as defaults for all repositories.
 +
 
 +
=== Internals ===
 +
 
 +
Additionally, the following options are available, or exist internally as part of the new repository configuration. Note that internally in Portage, repository configuration variables use intermediate underscores, but in configuration files, hyphens are used instead. Many of these need to be documented:
 +
 
 +
{{TableStart}}
 +
{{2Col|{{c|_eapis_banned}}|Used for repoman checks only. When set in layout.conf, tells repoman to print fatal errors if a given repo including ebuilds matching an eapis-banned entry.}}
 +
{{2Col|{{c|_eapis_deprecated}}|Same as banned. Tells repoman to print only a warning for ebuilds using EAPI matching eapis-deprecated entry.}}
 +
{{2Col|{{c|_masters_orig}}|Value of masters attribute read from layout.conf in repository, unaffected by potential override in
 +
              user's repos.conf, only for internal usage}}
 +
{{2Col|{{c|allow_missing_manifest}}|Option that allows the repository to be happy even if Manifest files are missing.}}
 +
{{2Col|{{c|allow_provide_virtual}}|Appears to have no function at the moment.}}
 +
{{2Col|{{c|cache_formats}}|The cache formats supported in the metadata tree. There is the old "pms" format and the newer "md5-dict" format.}}
 +
{{2Col|{{c|create_manifest}}|Enables 'allow_create" in Manifest object which presumably allows Manifests to be created.}}
 +
{{2Col|{{c|disable_manifest}}|Internal feature that disables parsing of Manifest file at all, by enabling "from_scratch" in Manifest object.}}
 +
{{2Col|{{c|eapi}}|}}
 +
{{2Col|{{c|eclass_db}}|Initialized to a portage.eclass_cache.cache object that allows Portage to resolve eclasses for this repository.}}
 +
{{2Col|{{c|eclass_locations}}|Eclasses used by given repository can be located in this repository or in master repositories. This variable is not configurable on user side}}
 +
{{2Col|{{c|find_invalid_path_char}}|A python function used to find invalid paths?}}
 +
{{2Col|{{c|local_config}}|Indicates whether there is local repos.conf configuration that applies to this repository.}}
 +
{{2Col|{{c|manifest_hashes}}|List of hashes to generate/check in Manifest files.  SHA1 SHA512 WHIRLPOOL for example.}}
 +
{{2Col|{{c|missing_repo_name}}|}}
 +
{{2Col|{{c|name}}|The name of the repository from the configuration file; i.e. "gentoo".}}
 +
{{2Col|{{c|profile_formats}}|Defaults to '(portage-2)' (a tuple)}}
 +
{{2Col|{{c|portage1_profiles}}|}}
 +
{{2Col|{{c|portage1_profiles_compat}}|}}
 +
{{2Col|{{c|sign_commit}}| Provides ability to sign commits in a given repo. Accept false or true value. For git, this require a key configured by `git config user.signingkey key_id` }}
 +
{{2Col|{{c|sign_manifest}}| Accept false or true value. Same functionality as sign_commit}}
 +
{{2Col|{{c|thin_manifest}}|Whether or not the repository supports "thin" manifests, i.e. "mini-manifests" -- Manifests without entries for local files.}}
 +
{{2Col|{{c|update_changelog}}|Accept false or true value. Indicates if repo set repoman's --echangelog{{=}}y/n option automatically.}}
 +
{{2Col|{{c|user_location}}|Appears to be the same as {{c|location}}, with symlinks resolved.}}
 +
{{TableEnd}}
  
 
=== Migrating existing configurations ===
 
=== Migrating existing configurations ===
 
The new configuration format provides replacement for existing configuration done through <code>/etc/portage/make.conf</code> and environment variables. While the variables are still supported for backwards compatibility, users are recommended to move to the new configuration scheme. Funtoo portage ebuild is planned to  make the migration unattended (repos.conf installed automatically to ease the config steps) with the following file:
 
The new configuration format provides replacement for existing configuration done through <code>/etc/portage/make.conf</code> and environment variables. While the variables are still supported for backwards compatibility, users are recommended to move to the new configuration scheme. Funtoo portage ebuild is planned to  make the migration unattended (repos.conf installed automatically to ease the config steps) with the following file:
 
   
 
   
{{file|name=/etc/portage/repos.conf/funtoo.conf|body=
+
{{file|name=/etc/portage/repos.conf/gentoo.conf|body=
[funtoo]
+
[gentoo]
 
location = /usr/portage
 
location = /usr/portage
 
sync-type = git
 
sync-type = git
sync-uri = git://github.com/funtoo/ports-2015.git
+
sync-uri = git://github.com/funtoo/ports-2012.git
 
auto-sync = yes
 
auto-sync = yes
 
}}
 
}}
 
The following replacements are provided for existing variables:
 
The following replacements are provided for existing variables:
;PORTDIR: Used to specify main tree location. Replaced by ''location'' key in the section corresponding to the default repository (<code>[funtoo]</code> by default).
+
 
;PORTDIR_OVERLAY: Used to specify locations of overlays. Each of the paths needs to be replaced with a separate repository section, with the path placed in ''location'' key. Additionally, ''priority'' may be used to force specific ordering of ebuild overrides.
+
{{TableStart}}
;SYNC: Used to specify URI for syncing the main repository, also implied a protocol for doing that. Replaced by the ''sync-uri'' and ''sync-type'' keys in the default repository section.
+
{{2Col|{{c|PORTDIR}}|Used to specify main tree location. Replaced by ''location'' key in the section corresponding to the default repository (<code>[gentoo]</code> by default).}}
;SYNC_UMASK: Used to specify umask for syncing repositories. Replaced by ''sync-umask'' key in repository configuration. Can be specified in <code>[DEFAULT]</code> section to apply to all repositories.
+
{{2Col|{{c|PORTDIR_OVERLAY}}|Used to specify locations of overlays. Each of the paths needs to be replaced with a separate repository section, with the path placed in ''location'' key. Additionally, ''priority'' may be used to force specific ordering of ebuild overrides.}}
;SYNC_USER: Used to specify user credentials for syncing repositories. Replaced by ''sync-user'' key in repository configuration. Can be specified in <code>[DEFAULT]</code> section to apply to all repositories.
+
{{2Col|{{c|SYNC}}|Used to specify URI for syncing the main repository, also implied a protocol for doing that. Replaced by the ''sync-uri'' and ''sync-type'' keys in the default repository section.}}
 +
{{2Col|{{c|SYNC_UMASK}}|Used to specify umask for syncing repositories. Replaced by ''sync-umask'' key in repository configuration. Can be specified in <code>[DEFAULT]</code> section to apply to all repositories.}}
 +
{{2Col|{{c|SYNC_USER}}|Used to specify user credentials for syncing repositories. Replaced by ''sync-user'' key in repository configuration. Can be specified in <code>[DEFAULT]</code> section to apply to all repositories.}}
 +
{{TableEnd}}
  
 
{{file|name=/etc/portage/make.conf|desc=Example old make.conf file|body=
 
{{file|name=/etc/portage/make.conf|desc=Example old make.conf file|body=
 
# user changed PORTDIR location
 
# user changed PORTDIR location
PORTDIR="/var/db/repos/funtoo"
+
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/var/db/repos/foo /var/db/repos/bar"
+
PORTDIR_OVERLAY="/root/git/foo /root/git/bar"
  
SYNC="git://github.com/funtoo/ports-2015.git"
+
SYNC="git://github.com/funtoo/ports-2012.git"
 
SYNC_USER="oleg"
 
SYNC_USER="oleg"
 
SYNC_UMASK="022"
 
SYNC_UMASK="022"
Line 71: Line 142:
 
sync-umask = 022
 
sync-umask = 022
  
[funtoo]
+
[gentoo]
location = /var/db/repos/funtoo
+
location = /usr/portage
 
sync-type = git
 
sync-type = git
sync-uri = git://github.com/funtoo/ports-2015.git
+
sync-uri = git://github.com/funtoo/ports-2012.git
  
 
[foo]
 
[foo]
location = /var/db/repos/foo
+
location = /root/git/foo
 
priority = 1
 
priority = 1
  
 
[bar]
 
[bar]
location = /var/db/repos/bar
+
location = /root/git/bar
 
priority = 2
 
priority = 2
 
}}
 
}}
 
+
Notice a [foo] and [bar]. This is a repository name. Let's take a look in <code>/root/git/foo/profiles/repo_name</code>.
 +
{{console|body=
 +
###i## cat /root/git/foo/profiles/repo_name
 +
foo
 +
###i## cat /root/git/bar/profiles/repo_name
 +
bar
 +
}}
 
The <code>repos.conf</code> configuration can be further extended with ''sync-type'' and ''sync-uri'' for overlays to get ''emerge --sync'' updating them automatically.
 
The <code>repos.conf</code> configuration can be further extended with ''sync-type'' and ''sync-uri'' for overlays to get ''emerge --sync'' updating them automatically.
  
let's see a real example of tree and overlays added.  
+
Let's see a real example of tree and overlays added.  
 
{{file|name=/etc/portage/repos.conf|desc=Replacement repos.conf file|body=
 
{{file|name=/etc/portage/repos.conf|desc=Replacement repos.conf file|body=
  
Line 101: Line 178:
 
location = /root/git/funtoo-gnome-overlay
 
location = /root/git/funtoo-gnome-overlay
 
}}
 
}}
funtoo-overlay and funtoo-gnome-overlay are an overlays added on top of regular portage tree.
+
funtoo-overlay and funtoo-gnome-overlay are an overlays added on top of regular portage tree. How to migrate? Portage ebuild is made to perform unattended migration. <code>/etc/portage/repos.conf/gentoo.conf</code> automatically created. Also, it trying to detect existing overlays and change <code>repos.conf</code> accordingly and ask users to run <code>dispatch-conf</code> or <code>etc-update</code> for configuration file update.
 +
 
 
[[Category:Portage]]
 
[[Category:Portage]]

Latest revision as of 03:45, July 24, 2015

Warning

This article is a work-in-progress referring to functionality in Gentoo's Portage that is being evaluated. It does not apply to the current Funtoo Portage version. Please do not update your configuration yet.

Starting with Portage-2.3.8, a switch to a new repository configuration framework is complete and users may want to update their configuration files. This document aims to describe the goals for the new framework and how to use it.

Repository layout design

One of the most important changes is the switch from the old overlay layout to a new cleaner repository system. The new layout is more flexible and more predictable. For example, repositories can now use resources (eclasses, for example) provided by other repositories. (Editor: How?)

The old layout was based on the concept of one main tree and optionally a number of overlays. The main tree provided base system ebuilds, eclasses, profiles, while overlays mostly were able to provide their own ebuilds. The ebuild provided by overlays overrode the ebuilds in main tree to the extent of making it impossible to install the main tree version. Overlays could also provide eclasses for their own ebuilds and package.* entries that applied to all overlays and to the main tree. The Package Manager is responsible for updating the main tree, while overlays are managed externally.

The new layout is based on the concept of one or more configurable repositories. Each repository can either be stand-alone or depend upon other repositories. Users can install more repositories at they will, the repositories providing their own ebuilds, eclasses and profiles as necessary and/or using them from other repositories. Users can explicitly choose the repository they want to install packages from. The Package Manager can update all repositories.

Old vs. New Design

Let's contrast the old vs. new repository configuration in a bit more detail. In the past, you might configure your system to use a main Portage tree, with a few overlays, as follows:

/etc/make.conf
PORTDIR_OVERLAY="/var/src/bleh-overlay /var/src/games-overlay /var/src/extra-overlay"

Above, you've added three overlays on top of your Portage tree in /usr/portage. The extra-overlay is the top-most overlay, which can override ebuilds and eclasses in all overlays and trees underneath it. As you can see, the old overlay system created a single, monolithic repository by stacking repositories and overlays on top of each other. As the stack became bigger and bigger, the interactions between the various items in the stack could become more complex. For example, maybe you wanted an ebuild from games-overlay, but extra-overlay happened to also contain a modified version of that ebuild. What if the dependencies for bleh-overlay's ebuilds were affected in some bad way by ebuilds in games-overlay, for example? The interactions between the various overlays may not always be obvious, and could cause problems.

With the new repository layout, bleh-overlay, games-overlay and extra-overlay can contain a metadata/layout.conf file which specifies that they "sit on top of" the "gentoo" repository. Or -- said another way -- and using the new terminology for overlays: "their master is gentoo."

What this means is that each overlay can inherit things like eclasses from the gentoo repository in /usr/portage without having any interaction with each other. In many ways, this is much improved behavior and massively reduces the possibility of those overlays interacting in strange and unfortunate ways, which was possible with the older "monolithic stack" system implemented via PORTDIR_OVERLAY. Of course, with the new masters system, it is possible for an overlay to have multiple masters, which works similarly to having a stack of overlays underneath -- but in this case, it is intentional and explicitly specified by the metadata/layout.conf file maintained by the overlay's maintainer, rather than a side-effect of how the overlays are hooked into Portage on the local system.

Portage configuration

New repository layout

The repository configuration should be stored in /etc/portage/repos.conf. It can be either a single file or a directory containing one or more .conf files. This configuration file specifies the various repositories on your system, and also allows you to override settings that are in each repository's/overlay's metadata/layout.conf file.

The default configuration is installed as /usr/share/portage/config/repos.conf:

/usr/share/portage/config/repos.conf
[DEFAULT]
main-repo = gentoo

[gentoo]
location = /usr/portage
sync-type = git
sync-uri = git://github.com/funtoo/ports-2012.git
auto-sync = yes

The configuration uses format similar to Windows .ini files. Each section heading (repository name in square brackets) signifies a single repository, followed by one or more key-value option pairs. This file is internal configuration file installed with portage ebuild and should not be modified. Instead, the configuration in /etc/portage/repos.conf can override the defaults in this file.

The following settings are valid in the [DEFAULT] section:

main-repoSpecifies the "main" repository name.
eclass-overridesPrimarily intended for use by eclass developers, this allows one to specify a space-delimited list of repositories (by name) to look for eclasses to satisfy eclass references for ebuilds in this repository, or in this case (since we are in the [DEFAULT] section, all repositories. Portage will look in the specified override repository and use the eclass if it exists; if not found, it will use the eclass in this repository.
forceSpace-delimited list of settings that should be respected by repoman.

For defining individual repositories, use the repos.conf configuration options listed below. Note that in the configuration file, hyphens are used instead of underscores for configuration variables -- the options below are listed using underscores, because that is how Portage stores these options internally.

OptionDescription
aliasesOther names that can be used to refer to this repository elsewhere. Alias names prefixed with a - will cause the alias, if already defined in i.e layout.conf, to be discarded.
auto_syncSpecifies whether emerge --sync should update the repository. Defaults to yes if sync-type is specified, no otherwise.
eclass_overridesSimilar to eclass_overrides in the [DEFAULT] section (see above), but applies only to this repository.
forceSimilar to force in the [DEFAULT] section (see above), but applies only to this repository.
formatCurrently unused.
locationObligatory. Specifies the directory where repository is/will be stored. If Portage knows how to sync the repository and the location does not exist, it will be created on next emerge --sync. Otherwise, the directory must exist.
mastersThis setting is typically used by repository maintainers and is set in a repository's layout.conf file. This setting contains a space-separated list of repository names which are considered 'masters' of this repository, or in other words, what this repository is designed to run 'on top of' (like an overlay). That means that when ebuilds are merged from this repository, the masters will be searched for eclasses, and in addition, profiles/package.* files will be inherited as well. Inheritance order is as follows: if repository C specifies masters of "B E", then eclasses will be searched for first in C, and if not found, then E, and if not found, then B. So, in other words, eclass search order when emerging ebuilds from repository C will be as if B is the primary repository, overlaid by E, and then overlaid by the current repository, C. For masking inheritance, things are a bit more complicated. If ~ is the function for applying masks, so that A~B means A's masks are modified by B's masks, then the masking order for repository C would be (B~C)~(E~C). In other words, for mask inheritance, C's masks are applied to each master's masks, before they are combined. While this may seem overly complex, it allows C's package.unmask, etc. to be applied properly to each master, and prevents B and E from clobbering each other. This can be explored in more detail in pym/portage/package/ebuild/_config/MaskManager.py, starting with line for repo in repositories.repos_with_profiles():
prioritySpecifies the priority used for ordering ebuilds from different repositories. If two repositories provide an ebuild with matching versions, the repository with higher priority will be used.
sync_cvs_repoWhen using CVS, the repository name needs to be specified separately from the URI in sync_uri. This is done using this variable.
sync_depthSpecifies --depth for git clone. Used only on initial sync. Defaults to 1. Can be set to 0 to force full clone (not pass --depth at all).
sync_typeSpecifies syncing/update method. Can be one of: cvs, git, rsync, svn.
sync_umaskSpecifies the umask used when updating/syncing the repository.
sync_uriSpecifies remote URI from which the repository will be cloned/synced. Can use any syntax valid for a particular syncing method.
sync_userSpecifies the user:group used to update/sync the repository. If FEATURES="usersync" is used, then defaults to the credentials of the directory owner.

Additionally a [DEFAULT] section may be specified. Options in this section are used as defaults for all repositories.

Internals

Additionally, the following options are available, or exist internally as part of the new repository configuration. Note that internally in Portage, repository configuration variables use intermediate underscores, but in configuration files, hyphens are used instead. Many of these need to be documented:

_eapis_bannedUsed for repoman checks only. When set in layout.conf, tells repoman to print fatal errors if a given repo including ebuilds matching an eapis-banned entry.
_eapis_deprecatedSame as banned. Tells repoman to print only a warning for ebuilds using EAPI matching eapis-deprecated entry.
_masters_origValue of masters attribute read from layout.conf in repository, unaffected by potential override in user's repos.conf, only for internal usage
allow_missing_manifestOption that allows the repository to be happy even if Manifest files are missing.
allow_provide_virtualAppears to have no function at the moment.
cache_formatsThe cache formats supported in the metadata tree. There is the old "pms" format and the newer "md5-dict" format.
create_manifestEnables 'allow_create" in Manifest object which presumably allows Manifests to be created.
disable_manifestInternal feature that disables parsing of Manifest file at all, by enabling "from_scratch" in Manifest object.
eapi
eclass_dbInitialized to a portage.eclass_cache.cache object that allows Portage to resolve eclasses for this repository.
eclass_locationsEclasses used by given repository can be located in this repository or in master repositories. This variable is not configurable on user side
find_invalid_path_charA python function used to find invalid paths?
local_configIndicates whether there is local repos.conf configuration that applies to this repository.
manifest_hashesList of hashes to generate/check in Manifest files. SHA1 SHA512 WHIRLPOOL for example.
missing_repo_name
nameThe name of the repository from the configuration file; i.e. "gentoo".
profile_formatsDefaults to '(portage-2)' (a tuple)
portage1_profiles
portage1_profiles_compat
sign_commit Provides ability to sign commits in a given repo. Accept false or true value. For git, this require a key configured by `git config user.signingkey key_id`
sign_manifest Accept false or true value. Same functionality as sign_commit
thin_manifestWhether or not the repository supports "thin" manifests, i.e. "mini-manifests" -- Manifests without entries for local files.
update_changelogAccept false or true value. Indicates if repo set repoman's --echangelog=y/n option automatically.
user_locationAppears to be the same as location, with symlinks resolved.

Migrating existing configurations

The new configuration format provides replacement for existing configuration done through /etc/portage/make.conf and environment variables. While the variables are still supported for backwards compatibility, users are recommended to move to the new configuration scheme. Funtoo portage ebuild is planned to make the migration unattended (repos.conf installed automatically to ease the config steps) with the following file:

/etc/portage/repos.conf/gentoo.conf
[gentoo]
location = /usr/portage
sync-type = git
sync-uri = git://github.com/funtoo/ports-2012.git
auto-sync = yes

The following replacements are provided for existing variables:

PORTDIRUsed to specify main tree location. Replaced by location key in the section corresponding to the default repository ([gentoo] by default).
PORTDIR_OVERLAYUsed to specify locations of overlays. Each of the paths needs to be replaced with a separate repository section, with the path placed in location key. Additionally, priority may be used to force specific ordering of ebuild overrides.
SYNCUsed to specify URI for syncing the main repository, also implied a protocol for doing that. Replaced by the sync-uri and sync-type keys in the default repository section.
SYNC_UMASKUsed to specify umask for syncing repositories. Replaced by sync-umask key in repository configuration. Can be specified in [DEFAULT] section to apply to all repositories.
SYNC_USERUsed to specify user credentials for syncing repositories. Replaced by sync-user key in repository configuration. Can be specified in [DEFAULT] section to apply to all repositories.
/etc/portage/make.conf - Example old make.conf file
# user changed PORTDIR location
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/root/git/foo /root/git/bar"

SYNC="git://github.com/funtoo/ports-2012.git"
SYNC_USER="oleg"
SYNC_UMASK="022"
/etc/portage/repos.conf - Replacement repos.conf file
[DEFAULT]
sync-user = oleg
sync-umask = 022

[gentoo]
location = /usr/portage
sync-type = git
sync-uri = git://github.com/funtoo/ports-2012.git

[foo]
location = /root/git/foo
priority = 1

[bar]
location = /root/git/bar
priority = 2

Notice a [foo] and [bar]. This is a repository name. Let's take a look in /root/git/foo/profiles/repo_name.

# cat /root/git/foo/profiles/repo_name
foo
# cat /root/git/bar/profiles/repo_name
bar

The repos.conf configuration can be further extended with sync-type and sync-uri for overlays to get emerge --sync updating them automatically.

Let's see a real example of tree and overlays added.

/etc/portage/repos.conf - Replacement repos.conf file
[gentoo]
location = /usr/portage
sync-type = git
sync-uri = git://github.com/funtoo/ports-2012.git
 
[funtoo-overlay]
location = /root/git/funtoo-overlay
 
[funtoo-gnome]
location = /root/git/funtoo-gnome-overlay

funtoo-overlay and funtoo-gnome-overlay are an overlays added on top of regular portage tree. How to migrate? Portage ebuild is made to perform unattended migration. /etc/portage/repos.conf/gentoo.conf automatically created. Also, it trying to detect existing overlays and change repos.conf accordingly and ask users to run dispatch-conf or etc-update for configuration file update.