Difference between pages "GlusterFS" and "Repository Configuration"

(Difference between pages)
 
(Undo revision 9202 by Duncan.britton (talk))
 
Line 1: Line 1:
{{Article
+
{{Warning|This article is a work-in-progress referring to a future Portage version. It does not apply to the current Funtoo Portage version. Please do not update your configuration yet.}}
|Author=Drobbins
+
}}
+
== GlusterFS Distribution ==
+
  
Below, we create a distributed volume using two bricks (XFS filesystems.) This spreads IO and files among two bricks.
+
Starting with Portage-2.3.8, a switch to a new repository configuration framework is complete and users may want to update their configuration files. This document aims to describe the goals for the new framework and how to use it.
  
<console>
+
== Multiple repository layout ==
# ##i##gluster peer status
+
One of the most important changes is the switch from the old ''overlay'' layout to a new cleaner ''repository'' system. The new layout is more flexible and more predictable. For example, repositories can now use resources (eclasses, for example) provided by other repositories.
No peers present
+
# ##i##gluster peer probe rhs-lab2
+
Probe successful
+
# ##i##gluster peer status
+
Number of Peers: 1
+
  
Hostname: rhs-lab2
+
The old layout was based on the concept of one ''main tree'' and optionally a number of overlays. The main tree provided base system ebuilds, eclasses, profiles, while overlays mostly were able to provide their own ebuilds. The ebuild provided by overlays overrode the ebuilds in main tree to the extend of making it impossible to install the main tree version. Overlays could also provide eclasses for their own ebuilds and package.* entries that applied to all overlays and to the main tree. The Package Manager is responsible for updating the main tree, while overlays are managed externally.
Uuid: 6b6c9ffc-da79-4d24-8325-086d44869338
+
State: Peer in Cluster (Connected)
+
# ##i##gluster peer probe rhs-lab3
+
Probe successful
+
# ##i##gluster peer probe rhs-lab4
+
Probe successful
+
# ##i##gluster peer status
+
Number of Peers: 3
+
  
Hostname: rhs-lab2
+
The new layout is based on the concept of one or more configurable repositories. Each repository can either be stand-alone or depend upon other repositories. The distribution provides a repository called ''funtoo'' (a drop-in replacement for Gentoo's ''gentoo'' repository). Users can install more repositories at they will, the repositories providing their own ebuilds, eclasses and profiles as necessary and/or using them from other repositories. Users can explicitly choose the repository they want to install packages from. The Package Manager can update all repositories.
Uuid: 6b6c9ffc-da79-4d24-8325-086d44869338
+
State: Peer in Cluster (Connected)
+
  
Hostname: rhs-lab3
+
== Portage configuration ==
Uuid: cbcd508e-5f80-4224-91df-fd5f8e12915d
+
=== New repository layout ===
State: Peer in Cluster (Connected)
+
The repository configuration should be stored in <code>/etc/portage/repos.conf</code>. It can be either a single file or a directory containing one or more ''.conf'' files.
  
Hostname: rhs-lab4
+
The default configuration is installed as <code>/usr/share/portage/config/repos.conf</code>. This file is internal configuration file installed with portage ebuild and should '''not''' be modified. Instead, the configuration in <code>/etc/portage/repos.conf</code> can override the defaults specified there.
Uuid: a02f68d8-88af-4b79-92d8-1057dd85af45
+
State: Peer in Cluster (Connected)
+
# ##i##gluster volume create dist rhs-lab1:/data/dist rhs-lab2:/data/dist
+
Creation of volume dist has been successful. Please start the volume to access data.
+
</console>
+
  
<console>
+
The configuration uses format similar to Windows .ini files. Each section heading (repository name in square brackets) signifies a single repository, followed by one or more key-value option pairs. For example, the following file copies default configuration for Funtoo repository:
# ##i##gluster volume info
+
+
Volume Name: dist
+
Type: Distribute
+
Volume ID: f9758871-20dc-4728-9576-a5bb5b24ca4f
+
Status: Created
+
Number of Bricks: 2
+
Transport-type: tcp
+
Bricks:
+
Brick1: rhs-lab1:/data/dist
+
Brick2: rhs-lab2:/data/dist
+
</console>
+
  
<console>
+
{{file|name=/etc/portage/repos.conf/funtoo.conf|desc=Example configuration override for Funtoo repository to move it to non-standard location|body=
# ##i##gluster volume start dist
+
[funtoo]
Starting volume dist has been successful
+
# moved to non-standard location!
</console>
+
location = /var/db/repos/funtoo
 +
sync-type = git
 +
sync-uri = git://github.com/funtoo/ports-2015.git
 +
auto-sync = yes
 +
}}
  
<console>
+
The most useful repository configuration options are listed below:
# ##i##gluster volume info
+
;location: ''Obligatory.'' Specifies the directory where repository is/will be stored. If Portage knows how to sync the repository and the location does not exist, it will be created on next ''emerge --sync''. Otherwise, the directory must exist.
+
;priority: Specifies the priority used for ordering ebuilds from different repositories. If two repositories provide an ebuild with matching versions, the repository with higher priority will be used.
Volume Name: dist
+
;auto-sync: Specifies whether ''emerge --sync'' should update the repository. Defaults to ''yes'' if ''sync-type'' is specified, ''no'' otherwise.
Type: Distribute
+
;sync-depth: Specifies ''--depth'' for git clone. Used only on initial sync. Defaults to 1. Can be set to 0 to force full clone (not pass ''--depth'' at all).
Volume ID: f9758871-20dc-4728-9576-a5bb5b24ca4f
+
;sync-type: Specifies syncing/update method. Can be one of: ''cvs'', ''git'', ''rsync'', ''svn''.
Status: Started
+
;sync-umask: Specifies the umask used when updating/syncing the repository.
Number of Bricks: 2
+
;sync-uri: Specifies remote URI from which the repository will be cloned/synced. Can use any syntax valid for a particular syncing method.
Transport-type: tcp
+
;sync-user: Specifies the user[:group] used to update/sync the repository. If ''FEATURES=usersync'' is used, defaults to the credentials of directory owner.
Bricks:
+
Brick1: rhs-lab1:/data/dist
+
Brick2: rhs-lab2:/data/dist
+
</console>
+
  
<console>
+
Additionally a <code>[DEFAULT]</code> section may be specified. Options in this section are used as defaults for all repositories.
# ##i##mount -t glusterfs rhs-lab1:/dist /mnt/dist
+
</console>
+
  
== GlusterFS Mirroring ==
+
=== Migrating existing configurations ===
 
+
The new configuration format provides replacement for existing configuration done through <code>/etc/portage/make.conf</code> and environment variables. While the variables are still supported for backwards compatibility, users are recommended to move to the new configuration scheme. Funtoo portage ebuild is planned to  make the migration unattended (repos.conf installed automatically to ease the config steps) with the following file:
Below, we mirror data between two bricks (XFS volumes). This creates a redundant system and also allows for read performance to be improved.
+
 
+
<console>
+
# ##i##gluster volume create mirror replica 2 rhs-lab1:/data/mirror rhs-lab2:/data/mirror
+
Creation of volume mirror has been successful. Please start the volume to access data.
+
# ##i##gluster volume start mirror
+
Starting volume mirror has been successful
+
# ##i##gluster volume info mirror
+
 
   
 
   
Volume Name: mirror
+
{{file|name=/etc/portage/repos.conf/funtoo.conf|body=
Type: Replicate
+
[funtoo]
Volume ID: 4edacef8-982c-46a9-be7e-29e34fa40f95
+
location = /usr/portage
Status: Started
+
sync-type = git
Number of Bricks: 1 x 2 = 2
+
sync-uri = git://github.com/funtoo/ports-2015.git
Transport-type: tcp
+
auto-sync = yes
Bricks:
+
}}
Brick1: rhs-lab1:/data/mirror
+
The following replacements are provided for existing variables:
Brick2: rhs-lab2:/data/mirror
+
;PORTDIR: Used to specify main tree location. Replaced by ''location'' key in the section corresponding to the default repository (<code>[funtoo]</code> by default).
# ##i##install -d /mnt/mirror
+
;PORTDIR_OVERLAY: Used to specify locations of overlays. Each of the paths needs to be replaced with a separate repository section, with the path placed in ''location'' key. Additionally, ''priority'' may be used to force specific ordering of ebuild overrides.
# ##i##mount -t glusterfs rhs-lab1:/mirror /mnt/mirror
+
;SYNC: Used to specify URI for syncing the main repository, also implied a protocol for doing that. Replaced by the ''sync-uri'' and ''sync-type'' keys in the default repository section.
 +
;SYNC_UMASK: Used to specify umask for syncing repositories. Replaced by ''sync-umask'' key in repository configuration. Can be specified in <code>[DEFAULT]</code> section to apply to all repositories.
 +
;SYNC_USER: Used to specify user credentials for syncing repositories. Replaced by ''sync-user'' key in repository configuration. Can be specified in <code>[DEFAULT]</code> section to apply to all repositories.
  
</console>
+
{{file|name=/etc/portage/make.conf|desc=Example old make.conf file|body=
 +
# user changed PORTDIR location
 +
PORTDIR="/var/db/repos/funtoo"
 +
PORTDIR_OVERLAY="/var/db/repos/foo /var/db/repos/bar"
  
== Growing GlusterFS ==
+
SYNC="git://github.com/funtoo/ports-2015.git"
 +
SYNC_USER="oleg"
 +
SYNC_UMASK="022"
 +
}}
  
Now we will add a new brick to our distributed filesystem. We will run a rebalance (optional) to get the files distributed ideally. This will involve distributing some existing files on to our new brick on rhs-lab3:
+
{{file|name=/etc/portage/repos.conf|desc=Replacement repos.conf file|body=
 +
[DEFAULT]
 +
sync-user = oleg
 +
sync-umask = 022
  
<console>
+
[funtoo]
# ##i##gluster volume add-brick dist rhs-lab3:/data/dist
+
location = /var/db/repos/funtoo
Add Brick successful
+
sync-type = git
# ##i##gluster volume rebalance dist start
+
sync-uri = git://github.com/funtoo/ports-2015.git
Starting rebalance on volume dist has been successful
+
</console>
+
  
After the rebalance, our distributed GlusterFS filesystem will have optimal performance and one third of the files will have moved to rhs-lab3.
+
[foo]
 +
location = /var/db/repos/foo
 +
priority = 1
  
<console>
+
[bar]
# ##i##gluster volume rebalance dist status
+
location = /var/db/repos/bar
                                    Node Rebalanced-files          size      scanned      failures        status
+
priority = 2
                              ---------      -----------  -----------  -----------  -----------  ------------
+
}}
                              localhost                0            0            0            0      completed
+
                                rhs-lab4                0            0            0            0      completed
+
                                rhs-lab3                0            0            0            0      completed
+
                                rhs-lab2                0            0            0            0      completed
+
  
</console>
+
The <code>repos.conf</code> configuration can be further extended with ''sync-type'' and ''sync-uri'' for overlays to get ''emerge --sync'' updating them automatically.
  
== Growing a GlusterFS Replicated Volume ==
+
let's see a real example of tree and overlays added.
 +
{{file|name=/etc/portage/repos.conf|desc=Replacement repos.conf file|body=
  
You can grow a replicated volume by adding pairs of bricks:
+
[gentoo]
 
+
location = /usr/portage
<console>
+
sync-type = git
# ##i##gluster volume add-brick mirror rhs-lab3:/data/mirror rhs-lab4:/data/mirror
+
sync-uri = git://github.com/funtoo/ports-2012.git
Add Brick successful
+
# ##i##gluster volume info mirror
+
 
   
 
   
Volume Name: mirror
+
[funtoo-overlay]
Type: Distributed-Replicate
+
location = /root/git/funtoo-overlay
Volume ID: 4edacef8-982c-46a9-be7e-29e34fa40f95
+
Status: Started
+
Number of Bricks: 2 x 2 = 4
+
Transport-type: tcp
+
Bricks:
+
Brick1: rhs-lab1:/data/mirror
+
Brick2: rhs-lab2:/data/mirror
+
Brick3: rhs-lab3:/data/mirror
+
Brick4: rhs-lab4:/data/mirror
+
</console>
+
 
+
== GlusterFS Brick Migration ==
+
 
+
Here is how you migrate data off of an existing brick and on to a new brick:
+
 
+
<console>
+
# ##i##gluster volume replace-brick dist rhs-lab3:/data/dist rhs-lab4:/data/dist start
+
replace-brick started successfully
+
# ##i##gluster volume replace-brick dist rhs-lab3:/data/dist rhs-lab4:/data/dist status
+
Number of files migrated = 0        Migration complete
+
# ##i##gluster volume replace-brick dist rhs-lab3:/data/dist rhs-lab4:/data/dist commit
+
replace-brick commit successful
+
# ##i##gluster volume info
+
 
   
 
   
Volume Name: dist
+
[funtoo-gnome]
Type: Distribute
+
location = /root/git/funtoo-gnome-overlay
Volume ID: f9758871-20dc-4728-9576-a5bb5b24ca4f
+
}}
Status: Started
+
funtoo-overlay and funtoo-gnome-overlay are an overlays added on top of regular portage tree.
Number of Bricks: 3
+
[[Category:Portage]]
Transport-type: tcp
+
Bricks:
+
Brick1: rhs-lab1:/data/dist
+
Brick2: rhs-lab2:/data/dist
+
Brick3: rhs-lab4:/data/dist
+
+
Volume Name: mirror
+
Type: Distributed-Replicate
+
Volume ID: 4edacef8-982c-46a9-be7e-29e34fa40f95
+
Status: Started
+
Number of Bricks: 2 x 2 = 4
+
Transport-type: tcp
+
Bricks:
+
Brick1: rhs-lab1:/data/mirror
+
Brick2: rhs-lab2:/data/mirror
+
Brick3: rhs-lab3:/data/mirror
+
Brick4: rhs-lab4:/data/mirror
+
</console>
+
 
+
== Removing a Brick ==
+
 
+
Here's how you remove a brick. The add-brick and remove-brick commands will ensure that you don't break mirrors, so you will need to remove both volumes in a mirror if you are working with a replicated volume.
+
 
+
<console>
+
# ##i##gluster volume remove-brick dist rhs-lab4:/data/dist start
+
Remove Brick start successful
+
# ##i##gluster volume remove-brick dist rhs-lab4:/data/dist status
+
                                    Node Rebalanced-files          size      scanned      failures        status
+
                              ---------      -----------  -----------  -----------  -----------  ------------
+
                              localhost                0            0            0            0    not started
+
                                rhs-lab3                0            0            0            0    not started
+
                                rhs-lab2                0            0            0            0    not started
+
                                rhs-lab4                0            0            0            0      completed
+
 
+
# ##i##gluster volume remove-brick dist rhs-lab4:/data/dist commit
+
Removing brick(s) can result in data loss. Do you want to Continue? (y/n) y
+
Remove Brick commit successful
+
 
+
</console>
+
 
+
== Georeplication ==
+
 
+
At the local GlusterFS site:
+
 
+
<console>
+
# gluster volume create georep rhs-lab1:/data/georep
+
Creation of volume georep has been successful. Please start the volume to access data.
+
# gluster volume start georep
+
Starting volume georep has been successful
+
# gluster volume info georep
+
+
Volume Name: georep
+
Type: Distribute
+
Volume ID: 001bc914-74ad-48e6-846a-1767a5b2cb58
+
Status: Started
+
Number of Bricks: 1
+
Transport-type: tcp
+
Bricks:
+
Brick1: rhs-lab1:/data/georep
+
# mkdir /mnt/georep
+
# mount -t glusterfs rhs-lab1:/georep /mnt/georep
+
# cd /mnt/georep/
+
# ls
+
# df -h .
+
Filesystem            Size  Used Avail Use% Mounted on
+
rhs-lab1:/georep      5.1G  33M  5.0G  1% /mnt/georep
+
</console>
+
 
+
At the remote site, set up a <tt>georep-dr</tt> volume:
+
 
+
<console>
+
# ##i##gluster volume create georep-dr rhs-lab4:/data/georep-dr
+
# ##i##gluster volume start georep-dr
+
</console>
+
 
+
Local side:
+
 
+
<console>
+
# ##i##gluster volume geo-replication georep status
+
MASTER              SLAVE                                              STATUS   
+
--------------------------------------------------------------------------------
+
# ##i##gluster volume geo-replication georep ssh://rhs-lab4::georep-dr start
+
Starting geo-replication session between georep & ssh://rhs-lab4::georep-dr has been successful
+
</console>
+
 
+
== GlusterFS Security ==
+
 
+
Currently, any GlusterFS peer can join your volume if it exists on your LAN. Securing GlusterFS can be accomplished with <tt>iptables</tt> by blocking TCP ports.
+
 
+
[[Category:Filesystems]]
+
[[Category:Articles]]
+
{{PageNeedsUpdates}}
+
{{ArticleFooter}}
+

Revision as of 04:52, February 27, 2015

Warning

This article is a work-in-progress referring to a future Portage version. It does not apply to the current Funtoo Portage version. Please do not update your configuration yet.

Starting with Portage-2.3.8, a switch to a new repository configuration framework is complete and users may want to update their configuration files. This document aims to describe the goals for the new framework and how to use it.

Multiple repository layout

One of the most important changes is the switch from the old overlay layout to a new cleaner repository system. The new layout is more flexible and more predictable. For example, repositories can now use resources (eclasses, for example) provided by other repositories.

The old layout was based on the concept of one main tree and optionally a number of overlays. The main tree provided base system ebuilds, eclasses, profiles, while overlays mostly were able to provide their own ebuilds. The ebuild provided by overlays overrode the ebuilds in main tree to the extend of making it impossible to install the main tree version. Overlays could also provide eclasses for their own ebuilds and package.* entries that applied to all overlays and to the main tree. The Package Manager is responsible for updating the main tree, while overlays are managed externally.

The new layout is based on the concept of one or more configurable repositories. Each repository can either be stand-alone or depend upon other repositories. The distribution provides a repository called funtoo (a drop-in replacement for Gentoo's gentoo repository). Users can install more repositories at they will, the repositories providing their own ebuilds, eclasses and profiles as necessary and/or using them from other repositories. Users can explicitly choose the repository they want to install packages from. The Package Manager can update all repositories.

Portage configuration

New repository layout

The repository configuration should be stored in /etc/portage/repos.conf. It can be either a single file or a directory containing one or more .conf files.

The default configuration is installed as /usr/share/portage/config/repos.conf. This file is internal configuration file installed with portage ebuild and should not be modified. Instead, the configuration in /etc/portage/repos.conf can override the defaults specified there.

The configuration uses format similar to Windows .ini files. Each section heading (repository name in square brackets) signifies a single repository, followed by one or more key-value option pairs. For example, the following file copies default configuration for Funtoo repository:

/etc/portage/repos.conf/funtoo.conf - Example configuration override for Funtoo repository to move it to non-standard location
[funtoo]
# moved to non-standard location!
location = /var/db/repos/funtoo
sync-type = git
sync-uri = git://github.com/funtoo/ports-2015.git
auto-sync = yes

The most useful repository configuration options are listed below:

location
Obligatory. Specifies the directory where repository is/will be stored. If Portage knows how to sync the repository and the location does not exist, it will be created on next emerge --sync. Otherwise, the directory must exist.
priority
Specifies the priority used for ordering ebuilds from different repositories. If two repositories provide an ebuild with matching versions, the repository with higher priority will be used.
auto-sync
Specifies whether emerge --sync should update the repository. Defaults to yes if sync-type is specified, no otherwise.
sync-depth
Specifies --depth for git clone. Used only on initial sync. Defaults to 1. Can be set to 0 to force full clone (not pass --depth at all).
sync-type
Specifies syncing/update method. Can be one of: cvs, git, rsync, svn.
sync-umask
Specifies the umask used when updating/syncing the repository.
sync-uri
Specifies remote URI from which the repository will be cloned/synced. Can use any syntax valid for a particular syncing method.
sync-user
Specifies the user[:group] used to update/sync the repository. If FEATURES=usersync is used, defaults to the credentials of directory owner.

Additionally a [DEFAULT] section may be specified. Options in this section are used as defaults for all repositories.

Migrating existing configurations

The new configuration format provides replacement for existing configuration done through /etc/portage/make.conf and environment variables. While the variables are still supported for backwards compatibility, users are recommended to move to the new configuration scheme. Funtoo portage ebuild is planned to make the migration unattended (repos.conf installed automatically to ease the config steps) with the following file:

/etc/portage/repos.conf/funtoo.conf
[funtoo]
location = /usr/portage
sync-type = git
sync-uri = git://github.com/funtoo/ports-2015.git
auto-sync = yes

The following replacements are provided for existing variables:

PORTDIR
Used to specify main tree location. Replaced by location key in the section corresponding to the default repository ([funtoo] by default).
PORTDIR_OVERLAY
Used to specify locations of overlays. Each of the paths needs to be replaced with a separate repository section, with the path placed in location key. Additionally, priority may be used to force specific ordering of ebuild overrides.
SYNC
Used to specify URI for syncing the main repository, also implied a protocol for doing that. Replaced by the sync-uri and sync-type keys in the default repository section.
SYNC_UMASK
Used to specify umask for syncing repositories. Replaced by sync-umask key in repository configuration. Can be specified in [DEFAULT] section to apply to all repositories.
SYNC_USER
Used to specify user credentials for syncing repositories. Replaced by sync-user key in repository configuration. Can be specified in [DEFAULT] section to apply to all repositories.
/etc/portage/make.conf - Example old make.conf file
# user changed PORTDIR location
PORTDIR="/var/db/repos/funtoo"
PORTDIR_OVERLAY="/var/db/repos/foo /var/db/repos/bar"

SYNC="git://github.com/funtoo/ports-2015.git"
SYNC_USER="oleg"
SYNC_UMASK="022"
/etc/portage/repos.conf - Replacement repos.conf file
[DEFAULT]
sync-user = oleg
sync-umask = 022

[funtoo]
location = /var/db/repos/funtoo
sync-type = git
sync-uri = git://github.com/funtoo/ports-2015.git

[foo]
location = /var/db/repos/foo
priority = 1

[bar]
location = /var/db/repos/bar
priority = 2

The repos.conf configuration can be further extended with sync-type and sync-uri for overlays to get emerge --sync updating them automatically.

let's see a real example of tree and overlays added.

/etc/portage/repos.conf - Replacement repos.conf file
[gentoo]
location = /usr/portage
sync-type = git
sync-uri = git://github.com/funtoo/ports-2012.git
 
[funtoo-overlay]
location = /root/git/funtoo-overlay
 
[funtoo-gnome]
location = /root/git/funtoo-gnome-overlay

funtoo-overlay and funtoo-gnome-overlay are an overlays added on top of regular portage tree.