Difference between revisions of "Portage Git Mirror"

From Funtoo
Jump to: navigation, search
(Cloning from the local git mirror)
 
(18 intermediate revisions by 7 users not shown)
Line 5: Line 5:
 
== Use case ==
 
== Use case ==
  
This tutorial will be about hosting a local mirror of funtoo git based portage tree.
+
The more local funtoo instances you are maintaining, the more useful this will be to you. This -- along with a local [[Portage_Distfiles_Mirror]] -- can make maintaining multiple funtoo instances much faster, much more bandwidth efficient, and much less burdensome to upstream resources.
 +
 
 +
===Terminology used herein===
  
 
Following terms should be adapted
 
Following terms should be adapted
  
<table border="1">
+
{{TableStart}}
 
<tr class="header">
 
<tr class="header">
 
<th align="left">Terms</th>
 
<th align="left">Terms</th>
Line 30: Line 32:
 
<td align="left">Base path of git-daemon</td>
 
<td align="left">Base path of git-daemon</td>
 
</tr>
 
</tr>
</table>
+
{{TableEnd}}
  
 
== Local mirror ==
 
== Local mirror ==
  
=== Git-daemon configuration ===
+
=== Prepare directories and get portage tree ===
 
+
{{console|body=
Prepare directories and clone portage-mini from github::
 
<console>
 
 
###i## mkdir /home/git-mirrors
 
###i## mkdir /home/git-mirrors
 
###i## chown nobody /home/git-mirrors
 
###i## chown nobody /home/git-mirrors
 
###i## su -s /bin/sh nobody
 
###i## su -s /bin/sh nobody
 
$##bl## cd /home/git-mirrors
 
$##bl## cd /home/git-mirrors
$##bl## git clone --mirror --bare git://github.com/funtoo/portage-2012.git portage.git
+
$##bl## git clone --mirror --bare git://github.com/funtoo/ports-2012.git portage.git
</console>
+
}}
 +
For security purposes we will use the {{c|nobody}} user .
  
Configure git-daemon in <code>/etc/conf.d/git-daemon</code>:
+
=== git-daemon configuration ===
<console>
+
{{file|name=/etc/conf.d/git-daemon|desc=|body=
###i## nano /etc/conf.d/git-daemon
+
GITDAEMON_OPTS="--syslog --verbose --enable=receive-pack --export-all"
GITDAEMON_OPTS=&quot;--syslog --verbose --enable=receive-pack --export-all&quot;
+
GITDAEMON_OPTS="${GITDAEMON_OPTS} --base-path=/home/git-mirrors /home/git-mirrors --interpolated-path=/home/git-mirrors"
GITDAEMON_OPTS=&quot;${GITDAEMON_OPTS} --base-path=/home/git-mirrors /home/git-mirrors&quot;
+
GIT_USER=nobody
GIT_USER=&quot;nobody&quot;
+
GIT_GROUP=nobody
GIT_GROUP=&quot;nobody&quot;
+
}}
</console>
 
  
Start the daemon:
+
=== Service configuration ===
<console>
+
To start daemon with a mirror machine boot add {{c|git-daemon}} to default runlevel
root@git.lan###i## /etc/init.d/git-daemon start
+
{{console|body=
root@git.lan###i## rc-update add git-daemon default
+
###i## rc-update add git-daemon default
</console>
+
}}
 +
To make changes start immediately just run <code>rc</code>
 +
{{console|body=
 +
###i## rc
 +
}}
  
 
=== Pull from remote ===
 
=== Pull from remote ===
  
Add the following to <code>/etc/cron.daily/funtoo-sync.sh</code>:
+
If desired, the local mirror can be updated periodically using {{c|cron}}. This example updates daily:
<console>
+
 
###i## nano /etc/cron.daily/funtoo-sync.sh  
+
{{file|name=/etc/cron.daily/funtoo-sync.sh|desc=|body=
 
#!/bin/sh
 
#!/bin/sh
 
cd /home/git-mirrors/portage.git
 
cd /home/git-mirrors/portage.git
su nobody -s &quot;/bin/sh&quot; -c &quot;git fetch --all&quot;
+
su nobody -s "/bin/sh" -c "git fetch"
</console>
+
}}
 +
 
 +
Then make the file executable:
 +
{{console|body=
 +
###i## chmod +x /etc/cron.daily/funtoo-sync.sh
 +
}}
  
== Cloning from local git-daemon ==
+
You can also update the local mirror using {{c|emerge --sync}} or {{c|eix-sync}} if you have {{c|app-portage/eix}} installed.
  
Local clone from <code>git.lan</code>:
+
= Cloning from the local git mirror =
<console>
+
 
###i## mv /usr/portage /usr/portage.old
+
In order to get a funtoo instance to sync with the local git mirror, the instance's portage directory will need some work.
###i## git clone git://git.lan/portage.git /usr/portage
+
 
 +
The following is used with any funtoo instance you wish to use your local git mirror for its tree. This will change the git remote origin to point to your local {{c|git.lan}} origin, and then start using it.  (This is an updated method which obviates the need to create a {{c|/usr/portage.old}} copy of the existing {{C|/usr/portage}} directory.)
 +
 
 +
Local clone from {{c|git.lan}}:
 +
{{console|body=
 
###i## cd /usr/portage
 
###i## cd /usr/portage
###i## git checkout funtoo.org
+
###i## git clone --depth 1 git://git.lan/portage.git tmp
</console>
+
###i## rm -r .git
 +
###i## mv tmp/.git .
 +
###i## git reset --hard
 +
###i## chown -R portage:portage /usr/portage
 +
###i## eix-sync
 +
}}
  
 +
{{note|Use {{c|emerge --sync}} if you do not have {{c|app-portage/eix}} installed.}}
 +
 +
A less reliable method is to leverage git's ability to switch between remote repositories.
 +
It can lead to conflicts and is generally more difficult to deal with if you're unfamiliar with git.
 +
 +
{{note|github.com/funtoo/ports-2012 uses funtoo.org as its main branch. github.com/funtoo/ports-2017 uses master.
 +
Substitute --track funtoo.org with --track master in the below command for the ports-2017 portage tree.
 +
}}
 +
 +
Local clone from {{c|git.lan}}:
 +
{{console|body=
 +
###i## cd /usr/portage
 +
###i## git remote show origin
 +
###i## git remote rm origin
 +
###i## git remote add --track funtoo.org origin git://git.lan/portage.git
 +
###i## git remote show origin
 +
###i## git branch --set-upstream-to=origin/master master
 +
###i## emerge --sync
 +
}}
 
[[Category:HOWTO]]
 
[[Category:HOWTO]]

Latest revision as of 10:46, April 9, 2017

Setting up local git mirror

This tutorial explains how to save bandwidth when several local computers need to pull updates from a single remote git repository.

Use case

The more local funtoo instances you are maintaining, the more useful this will be to you. This -- along with a local Portage_Distfiles_Mirror -- can make maintaining multiple funtoo instances much faster, much more bandwidth efficient, and much less burdensome to upstream resources.

Terminology used herein

Following terms should be adapted

Terms Definition
git.lan The git-daemon local mirror host
localhost Any local host
nobody Owner user of .git files
/home/git-mirrors Base path of git-daemon

Local mirror

Prepare directories and get portage tree

root # mkdir /home/git-mirrors
root # chown nobody /home/git-mirrors
root # su -s /bin/sh nobody
user $ cd /home/git-mirrors
user $ git clone --mirror --bare git://github.com/funtoo/ports-2012.git portage.git

For security purposes we will use the nobody user .

git-daemon configuration

   /etc/conf.d/git-daemon
GITDAEMON_OPTS="--syslog --verbose --enable=receive-pack --export-all"
GITDAEMON_OPTS="${GITDAEMON_OPTS} --base-path=/home/git-mirrors /home/git-mirrors --interpolated-path=/home/git-mirrors"
GIT_USER=nobody
GIT_GROUP=nobody

Service configuration

To start daemon with a mirror machine boot add git-daemon to default runlevel

root # rc-update add git-daemon default

To make changes start immediately just run rc

root # rc

Pull from remote

If desired, the local mirror can be updated periodically using cron. This example updates daily:

   /etc/cron.daily/funtoo-sync.sh
#!/bin/sh
cd /home/git-mirrors/portage.git
su nobody -s "/bin/sh" -c "git fetch"

Then make the file executable:

root # chmod +x /etc/cron.daily/funtoo-sync.sh

You can also update the local mirror using emerge --sync or eix-sync if you have app-portage/eix installed.

Cloning from the local git mirror

In order to get a funtoo instance to sync with the local git mirror, the instance's portage directory will need some work.

The following is used with any funtoo instance you wish to use your local git mirror for its tree. This will change the git remote origin to point to your local git.lan origin, and then start using it. (This is an updated method which obviates the need to create a /usr/portage.old copy of the existing /usr/portage directory.)

Local clone from git.lan:

root # cd /usr/portage
root # git clone --depth 1 git://git.lan/portage.git tmp
root # rm -r .git
root # mv tmp/.git .
root # git reset --hard
root # chown -R portage:portage /usr/portage
root # eix-sync
   Note

Use emerge --sync if you do not have app-portage/eix installed.

A less reliable method is to leverage git's ability to switch between remote repositories. It can lead to conflicts and is generally more difficult to deal with if you're unfamiliar with git.

   Note

github.com/funtoo/ports-2012 uses funtoo.org as its main branch. github.com/funtoo/ports-2017 uses master. Substitute --track funtoo.org with --track master in the below command for the ports-2017 portage tree.

Local clone from git.lan:

root # cd /usr/portage
root # git remote show origin
root # git remote rm origin
root # git remote add --track funtoo.org origin git://git.lan/portage.git
root # git remote show origin
root # git branch --set-upstream-to=origin/master master
root # emerge --sync