Difference between revisions of "How to set up a binary package server"

Latest revision as of 05:45, 30 May 2013

[edit] Why would you want a binary package server?

Even though Funtoo is a source-based distribution, it does not mean that compiling from source every single time is beneficial. For instance, if you have multiple computers in your home network, and they all need upgrades, it wouldn't be convenient to compile the same package over and over again on each machine. Rather you could make your fastest computer compile the packages, and then just share those packages over the network or via USB drive (sneakernet).

[edit] Recommendations

In order to make the most use of this, I recommend the following:

1. Set all USE flags on all machines to be the same (Lowest Common Denominator)
2. Compile for a generic subarch. (For example, if all your machines are amd64, compile for amd64 not no specific ABIs like core2, etc.)

[edit] Configuring the make.conf for host machine

We first need to enable automatic binary packaging on the host. This means that when portage compiles the package, it will put it into a .tbz2 file right after. To do this we will add buildpkg to the FEATURES var in /etc/make.conf.

My make.conf looks like this::

CFLAGS="-mtune=generic -O2 -pipe"
CXXFLAGS="${CFLAGS}"
USE="mmx sse sse2"
MAKEOPTS="-j9"
FEATURES="buildpkg userfetch"
EMERGE_DEFAULT_OPTS="--quiet-build=y"
VIDEO_CARDS="nvidia nouveau intel"
INPUT_DEVICES="evdev synaptics"
LINGUAS="en en_US"

Now that FEATURES="buildpkg" is set, from now on, every time you build a new package, it will automatically package it and you can find the .tbz2 file in /usr/portage/packages by default. You can change the destination by using the PKGDIR setting in make.conf.

[edit] What will you need?

Besides the above advice, you will need one of the protocols that can be used for sharing (Like FTP, HTTP, or ssh).I recommend HTTP, so we will be using lighttpd server since it's small, quick, and efficient for these basic tasks.

[edit] Setting up the host machine

Install lighttpd

# emerge -av lighttpd

After you install lighttpd, we will set a symlink in the htdocs folder to point to your packages folder (By default it's /usr/portage/packages):

# cd /var/www/localhost/htdocs
# mkdir funtoo && cd funtoo
# ln -s /usr/portage portage
# ln -s /usr/portage/distfiles distfiles
# ln -s /usr/portage/packages packages

Now that that's set, the link to your packages will be http://ip-of-computer/funtoo/packages.

If the lighttpd server were on now and you navigated to the above address, you would get a 404 error because lighttpd by default doesn't show the directory listing. Add the following line to your /etc/lighttpd/lighttpd.conf so that if you request a page that doesn't contain an index page, it will just show the directory contents:

# Activate directory listings globally
dir-listing.activate = "enable"

Add the service to your start up process and start it up:

# rc-config add lighttpd default
# rc

Your host machine is now ready to serve out binaries.

[edit] Setting up the client machine

Basically the client machine will be receiving packages from a server that might not necessarily have the exact same USE flags that the client machine has. Even thought we set all the USE flags on all machines to the Lowest Common Denominator, sometimes things deviate. In order to solve this problem, we will make portage not respect USE flags. Meaning that if there are different USE flags on machines, it will still install those packages.

To do this we need to add the --binpkg-respect-use=n to the end of emerge.

Before we use the above option, we also need to tell emerge to not only search for ebuilds, but also to search for packages as well. In order for it to search for packages, emerge will need to know where to look (the host) and set an option on portage to actually let it know to look for binaries.

We do this by adding the --getbinpkg option to the end of emerge, and setting the PACKAGE_BINHOST variable in /etc/make.conf.

This is how my client's /etc/make.conf looks:

CFLAGS="-mtune=generic -O2 -pipe"
CXXFLAGS="${CFLAGS}"
USE="mmx sse sse2"
MAKEOPTS="-j5"
FEATURES="buildpkg userfetch getbinpkg"
EMERGE_DEFAULT_OPTS="--quiet-build=y --binpkg-respect-use=n"
VIDEO_CARDS="nvidia intel"
INPUT_DEVICES="evdev synaptics"
LINGUAS="en en_US"
PORTAGE_BINHOST="http://ip-address-of-build-server/gentoo/packages"

After you set the package host (via PORTAGE_BINHOST), set the getbinpkg option in FEATURES, and add --binpkg-respect-use=n to EMERGE_DEFAULT_OPTS, you are then set. Just do a emerge --sync, and emerge -uDav world or whatever you want. You should now see all your packages being pulled from the host server.

Normally you could just do a

# emerge -av <package> --binpkg-respect-use=n

But you will have to do that each time you want to install a binary with different use flags, so in order to automate this, we add it to the EMERGE_DEFAULT_OPTS.