Difference between revisions of "How to set up a binary package server"

Revision as of 03:28, 16 July 2012

Why would you want a binary package server?

Even though Funtoo is a source based distribution, it does not mean that compiling from source every single time is beneficial. For instance, if you have multiple computers in your home network, and they all need upgrades, it wouldn't be convenient to compile the same package over and over again on each machine. Rather you could make your fastest computer compile the packages, and then just share those packages over the network or via usb drive (sneakernet).

Recommendations

In order to make the most use of this, I recommend the following:

1. Set all USE flags on all machines to be the same (Lowest Common Denominator) 2. Compile for a generic type. (If all your machines are amd64, compile for amd64 not no specific ABIs like core2 etc)

Configuring the make.conf for host machine

We first need to enable automatic binary packaging on the host. This means that when portage compiles the package, it will put it into a .tbz2 file right after. To do this we will add the "buildpkg" to the FEATURES var in /etc/make.conf

My make.conf looks like this (on Gentoo, it's similar on Funtoo as well, just that the mirrors change):

    # These settings were set by the catalyst build script that automatically
    # built this stage.
    # Please consult /usr/share/portage/config/make.conf.example for a more
    # detailed example.

    ACCEPT_KEYWORDS="~amd64"
    CFLAGS="-mtune=generic -O2 -pipe"
    CXXFLAGS="${CFLAGS}"
    CHOST="x86_64-pc-linux-gnu"

    USE="mmx sse sse2"

    MAKEOPTS="-j9"

    FEATURES="buildpkg userfetch"
 
    EMERGE_DEFAULT_OPTS="--quiet-build=y"

    VIDEO_CARDS="nvidia nouveau intel"

    INPUT_DEVICES="evdev synaptics"

    LINGUAS="en en_US"

Now that FEATURES="buildpkg" is set, from now on, every time you build a new package, it will automatically package it and you can find the .tbz2 file in /usr/portage/packages.

What will you need?

Besides the above advice, you will need one of the protocols that can be used for sharing (Like ftp, http, or ssh).

I recommend http, so we will be using lighttpd server since it's small, quick, and efficient for these basic tasks.

Setting up the host machine

Install lighttpd

    # emerge -av lighttpd

After you install lighttpd, we will set a symlink in the htdocs folder to point to your packages folder (By default it's /usr/portage/packages)

    # cd /var/www/localhost/htdocs
    # mkdir funtoo && cd funtoo
    # ln -s /usr/portage/packages packages

Now that that's set, the link to your packages will be http colon //ip-of-computer/funtoo/packages

Add the service to your start up process and start it up

    # rc-config add lighttpd default
    # rc

Your host machine is now ready to serve out binaries.

Setting up the client machine

Basically the client machine will be receiving packages from a server that might not necessarily have the exact same USE flags that the client machine has. Even thought we set all the USE flags on all machines to the Lowest Common Denominator, sometimes things deviate. In order to solve this problem, we will make portage not respect USE flags. Meaning that if there are different USE flags on machines, it will still install those packages.

To do this we need to add the --binpkg-respect-use=n to the end of emerge.

Before we use the above option, we also need to tell emerge to not only search for ebuilds, but also to search for packages as well. In order for it to search for packages, emerge will need to know where to look (the host) and set an option on portage to actually let it know to look for binaries.

We do this by adding the --getbinpkg option to the end of emerge, and setting the PACKAGE_BINHOST variable in /etc/make.conf

This is how my client's /etc/make.conf looks like.

    # These settings were set by the catalyst build script that automatically
    # built this stage.
    # Please consult /usr/share/portage/config/make.conf.example for a more
    # detailed example.

    ACCEPT_KEYWORDS="~amd64"
    CFLAGS="-mtune=generic -O2 -pipe"
    CXXFLAGS="${CFLAGS}"
    CHOST="x86_64-pc-linux-gnu"

    USE="mmx sse sse2"

    MAKEOPTS="-j5"

    FEATURES="buildpkg userfetch getbinpkg"

    EMERGE_DEFAULT_OPTS="--quiet-build=y --binpkg-respect-use=n"

    VIDEO_CARDS="nvidia intel"

    INPUT_DEVICES="evdev synaptics"

    LINGUAS="en en_US"

    PORTAGE_BINHOST="http colon //ip-address-of-build-server/gentoo/packages"

After you set the package host (PORTAGE_BINHOST), set the "getbinpkg" option in FEATURES, and add "--binpkg-respect-use=n" to EMERGE_DEFAULT_OPTS, you are then set. Just do a emerge --sync, and emerge -uDav world or w/e you want. You should now see all your packages being pulled from the host server.

Normally you could just do a

    # emerge -av <package> --binpkg-respect-use=n

But you will have to do that each time you want to install a binary with different use flags, so in order to automate this, we add it to the EMERGE_DEFAULT_OPTS.