Difference between pages "Metro Quick Start Tutorial/pt-br" and "FLOP:FFmpeg"

< Metro Quick Start Tutorial(Difference between pages)
 
 
Line 1: Line 1:
= Prefácio =
+
{{FLOP
 
+
|Created on=2015/01/31
== Como o Metro Funciona ==
+
|Summary=Funtoo Linux prefers FFmpeg. Some enlightenment about our choice and why we prefer this or could switch to alternative in future.
 
+
|Author=Oleg, Mgorny
Você pode estar se perguntando como o [[Metro]] cria seu primeiro stage tarball. Como você pode ter imaginado, [[Metro]] não pode criar um stage tarball. Para construi um novo stage tarball, [[Metro]] deve utilizar um já existente, antigo stage tarball chamado de uma &quot;seed&quot; stage. Essa semente &quot;semente&quot; stage tipicamente é utilizado com o ''ambiente build'' para a criação do stage que queremos.
+
|Maintainer=Oleg
 
+
|Reference Bug=FL-844
[[Metro]] pode utilizar dois tipos de semente stages. Tradicionalmente, [[Metro]] tem utilizado um stage3 como uma semente stage. Esse stage3 é então utilizado para construir um novo stage1, que em troca é utlilizado para construir um novo stage2, e então um novo stage3. Esse é geralmente o jeito mais confiável de construir [[Gentoo_Linux/pt-br|Gentoo Linux]] ou [[Funtoo Linux]], entao essa é a recomendação recomendada.
+
{{fancyimportant|'''Depois de portar o metro builds para o perfil do Funtoo, os stages do Gentoo não são mais fornecidos'''!}}
+
 
+
== Sementes e Build Isolation ==
+
 
+
Um outro conceito importante a mencionar aqui é algo chamando de ''build isolation''. Por que o [[Metro]] cria um ambiente build isolado, e o ambiente build é explicitamente definido utilizando entidades existentes, tangíveis -- uma semente stage e um portage snapshot -- você obterá resultados consistentes, repetíveis. Em outras palavras, a mesma semente stage, portage snapshot e instrções build gerarão resultados identico, mesmo se você desempenhar o um mês mais tarde em algum workstation de outro pessoa.
+
 
+
== Local Build ==
+
 
+
Digamos que você queira construir um novo tarball stage3 <tt>pentium4</tt>. O módo recomendado de se fazer isso seria apanhar um tarball stage3 <tt>pentium4</tt> existente como sua semente stage3 stage. Será dito ao [[Metro]] utilizar o stage3 <tt>pentium4</tt> existente para construir um novo stage1 para o mesmo <tt>pentium4</tt>. Para esse process, o stage3 para o <tt>pentium4</tt> genérico forneceria o ''ambiente build'' para a criação de nosso novo stage1. Então, o novo stage1 serviria como o ambiente build (build environment) para a criação do novo stage2 <tt>pentium4</tt>. E o novo stage2 para <tt>pentium4</tt> serviria como o ambiente build para a criação do novo stage3 para <tt>pentium4</tt>.
+
 
+
Na terminologia [[Metro]] isso é chamado de um '''local build''', que significa que um stage3 de uma dada arquitetura é utilizada para semear um build novo em folha da mesma arquitetura. Incidentalmente esse será o primeiro exercício que vamos realizar nesse tutorial.
+
 
+
Uma semana depois, você pode querer construir um tarball stage3 para <tt>pentium4</tt> novo em folha. Ao invés de iniciar a partir do stage3 para <tt>pentium4</tt> original de novo, você provavelmente configuraria o [[Metro]] para utilizar o built stage3 para <tt>pentium4</tt> mais recente construído como a semente. [[Metro]] possui funcionalidade built-in para tornar isso fácil, permitindo facilmente encontrar e rastrear a semente stage3 mais recente disponível.
+
 
+
== Build Remoto ==
+
 
+
[[Metro]] pode também desempenhar '''build remoto (remote build)''', onde um stage3 de uma arquitetura diferente, mas compatível binariamente, é utilizado como uma semente para construir um stage3 de arquitetura diferente. Consequencialidade, no segundo exercício que vamos realizar nesse tutorial será construir um tarball stage3 para <tt>core2 32bit</tt> a partir do tarball stage3 do <tt>pentium4</tt> que acabamos de construir.
+
 
+
TODO: adicione  ressalvas a respeito de quais arquiteturas podem ser semeadas e quais não podem (talvez um table?)
+
 
+
== Build Adaptado ==
+
 
+
Por ultimo, também é válido tanto <tt>local</tt> e <tt>builds remotos</tt>, [[Metro]] pode ser configurado para adicionar e/ou remover pacotes individuais ao ultimo tarball.
+
Digamos que você não consiga viver sem <tt>app-misc/screen</tt>, no final desse tutorial, mostraremos como obter o seu stage3 adaptado para incluí-lo.
+
 
+
== Instalar o Metro ==
+
 
+
'''O método recomendado e que possui suporte''' é utilizar o repositório Git do [[Metro]]. 
+
 
+
Asegure-se de que {{Package|dev-vcs/git}} e {{Package|dev-python/boto}} (opcional; exigido para suporte EC2) sejam estalados no seu sistema:
+
 
+
<console>
+
# ##i##emerge dev-vcs/git
+
# ##i##emerge dev-python/boto
+
</console>
+
 
+
Depois, clone o master git repository como a seguir:
+
 
+
<console>
+
# ##i##cd /root
+
# ##i##git clone git://github.com/funtoo/metro.git
+
# ##i##cp /root/metro/metro.conf ~/.metro
+
</console>
+
 
+
Agora você terá o diretório chamado <tt>/root/metro</tt> que contem todo o código fonte do [[Metro]].
+
 
+
O Metro agora está instalado. É hora de personalizá-lo para o seu sistema local.
+
 
+
= Configuring Metro =
+
 
+
{{Note|Metro não é atualmente capaz de construir stages do Gentoo. Veja {{Bug|FL-901}}.}}
+
 
+
[[User:Drobbins|Daniel Robbins]] mantem o  [[Metro]], então ele vem pré-configurado para construir com sucesso os lançamentos do [[Funtoo Linux]]. Antes de ler mais adiante, você pode querer personalizar algumas configurações básicas como o o número de de jobs simultâneos para se encaixar as compatibilidades do seu hardware ou o diretório para utilizar arquivos stage produzidos. Isso é excelente ao editar <tt>~/.metro</tt> a qual é o arquivo de configuração [[Metro]]'s master.
+
 
+
Por favor, note que o <code>path/install</code> deve apontar para aonde o metro foi instalado. Aponte <code>path/distfiles</code> para aonde seus distfiles residem. Defina também <code>path/mirror/owner</code> e <code>path/mirror/group</code> para o proprietário e grupo de todos os arquivos que serão escritos para construir o diretório do repositório, que por padrão (como por arquivo de configuração) está em <code>/home/mirror/funtoo</code>. O diretório cache normalmente reside dentro do diretório temporário -- Isso pode ser modificado como desejado. O diretório cache pode terminar contendo muitos pacotes .tbz2 em cache, e comer um monte de armazenamento (storage). Você pode de repente querer colocar o diretório temporário em armazenamento mais rápido, para tempos de compilação mais rápidas, e colocar o diretório cache em armazenamento mais lento, porém mais abundante.
+
 
+
{{file|name=.metro|desc=Metro configuration|body=
+
# Main metro configuration file - these settings need to be tailored to your install:
+
 
+
[section path]
+
install: /root/metro
+
tmp: /var/tmp/metro
+
cache: $[path/tmp]/cache
+
distfiles: /var/src/distfiles
+
work: $[path/tmp]/work/$[target/build]/$[target/name]
+
 
+
[section path/mirror]
+
 
+
: /home/mirror/funtoo
+
owner: root
+
group: repomgr
+
dirmode: 775
+
 
+
[section portage]
+
 
+
MAKEOPTS: auto
+
 
+
[section emerge]
+
 
+
options: --jobs=4 --load-average=4 --keep-going=n
+
 
+
# This line should not be modified:
+
[collect $[path/install]/etc/master.conf]
+
 
}}
 
}}
 +
== Introduction ==
 +
FFmpeg and Libav are library sets for multimedia decoding (and more). Both libraries expose similar API and features.
  
== Arch e Subarch ==
+
Both project have common origins and diverged only recently. The developers share the same bad coding practices causing permanent lack of API and ABI stability, therefore requiring frequent rebuilds of reverse dependencies.
  
No exemplo seguinte estamos gerando um stage 3 para pentium4 compilado para a compatibilidade binária x86-32bit. Pentium4 é uma subarch da arquitetura x86-32bit. Uma vez que você tenha o metro instalado, você pode encontrar uma lista completa de cada subarch em seu diretório <tt>/root/metro/subarch</tt>; cada  arquitetura terá a extensão de arquivo .spec
+
Worse than that, after the split projects use colliding SONAMEs for libraries with potentially different ABI. This means that after switching from one implementation to another, the reverse dependencies may become broken instantly (preserved-libs doesn't help) and need to be rebuilt ASAP.
Example:
+
<console>
+
###i## ls /root/metro/subarch
+
# ls subarch/
+
amd64-bulldozer-pure64.spec  armv7a.spec          core-avx-i.spec        i686.spec        pentium.spec
+
amd64-bulldozer.spec        armv7a_hardfp.spec  core2_32.spec          k6-2.spec        pentium2.spec
+
amd64-k10-pure64.spec        athlon-4.spec        core2_64-pure64.spec    k6-3.spec        pentium3.spec
+
amd64-k10.spec              athlon-mp.spec      core2_64.spec          k6.spec          pentium4.spec
+
amd64-k8+sse3.spec          athlon-tbird.spec    corei7-pure64.spec      native_32.spec    pentiumpro.spec
+
amd64-k8+sse3_32.spec        athlon-xp.spec      corei7.spec            native_64.spec    prescott.spec
+
amd64-k8-pure64.spec        athlon.spec          generic_32.spec        niagara.spec      ultrasparc.spec
+
amd64-k8.spec                atom_32.spec        generic_64-pure64.spec  niagara2.spec    ultrasparc3.spec
+
amd64-k8_32.spec            atom_64-pure64.spec  generic_64.spec        nocona.spec      xen-pentium4+sse3.spec
+
armv5te.spec                atom_64.spec        generic_sparcv9.spec    opteron_64.spec  xen-pentium4+sse3_64.spec
+
armv6j.spec                  btver1.spec          geode.spec              pentium-m.spec
+
armv6j_hardfp.spec          btver1_64.spec      i486.spec              pentium-mmx.spec
+
</console>
+
  
= First stages build (local build) =
+
Many packages for video decoding, are done via FFmpeg or Libav.  Differences between FFmpeg and Libav can have a major impact on its behavior: the number of files it can decode, whether it decodes correctly, what video and audio filters are provided, network behavior, and more.
  
To get this all started, we need to bootstrap the process by downloading an initial seed stage3 to use for building and place it in its proper location in <tt>/home/mirror/funtoo</tt>, so that [[Metro]] can find it. We will also need to create some special &quot;control&quot; files in <tt>/home/mirror/funtoo</tt>, which will allow [[Metro]] to understand how it is supposed to proceed.
+
== Current status ==
 +
=== Funtoo ===
 +
Funtoo supports only FFmpeg. It is forced by Funtoo version of virtual/ffmpeg. This provides the ability to avoid the Gentoo issues, as explained below.
  
== Step 1: Set up pentium4 repository (local build) ==
+
Decision made by Oleg, forced by #funtoo community and bugtracker reports.
  
Assuming we're following the basic steps outlined in the previous section, and building an unstable funtoo (<tt>funtoo-current</tt>) build for the <tt>pentium4</tt>, using a generic <tt>pentium4</tt> stage3 as a seed stage, then here the first set of steps we'd perform:
+
=== Gentoo ===
 +
Gentoo supports both ffmpeg and libav, with a preference towards libav. The preference is caused by package order in virtual/ffmpeg — when no other circumstance affects the package choice, Portage will prefer libav. However, if ffmpeg is already installed or a package incompatible with libav is requested, Portage will use ffmpeg instead.
  
<console>
+
There are two major technical issues with this design:
# ##i##install -d /home/mirror/funtoo/funtoo-current/x86-32bit/pentium4
+
# there is no technically correct way of forcing rebuilds on ABI changes — subslot dependencies do not work with virtuals or || () deps,
# ##i##install -d /home/mirror/funtoo/funtoo-current/snapshots
+
# there is no way of forcing rebuilds when switching from libav to ffmpeg, and the other way around.
# ##i##cd /home/metro/mirror/funtoo/funtoo-current/x86-32bit/pentium4
+
# ##i##install -d 2011-12-13
+
# ##i##cd 2011-12-13
+
# ##i##wget -c http://ftp.osuosl.org/pub/funtoo/funtoo-current/x86-32bit/pentium4/2011-12-13/stage3-pentium4-funtoo-current-2011-12-13.tar.xz
+
# ##i##cd ..
+
# ##i##install -d .control/version
+
# ##i##echo "2011-12-13" > .control/version/stage3
+
# ##i##install -d .control/strategy
+
# ##i##echo local >  .control/strategy/build
+
# ##i##echo stage3 > .control/strategy/seed
+
</console>
+
  
OK, let's review the steps above. First, we create the directory <tt>/home/mirror/funtoo/funtoo-current/x86-32bit/pentium4</tt>, which is where Metro will expect to find unstable <tt>funtoo-current</tt> pentium4 builds -- it is configured to look here by default. Then we create a specially-named directory to house our seed x86 stage3. Again, by default, Metro expects the directory to be named this way. We enter this directory, and download our seed x86 stage3 from funtoo.org. Note that the <tt>2010-12-24</tt> version stamp matches. Make sure that your directory name matches the stage3 name too. Everything has been set up to match Metro's default filesystem layout.
+
== Future status ==
 +
=== Funtoo ===
 +
Funtoo has no plan to change default FFmpeg, it only needs to mask libav in the profiles. Then dependencies on updated packages will unconditionally use FFmpeg. Eventually Funtoo will want to remove virtual/ffmpeg and depend on media-video/ffmpeg:0= directly in forked packages.
  
Next, we go back to the <tt>/home/mirror/metro/funtoo-current/x86-32bit/pentium4</tt> directory, and inside it, we create a <tt>.control</tt> directory. This directory and its subdirectories contain special files that Metro references to determine certain aspects of its behavior. The <tt>.control/version/stage3</tt> file is used by Metro to track the most recently-built stage3 for this particular build and subarch. Metro will automatically update this file with a new version stamp after it successfully builds a new stage3. But because Metro didn't actually ''build'' this stage3, we need to set up the <tt>.control/version/stage3</tt> file manually. This will allow Metro to find our downloaded stage3 when we set up our pentium4 build to use it as a seed. Also note that Metro will create a similar <tt>.control/version/stage1</tt> file after it successfully builds an pentium4 funtoo-current stage1.
+
=== Gentoo ===
 +
There is a planned Gentoo change which will eventually replace virtual/ffmpeg and explicit || () deps with 'libav' USE flag. The flag will be added to all packages that support both FFmpeg and libav. When the flag is enabled, the package will use libav; otherwise it will use FFmpeg. The choice of flag name is forced by the fact that USE=ffmpeg is already used as generic ffmpeg-or-libav flag.
  
We also set up <tt>.control/strategy/build</tt> and <tt>.control/strategy/seed</tt> files with values of <tt>local</tt> and <tt>stage3</tt> respectively. These files define the building strategy Metro will use when we build pentium4 funtoo-current stages. With a build strategy of <tt>local</tt>, Metro will source its seed stage from funtoo-current pentium4, the current directory. And with a seed strategy of <tt>stage3</tt>, Metro will use a stage3 as a seed, and use this seed to build a new stage1, stage2 and stage3.
+
This change fixes both Gentoo issues:
 +
# USE-conditional dependencies allow subslot dependencies to force rebuilds on ABI changes,
 +
# provider change will force rebuild because of USE flag change.
  
== Step 2: Building the pentium4 stages ==
+
The change may also eventually make it possible to install FFmpeg and libav side-by-side. Until then, the flag state would involve 'strong' preference of one implementation over the other, and user will have to change USE=libav as a global flag. '''Installing a package that supports only one of the two implementations will result in blocker that needs to be handled manually'''.
  
Incidentally, if all you wanted to do at this point was to build a new pentium4 funtoo-current stage1/2/3 (plus openvz and vserver templates). You would begin the process by typing:
 
  
<console>
+
== Detailed information on FFmpeg and libav ==
# ##i##cd /root/metro
+
=== FFmpeg and Libav history ===
# ##i##scripts/ezbuild.sh funtoo-current pentium4
+
In 2011, parts of the FFmpeg developers were unhappy about the FFmpeg leadership, and decided to take over. This didn't quite work out. Apparently Fabrice Bellard, original FFmpeg developer and owner of the ffmpeg.org domain name, decided not to hand over the domain name to the new maintainers. So they followed Plan B, and forked FFmpeg, resulting in Libav. Since then, Libav did its own development, and completely ignored whatever FFmpeg did. FFmpeg, on the other hand, started to merge literally everything Libav did.
</console>
+
  
If you have a slow machine, it could take several hours to be completed because several "heavy" components like gcc or glibc have to be recompiled in each stage. Once a stage has been successfully completed, it is placed in the <tt>"${METRO_MIRROR}/funtoo-current/x32-bit/pentium4/YYYY-MM-DD"</tt> subdirectory, where <tt>YYYY-MM-DD</tt> is today's date at the time the <tt>ezbuild.sh</tt> script was started or the date you put on the ezscript.sh command line.
+
The reason for the fork is most likely that the developers hate each other. While this formulation seems somewhat sloppy, it is most likely the truth. To this date, the #libav-devel IRC channel still has Michael Niedermayer (the FFmpeg maintainer since 2004 according to Wikipedia) on their ban list (similar misbehavior is exhibited by some FFmpeg developers). There is little to no cooperation between the two projects.
  
= Building for another binary compatible architecture (remote build) =
+
More about FFmpeg's history and the fork incident can be found on Wikipedia
  
As written above, [[Metro]] is able to perform '''remote build''' building different architecture stage3 from a binary compatible seeding stage3 (e.g. using a pentium4 stage3 to seed a <tt>Intel Core2 32bits</tt> stage3).  
+
=== Situation today ===
 +
FFmpeg has more features and slightly more active development than Libav, going by mailing list and commit volume. In particular, FFmpeg's features are a superset of Libav's features. This is because FFmpeg merges Libav's git master on a daily basis. Libav on the other hand seems to prefer to ignore FFmpeg development (with occasional cherry-picking of bug fixes and features).
  
In the Metro terminology this is called a '''remote build''' (a stage 3 of a different, but binary compatible, architecture is used as a seed).  
+
Some Linux distributions, especially those that had Libav developers as FFmpeg package maintainers, replaced FFmpeg with Libav, while other distributions stick with FFmpeg. Application developers typically have to make sure their code works with both libraries. This can be trivial to hard, depending on the details. One larger problem is that the difference between the libraries makes it hard to keep up a consistent level of the user experience, since either library might silently or blatantly be not up to the task. It also encourages library users to implement some features themselves, rather than dealing with the library differences, or the question to which project to contribute.
What's not compatible? You can't use a <tt>Sparc</tt> architecture to generate an <tt>x86</tt> or <tt>ARM</tt> based stage and vice-versa. If you use a 32bit stage then you don't want to seed a 64bit build from it. Be sure that you are using a stage from the same architecture that you are trying to seed. Check [http://ftp.osuosl.org/pub/funtoo/funtoo-current/ Funtoo-current FTP Mirror] for a stage that is from the same Architecture that you will be building.
+
  
{{Note|Often, one build (ie. funtoo-current) can be used as a seed for another build such as funtoo-stable. However, hardened builds require hardened stages as seeds in order for the build to complete successfully.}}
+
FFmpeg and Libav developers also seem to have the tendency to ignore the damage their rivalry is causing. Apparently fighting out these issues on the users' backs is better than reconciling. This means everyone using these libraries either has to suffer from the differences, or reimplement functionality that is not the same between FFmpeg and Libav.
 
+
{{FLOPFooter}}
== Step 1: Set up Core_2 32bit repository ==
+
 
+
In this example, we're going to use this pentium4 funtoo-current stage3 to seed a new Core_2 32bit funtoo-current build. To get that done, we need to set up the pentium4 build directory as follows:
+
 
+
<console>
+
# ##i## cd /home/mirror/funtoo/funtoo-current/x86-32bit
+
# ##i##install -d core2_32
+
# ##i##cd core2_32
+
# ##i##install -d .control/strategy
+
# ##i##echo remote > .control/strategy/build
+
# ##i##echo stage3 > .control/strategy/seed
+
# ##i##install -d .control/remote
+
# ##i##echo funtoo-current > .control/remote/build
+
# ##i##echo x86-32bit > .control/remote/arch_desc
+
# ##i##echo pentium4 > .control/remote/subarch
+
</console>
+
 
+
The steps we follow are similar to those we performed for a ''local build'' to set up our pentium4 directory for local build. However, note the differences. We didn't download a stage, because we are going to use the pentium4 stage to build a new Core_2 32bit stage. We also didn't create the <tt>.control/version/stage{1,3}</tt> files because Metro will create them for us after it successfully builds a new stage1 and stage3. We are still using a <tt>stage3</tt> seed strategy, but we've set the build strategy to <tt>remote</tt>, which means that we're going to use a seed stage that's not from this particular subdirectory. Where are we going to get it from? The <tt>.control/remote</tt> directory contains this information, and lets Metro know that it should look for its seed stage3 in the <tt>/home/mirror/funtoo/funtoo-current/x86-32bit/pentium4</tt> directory. Which one will it grab? You guessed it -- the most recently built ''stage3'' (since our seed strategy was set to <tt>stage3</tt>) that has the version stamp of <tt>2010-12-24</tt>, as recorded in <tt>/home/mirror/funtoo-current/x86-32bit/pentium4/.control/version/stage</tt>. Now you can see how all those control files come together to direct Metro to do the right thing.
+
 
+
{{Note|<code>arch_desc</code> should be set to one of: <code>x86-32bit</code>, <code>x86-64bit</code> or <code>pure64</code> for PC-compatible systems. You must use a 32-bit build as a seed for other 32-bit builds, and a 64-bit build as a seed for other 64-bit builds.}}
+
 
+
== Step 2: Building the Core_2 32bit stages ==
+
 
+
Now, you could start building your new Core_2 32bit stage1/2/3 (plus openvz and vserver templates) by typing the following:
+
 
+
<console>
+
# ##i##/root/metro/scripts/ezbuild.sh funtoo-current core2_32
+
</console>
+
 
+
In that case, the produced stages are placed in the <tt>/home/mirror/funtoo/funtoo-current/x32-bit/core2_32/YYYY-MM-DD</tt> subdirectory.
+
 
+
== Step 3: The Next Build ==
+
 
+
At this point, you now have a new Core_2 32bit stage3, built using a "remote" pentium4 stage3. Once the first remote build completes successfully, metro will automatically change <code>.control/strategy/build</code> to be <code>local</code> instead of <code>remote</code>, so it will use the most recently-built Core_2 32bit stage3 as a seed for any new Core_2 32bit builds from now on.
+
 
+
= Build your own tailored stage3 =
+
 
+
Metro can be easily configured for building custom stage3 by including additional packages. Edit the following configuration file <tt>/root/metro/etc/builds/funtoo-current/build.conf</tt>:
+
{{file|name=funtoo-current/build.conf|body=
+
[collect ../../fslayouts/funtoo/layout.conf]
+
 
+
[section release]
+
 
+
author: Daniel Robbins <drobbins@funtoo.org>
+
 
+
[section target]
+
 
+
compression: xz
+
 
+
[section portage]
+
 
+
FEATURES:
+
SYNC: $[snapshot/source/remote]
+
USE:
+
 
+
[section profile]
+
 
+
format: new
+
path: gentoo:funtoo/1.0/linux-gnu
+
arch: $[:path]/arch/$[target/arch_desc]
+
build: $[:path]/build/current
+
flavor: $[:path]/flavor/core
+
mix-ins:
+
 
+
[section version]
+
 
+
python: 2.7
+
 
+
[section emerge]
+
 
+
 
+
[section snapshot]
+
 
+
type: live
+
compression: xz
+
 
+
[section snapshot/source]
+
 
+
type: git
+
branch: funtoo.org
+
# branch to have checked out for tarball:
+
branch/tar: origin/master
+
name: ports-2012
+
remote: git://github.com/funtoo/ports-2012.git
+
options: pull
+
 
+
[section metro]
+
 
+
options:
+
options/stage: cache/package
+
target: gentoo
+
 
+
[section baselayout]
+
 
+
services: sshd
+
 
+
[section multi]
+
 
+
snapshot: snapshot
+
 
+
[section files]
+
 
+
motd/trailer: [
+
 
+
>>> Send suggestions, improvements, bug reports relating to...
+
 
+
>>> This release:                  $[release/author]
+
>>> Funtoo Linux (general):        Funtoo Linux (http://www.funtoo.org)
+
>>> Gentoo Linux (general):        Gentoo Linux (http://www.gentoo.org)
+
]
+
 
+
[collect ../../multi-targets/$[multi/mode:zap]]
+
}}
+

Latest revision as of 07:44, February 1, 2015

Created on
2015/01/31
Original Author(s)
{{#set:Author={{ #show: User:Oleg | ?Full name }}}}{{ #show: User:Oleg | ?Full name }},{{#set:Author={{ #show: User:Mgorny | ?Full name }}}}{{ #show: User:Mgorny | ?Full name }}
Current Maintainer(s)
Oleg Vinichenko
Status
Reference Bug
FL-844

Funtoo Linux Optimization Proposal: FFmpeg

Funtoo Linux prefers FFmpeg. Some enlightenment about our choice and why we prefer this or could switch to alternative in future.

Introduction

FFmpeg and Libav are library sets for multimedia decoding (and more). Both libraries expose similar API and features.

Both project have common origins and diverged only recently. The developers share the same bad coding practices causing permanent lack of API and ABI stability, therefore requiring frequent rebuilds of reverse dependencies.

Worse than that, after the split projects use colliding SONAMEs for libraries with potentially different ABI. This means that after switching from one implementation to another, the reverse dependencies may become broken instantly (preserved-libs doesn't help) and need to be rebuilt ASAP.

Many packages for video decoding, are done via FFmpeg or Libav. Differences between FFmpeg and Libav can have a major impact on its behavior: the number of files it can decode, whether it decodes correctly, what video and audio filters are provided, network behavior, and more.

Current status

Funtoo

Funtoo supports only FFmpeg. It is forced by Funtoo version of virtual/ffmpeg. This provides the ability to avoid the Gentoo issues, as explained below.

Decision made by Oleg, forced by #funtoo community and bugtracker reports.

Gentoo

Gentoo supports both ffmpeg and libav, with a preference towards libav. The preference is caused by package order in virtual/ffmpeg — when no other circumstance affects the package choice, Portage will prefer libav. However, if ffmpeg is already installed or a package incompatible with libav is requested, Portage will use ffmpeg instead.

There are two major technical issues with this design:

  1. there is no technically correct way of forcing rebuilds on ABI changes — subslot dependencies do not work with virtuals or || () deps,
  2. there is no way of forcing rebuilds when switching from libav to ffmpeg, and the other way around.

Future status

Funtoo

Funtoo has no plan to change default FFmpeg, it only needs to mask libav in the profiles. Then dependencies on updated packages will unconditionally use FFmpeg. Eventually Funtoo will want to remove virtual/ffmpeg and depend on media-video/ffmpeg:0= directly in forked packages.

Gentoo

There is a planned Gentoo change which will eventually replace virtual/ffmpeg and explicit || () deps with 'libav' USE flag. The flag will be added to all packages that support both FFmpeg and libav. When the flag is enabled, the package will use libav; otherwise it will use FFmpeg. The choice of flag name is forced by the fact that USE=ffmpeg is already used as generic ffmpeg-or-libav flag.

This change fixes both Gentoo issues:

  1. USE-conditional dependencies allow subslot dependencies to force rebuilds on ABI changes,
  2. provider change will force rebuild because of USE flag change.

The change may also eventually make it possible to install FFmpeg and libav side-by-side. Until then, the flag state would involve 'strong' preference of one implementation over the other, and user will have to change USE=libav as a global flag. Installing a package that supports only one of the two implementations will result in blocker that needs to be handled manually.


Detailed information on FFmpeg and libav

FFmpeg and Libav history

In 2011, parts of the FFmpeg developers were unhappy about the FFmpeg leadership, and decided to take over. This didn't quite work out. Apparently Fabrice Bellard, original FFmpeg developer and owner of the ffmpeg.org domain name, decided not to hand over the domain name to the new maintainers. So they followed Plan B, and forked FFmpeg, resulting in Libav. Since then, Libav did its own development, and completely ignored whatever FFmpeg did. FFmpeg, on the other hand, started to merge literally everything Libav did.

The reason for the fork is most likely that the developers hate each other. While this formulation seems somewhat sloppy, it is most likely the truth. To this date, the #libav-devel IRC channel still has Michael Niedermayer (the FFmpeg maintainer since 2004 according to Wikipedia) on their ban list (similar misbehavior is exhibited by some FFmpeg developers). There is little to no cooperation between the two projects.

More about FFmpeg's history and the fork incident can be found on Wikipedia

Situation today

FFmpeg has more features and slightly more active development than Libav, going by mailing list and commit volume. In particular, FFmpeg's features are a superset of Libav's features. This is because FFmpeg merges Libav's git master on a daily basis. Libav on the other hand seems to prefer to ignore FFmpeg development (with occasional cherry-picking of bug fixes and features).

Some Linux distributions, especially those that had Libav developers as FFmpeg package maintainers, replaced FFmpeg with Libav, while other distributions stick with FFmpeg. Application developers typically have to make sure their code works with both libraries. This can be trivial to hard, depending on the details. One larger problem is that the difference between the libraries makes it hard to keep up a consistent level of the user experience, since either library might silently or blatantly be not up to the task. It also encourages library users to implement some features themselves, rather than dealing with the library differences, or the question to which project to contribute.

FFmpeg and Libav developers also seem to have the tendency to ignore the damage their rivalry is causing. Apparently fighting out these issues on the users' backs is better than reconciling. This means everyone using these libraries either has to suffer from the differences, or reimplement functionality that is not the same between FFmpeg and Libav.

blog comments powered by Disqus