Changes

Jump to: navigation, search

User:Pytony/Minimize Python2 Usage

14,616 bytes added, 3 years ago
Created page with "In this document I describe how I moved to python 3.5 with less python 2.7 as possible. First thing to do was to unmask python-3.5 and install it : <console> ###i## echo '=d..."
In this document I describe how I moved to python 3.5 with less python 2.7 as possible.

First thing to do was to unmask python-3.5 and install it :

<console>
###i## echo '=dev-lang/python-3.5*' >> /etc/portage/package.unmask
###i## emerge '=dev-lang/python3.5*'
</console>

Then I set PYTHON_SINGLE_TARGET and PYTHON_TARGETS to python3_5 in my make.conf:

{{file|name=/etc/portage/make.conf|body=
PYTHON_SINGLE_TARGET="python3_5"
PYTHON_TARGETS="${PYTHON_SINGLE_TARGET}"
}}

Now let's try to emerge to update <code>@world</code>. It is very likely that it will fail
because of packages requiring python but not supporting python 3.5.

In most cases you should be able to just disable python support. For instance, here is how
I handled <code>sci-libs/geos</code>:

<console>
➜ ~ emerge -pvuDN @world

These are the packages that would be merged, in order:

Calculating dependencies... done!

!!! The ebuild selected to satisfy ">=sci-libs/geos-3.5.0" has unmet requirements.
- sci-libs/geos-3.6.1::gentoo USE="python -doc -ruby -static-libs" PYTHON_TARGETS="-python2_7"

The following REQUIRED_USE flag constraints are unsatisfied:
python? ( python_targets_python2_7 )

(dependency required by "dev-db/postgis-2.3.2-r1::gentoo" [installed])
(dependency required by "@selected" [set])
(dependency required by "@world" [argument])
➜ ~ sudo chuse geos -python --because it requires python2.7
</console>

In some cases, python 2.7 will be mandatory:

<console>
➜ ~ emerge -pvuDN @world

These are the packages that would be merged, in order:

Calculating dependencies... done!

!!! The ebuild selected to satisfy ">=dev-libs/gobject-introspection-1.46.0:=" has unmet requirements.
- dev-libs/gobject-introspection-1.48.0::gentoo USE="-cairo -doctool -test" PYTHON_TARGETS="-python2_7"

The following REQUIRED_USE flag constraints are unsatisfied:
python_targets_python2_7

The above constraints are a subset of the following complete expression:
python_targets_python2_7 test? ( cairo )

(dependency required by "dev-python/pygobject-3.20.1::gentoo" [ebuild])
(dependency required by "x11-wm/qtile-0.10.7::gentoo[dbus]" [ebuild])
(dependency required by "@selected" [set])
(dependency required by "@world" [argument])
➜ ~ sudo chuse gobject-introspection python_targets_python2_7 --because only python2.7 available
</console>

In some other cases, python 2.7 support will be required by another package:

<console>
The following USE changes are necessary to proceed:
(see "package.use" in the portage(5) man page for more details)
# required by dev-util/itstool-2.0.2::gentoo
# required by app-text/yelp-tools-3.18.0::gentoo
# required by app-text/evince-3.20.1::gentoo
# required by @selected
# required by @world (argument)
>=dev-libs/libxml2-2.9.4-r1 python_targets_python2_7
➜ ~ sudo chuse libxml2 python_targets_python2_7 --because required by evince
</console>

Finally once you fixed every issue raised by <code>emerge -pvuDN @world</code>, you can run it
without pretend option.

Still some packages may fail to build.

<console>
➜ ~ sudo emerge -uDN @world
Calculating dependencies... done! >>> Verifying ebuild manifests
>>> Running pre-merge checks for dev-python/pycparser-2.17 >>> Emerging (1 of 53) sci-libs/geos-3.6.1::gentoo
>>> Emerging (2 of 53) virtual/jpeg-62::gentoo
>>> Emerging (3 of 53) dev-python/six-1.10.0::gentoo
>>> Installing (2 of 53) virtual/jpeg-62::gentoo >>> Installing (1 of 53) sci-libs/geos-3.6.1::gentoo
>>> Installing (3 of 53) dev-python/six-1.10.0::gentoo
>>> Emerging (4 of 53) dev-python/appdirs-1.4.3::gentoo
>>> Installing (4 of 53) dev-python/appdirs-1.4.3::gentoo
>>> Emerging (5 of 53) dev-python/pyparsing-2.2.0::gentoo
>>> Installing (5 of 53) dev-python/pyparsing-2.2.0::gentoo
>>> Emerging (6 of 53) dev-python/pyelftools-0.23-r1::gentoo
>>> Emerging (7 of 53) dev-python/virtualenv-clone-0.2.6::gentoo
>>> Emerging (8 of 53) dev-python/packaging-16.8::gentoo
>>> Failed to emerge dev-python/virtualenv-clone-0.2.6, Log file:
>>> '/var/tmp/portage/dev-python/virtualenv-clone-0.2.6/temp/build.log'
>>> Installing (6 of 53) dev-python/pyelftools-0.23-r1::gentoo
>>> Installing (8 of 53) dev-python/packaging-16.8::gentoo
>>> Jobs: 7 of 53 complete, 1 failed Load avg: 2.12, 1.25, 0.75
*** Resuming merge...
</console>

In the above case, I noticed in <code>/var/tmp/portage/dev-python/virtualenv-clone-0.2.6/temp/build.log</code>
that the issue was that setuptools was not yet built with python3.5 support. Probably a missing dependency in
<code>dev-python/virtualenv-clone</code>. In such case, just rebuild setuptools before updating world:

<console>
###i## emerge -1 setuptools
###i## emerge -uDN @world
</console>

I also faced a more specific error with <code>sys-libs/cracklib</code> which was that
it was not explicitly set in <codel>REQUIRED_USE</code> that <code>python</code> useflags
required <code>python_targets_python2_7</code>. So it failed at build time. The fix was just
to remove <code>python</code> useflag.

Once you have successfully updated <code>@world</code>, it is not entirely over. Indeed, there
might still be packages that were not updated though python targets changed. I can't explain this
behavior. Yet I could easily find these packages using <code>emerge -pve @world</code> and fix
issues (which are basically the same as with former <code>emerge -pvuDN @world</code>) until there
is no more error raised.

Once <code>emerge -pve @world</code> gives you a clean output, it would be a shame to loose time
recompiling every single package with <code>emerge -e @world</code>. So, I just listed packages
which had a useflag change, and emerged these packages only:

<console>
➜ ~ emerge -pve @world | grep '*'
[ebuild R ] dev-libs/libxml2-2.9.4-r1:2::gentoo USE="icu ipv6 python readline -debug -examples -lzma -static-libs {-test}" ABI_X86="32 (64) (-x32)" PYTHON_TARGETS="python2_7* python3_5 -python3_4" 0 KiB
[ebuild R ] dev-lang/yasm-1.3.0::gentoo USE="nls -python*" PYTHON_TARGETS="-python2_7*" 0 KiB
[ebuild R ] dev-python/pygments-2.2.0::gentoo USE="-doc {-test}" PYTHON_TARGETS="python3_5* -pypy -pypy3 -python2_7* -python3_4* -python3_6" 0 KiB
[ebuild R ] dev-python/docutils-0.13.1::gentoo PYTHON_TARGETS="python3_5* -pypy -pypy3 -python2_7* -python3_4* -python3_6" 0 KiB
[ebuild R ] dev-util/gdbus-codegen-2.48.2::gentoo PYTHON_TARGETS="python3_5* -python2_7* -python3_4*" 0 KiB
➜ ~ sudo emerge -1 libxml2 yasm pygments docutils gdbus-codegen
Calculating dependencies... done!
>>> Verifying ebuild manifests
>>> Emerging (1 of 5) dev-python/pygments-2.2.0::gentoo
>>> Emerging (2 of 5) dev-libs/libxml2-2.9.4-r1::gentoo
>>> Installing (1 of 5) dev-python/pygments-2.2.0::gentoo
>>> Emerging (3 of 5) dev-python/docutils-0.13.1::gentoo
>>> Installing (3 of 5) dev-python/docutils-0.13.1::gentoo
>>> Installing (2 of 5) dev-libs/libxml2-2.9.4-r1::gentoo
>>> Emerging (4 of 5) dev-lang/yasm-1.3.0::gentoo
>>> Emerging (5 of 5) dev-util/gdbus-codegen-2.48.2::gentoo
>>> Installing (5 of 5) dev-util/gdbus-codegen-2.48.2::gentoo
>>> Installing (4 of 5) dev-lang/yasm-1.3.0::gentoo
>>> Jobs: 5 of 5 complete Load avg: 2.67, 1.50, 0.79
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.
</console>

Finally, you can remove packages that became unneeded:

<console>
sudo emerge -c

* Always study the list of packages to be cleaned for any obvious
* mistakes. Packages that are part of the world set will always
* be kept. They can be manually added to this set with
* `emerge --noreplace <atom>`. Packages that are listed in
* package.provided (see portage(5)) will be removed by
* depclean, even if they are part of the world set.
*
* As a safety measure, depclean will not remove any packages
* unless *all* required dependencies have been resolved. As a
* consequence of this, it often becomes necessary to run
* `emerge --update --newuse --deep @world` prior to depclean.

Calculating dependencies... done!
>>> Calculating removal order...

dev-python/pyxattr
selected: 0.6.0
protected: none
omitted: none

dev-python/pygtk
selected: 2.24.0-r4
protected: none
omitted: none

dev-python/trollius
selected: 2.1
protected: none
omitted: none

gnome-base/libglade
selected: 2.6.4-r2
protected: none
omitted: none

dev-python/pygobject
selected: 2.28.6-r55
protected: none
omitted: 3.20.1

virtual/python-futures
selected: 0
protected: none
omitted: none

dev-python/futures
selected: 3.1.1
protected: none
omitted: none

dev-lang/python
selected: 3.4.6
protected: none
omitted: 2.7.13 3.5.3

All selected packages: =dev-python/pyxattr-0.6.0 =dev-lang/python-3.4.6 =gnome-base/libglade-2.6.4-r2 =dev-python/pygtk-2.24.0-r4 =dev-python/trollius-2.1 =virtual/python-futures-0 =dev-python/pygobject-2.28.6-r55 =dev-python/futures-3.1.1

>>> 'Selected' packages are slated for removal.
>>> 'Protected' and 'omitted' packages will not be removed.

>>> Waiting 5 seconds before starting...
>>> (Control-C to abort)...
>>> Unmerging in: 5 4 3 2 1
>>> Unmerging (1 of 8) dev-python/pyxattr-0.6.0...
>>> Unmerging (2 of 8) dev-python/pygtk-2.24.0-r4...
>>> Unmerging (3 of 8) dev-python/trollius-2.1...
>>> Unmerging (4 of 8) gnome-base/libglade-2.6.4-r2...
>>> Unmerging (5 of 8) dev-python/pygobject-2.28.6-r55...
>>> Unmerging (6 of 8) virtual/python-futures-0...
>>> Unmerging (7 of 8) dev-python/futures-3.1.1...
>>> Unmerging (8 of 8) dev-lang/python-3.4.6...
Packages installed: 579
Packages in world: 30
Packages in system: 78
Required packages: 579
Number removed: 8
</console>

In the end, here is what my <code>package.use</code> looks like:

{{file|name=/etc/portage/package.use|body=
dev-lang/yasm -python
dev-libs/gobject-introspection python_targets_python2_7
dev-libs/libxml2 python_targets_python2_7
dev-libs/libxslt -python
dev-util/boost-build -python
dev-util/itstool python_targets_python2_7
dev-vcs/git -python
media-gfx/gimp -python
media-libs/alsa-lib -python
net-libs/libproxy -python
sci-libs/geos -python
sys-apps/usbutils -python
sys-boot/boot-update python_targets_python2_7
sys-devel/llvm python_targets_python2_7
sys-libs/cracklib -python
x11-wm/qtile -python_targets_python2_7 widget-mpd
}}

Please ping me on IRC, if you have any question about it or wish to suggest me other issue examples relevant to python2 removal.

PS : If not already done, you might want to set python 3.5 as default interpreter as well :

<console>
###i## eselect python set python3.5
</console>
wiki-staff
336
edits

Navigation menu