Changes

Jump to: navigation, search

Ruby-kit

8,266 bytes added, 9 months ago
Portage Limitations (Technical)
;{{c|ruby-2.7}}: This is the Funtoo OS "preferred" Ruby for satisfying system dependencies. This is the Ruby version that applications will try to use if 2.7 support is actually available in ebuilds and dependencies.
;{{c|ruby-2.6}}: This is the Funtoo OS "compatible" Ruby for satisfying system dependencies. This is the version that all applications are likely to support and is more universally supported in ebuilds. 2.6 is also our **''backup** '' version of Ruby that is used by our eclass logic(more info below).;{{c|ruby-3.0}}(Not yet added): This version of Ruby is being made available but is not currently used for resolving dependencies for Funtoo systems. It can be installed by Ruby developers and Ruby modules can be installed via {{c|gem}}.
== Funtoo Changes ==
* {{c|ruby-single.eclass}}
* {{c|ruby-utils.eclass}}
 
== The Problem Being Solved ==
 
These changes were made to solve a specific issue. We have found that in Gentoo, Ruby versions (this also happens with Python versions) can be very "sticky". By "sticky", I mean that it can be very
hard to get rid of an older version of Ruby (or Python), and upstream Gentoo tends to try to handle this by orchestrating a mass purge of all ebuilds that reference this older implementation. Only
then can this older version of the language be removed or deprecated in Gentoo. This involves updating hundreds of ebuilds, which is a lot of manual work, and thus isn't really a great solution. But
it does fit in with the Gentoo model of having a large development team that "shuffles" ebuilds along individually to either add support for new language versions, or drop old versions. While this is feasible for a larger development team, it isn't really that pleasant to do, and can result in some challenges for users as well. For Funtoo,
this isn't really a workable option so a better solution was needed.
 
For Funtoo, I wanted to have a way to define what versions of Ruby were active in a build of Funtoo, in as central a way as possible, and not have annoying, soul-crushing dependency problems that
users or Funtoo developers had to deal with. The solution used was to modify Ruby eclasses to more aggressively mask the {{c|USE_RUBY}} settings from the ebuilds based on what Ruby implementation
we actually want "active" in Funtoo.
 
A specific issue I wanted to address was to be able to easily deprecate Ruby 2.5. And yet, many ebuilds still referenced {{c|ruby25}} or even earlier as the maximum version that they support. To fix this issue, I
introduced an idea of a "backup Ruby version" which would get enabled if our more aggressive masking of {{c|USE_RUBY}} resulted in a Ruby package or module not having support for ''any'' Ruby
implementation at all. In this case, the eclass will enable the backup Ruby version -- in this case Ruby 2.6, which has a high degree of compatibility with Ruby 2.5 that we are deprecating so it
is likely to work in the majority of cases for older Ruby ebuilds, and in the few cases where it may not, we can deal with these problems on a case-by-case basis.
 
This way, we can set Ruby 2.6 as our "baseline", or "compatibility" version that our minimum supported version in Funtoo ebuilds. Ensuring everything works with 2.6 should not be tremendously hard,
and we can move to 2.6 without changing every single ebuild that only references a 2.5 or earlier Ruby version. 2.7 is our "preferred" version -- and will be used if an ebuild supports it. But many
ebuilds are missing {{c|ruby27}} in {{c|USE_RUBY}}. They may work fine with Ruby 2.7 and simply need to be updated to reflect this, or they may not. I view this as a bit more of a stretch at this
time as compared to 2.6, so I did not select Ruby 2.7 as our "compatibility" version, although I would have preferred to do this. Ebuilds still need to "opt in" to Ruby 2.7+ support.
 
These masking changes also have an interesting side-effect where we can make Ruby 3.0 available for developers, but Ruby-using packages will not actually depend on Ruby 3.0, ''even if they reference
{{c|ruby30}} in {{c|USE_RUBY}}, until we "flip the switch" and tweak our eclasses. This allows Ruby 3.0 to be available to merge but not get entangled in system dependencies. This does mean that you
would need to install Ruby 3.0 dependencies using {{c|gem}}, which is a fine solution since no ebuilds in Funtoo will need any ruby-module ebuilds installed. Basically we create a clean-room where
Ruby 3.0 can be used for now, until it becomes the official Ruby implementation in a future version of this kit -- when we choose.
== Change Details ==
RUBY_TARGETS="ruby27 ruby26"
}}
 
{{Important|1=Note that the ''preferred ruby version should be listed first.'' This is because the ordering here will be used to define the order in some "or" ({{c|{{!}}{{!}} ( opt1 opt2 ... )}}) dependencies in ebuilds, and the first one listed in this dependency is "preferred" by Portage.}}
In addition, Ruby-using ebuilds that leverage the Ruby eclasses will set something like this:
The Funtoo changes cause the {{c|USE_RUBY}} settings in ebuilds to be more aggressively ''masked'' based on the Funtoo OS settings for {{c|RUBY_TARGETS}}. In addition, if our more aggressive masking causes a Ruby module or package to not have any valid Ruby implementations, a "backup" typically safe Ruby implementation will be enabled automatically.
 
This gives us the ability to "focus" our Ruby support around specific versions of Ruby.
 
== Portage Limitations (Technical) ==
 
To implement all this, the {{c|RUBY_TARGETS}} settings in the profiles must be kept in sync with the forked Ruby eclasses in Core-kit, so that they reference the same Ruby versions. Ideally, we would be able to
set {{c|RUBY_TARGETS}} in one central place, and not keep multiple things in sync. But due to limitations in Portage, this is how it needs to be done.
 
'''The reason why we can't set Ruby settings all in one place is because Portage expects that "an ebuild's dependencies" come from the ebuild (tracked by md5 digest), potentially modified by eclasses (tracked by md5 digest), and that there are no other external influences.'''
 
{{c|RUBY_TARGETS}} is a profile setting, which gets USE expanded to {{c|ruby_targets_ruby26}}, etc. Dependencies "interact" with USE settings via ''conditionals'' which are actually stored in the dependency itself, and in the md5-cache, so dependencies can "dynamically adapt" to USE settings (this is why USE settings are used so heavily by Gentoo for so many strange things.) This means that getting dynamic changes from USE changes is "safe" from a Portage perspective.
 
{{c|USE_RUBY}} is an ebuild setting, which is hard-coded into each ebuild. This is also OK -- no problems here.
 
But the Ruby eclasses cannot use the setting of {{c|RUBY_TARGETS}} to influence the content of {{c|DEPEND}} and other variables in the ebuild -- here is why. Ebuild dependencies get written to the Portage ebuild md5-cache. The Portage ebuild md5-cache can detect when eclasses have been changed via their digest (md5 hash,) and when this happens, the cache entries will be invalidated and new data will be extracted from the ebuild to ensure that the dependency values are correct. But the Portage ebuild md5-cache cannot detect when a change to ''profiles'' would impact dependencies.
 
Because of this design, settings in profiles that have the potential of changing are not allowed to "influence" the eclass directly. Eclasses are just allowed to "augment" things like dependencies in an ebuild based on their own, self-contained logic. Thus, {{c|RUBY_TARGETS}} is not allowed to be simply read and used by the eclass. Doing this would result in the Portage md5-cache containing incorrect information. So we are unfortunately stuck in a position where we need to "keep the beams from touching" and need to have hard-coded logic or values in the eclass so that Portage can track any changes to this logic. If we change the values in the eclass, then its digest will change, and thus the md5-cache will detect this change and update its cache of ebuild metadata.
 
{{Note|The current rule for Portage is that dependencies in packages must be defined in the ebuild itself, and may be modified by eclasses. But eclasses cannot be 'influenced' by profile settings, because Portage is unable to detect these changes and invalidate its internal cache.}}
 
There is no trivial solution to this that I am aware, but there are potential solutions:
 
# Deprecate {{c|RUBY_TARGETS}}, and have the OS Ruby settings set in the eclass. Now they are defined in one place. This means that users can't override them in {{c|/etc/make.conf}}. The eclasses must live in core-kit for technical reasons, and thus couldn't change automatically if a user changed their Ruby kit. So there are some downsides to this.
# Deprecate md5-caching, or at least Funtoo-generated cache entries for improved performance. Then a user's local profile settings could be factored in to the md5-cache calculation. This would allow eclasses to interact with dynamic profile settings. It could reduce speed of Portage temporarily since it would need to rebuild its cache more often. If cache regen were fast enough, this wouldn't matter (and Funtoo has much faster cache regeneration code than Gentoo which could be leveraged.)
# Just deal with the current situation for now, which is at least an improvement. Users don't generally need access to different ruby-kits, except developers when testing, and we have tools for developers to deal with this easily anyway ({{c|merge-kits}}).
[[Category:Official Documentation]]
[[Category:Kits]]
Bureaucrats, Administrators, wiki-admins, wiki-staff
6,633
edits

Navigation menu