Bug 10096 - Allow PREFERRED_VERSION miss to be fatal
Summary: Allow PREFERRED_VERSION miss to be fatal
Status: RESOLVED FIXED
Alias: None
Product: BitBake
Classification: Build System, Metadata & Runtime
Component: bitbake (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium+ enhancement
Target Milestone: 3.3 M3
Assignee: Charles Davies
QA Contact:
URL:
Whiteboard: NEWCOMER
Depends on:
Blocks:
 
Reported: 2016-08-08 11:18 UTC by Niko Mauno
Modified: 2021-03-18 15:03 UTC (History)
5 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Yes (doc changes required)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Niko Mauno 2016-08-08 11:18:20 UTC
When creating releases of Yocto-framework generated OS images containing components which should be enforced to a certain version, it sometimes occurs that an undesired version of some package(s) is/are included, which has the tendency to slow down release cycles. 

Yocto/OE have PREFERRED_VERSION_packagename = "<version>" support already since OE-classic, but even years have passed, a sufficiently well-established method to enforce certain packages to certain specific versions when bitbaking, doesn't however seem to have emerged.

I currently am able to fatalize PREFERRED_VERSION misses by definitions like:
RDEPENDS_${PN} = "package-foo ( = 1.2.3 )"
but this doesn't quite provide the desired behaviour, as the version miss error seems to occur typically at very late point of bitbaking process, namely do_rootfs @ satisfy_dependencies_for. 
If version misses would break the build already when bitbake performs initial recipe parsing, right after it is started, it wouldn't take potentially several hours to detect the version miss.
Furthermore packages which are built, but aren't installed to target image don't seem to be able to utilize the above RDEPENDS_${PN} approach.

Thus I respectfully call for a method to fatalize situations when a PREFERRED_VERSION of package is unavailable, as in current "fallback" behaviour bitbake (despite reporting such incidences) happily continues building non-preferred version of the package (as long as there is another version of package/recipe available to fall back to).

I briefly looked into this and found that the following addition, which introduces a new PREFERRED_VERSIONS_ARE_REQUIRED (feel free to change to more apposite name) variable, would add an optional "quick fatalization" behaviour in case of PREFERRED_VERSION miss to bitbake:

diff --git a/bitbake/lib/bb/providers.py b/bitbake/lib/bb/providers.py
index 637e1fa..b6bf2ba 100644
--- a/bitbake/lib/bb/providers.py
+++ b/bitbake/lib/bb/providers.py
@@ -174,6 +174,9 @@ def findPreferredProvider(pn, cfgData, dataCache, pkg_pn = None, item = None):
             if available_vers:
                 available_vers.sort()
                 logger.info("versions of %s available: %s", pn, ' '.join(available_vers))
+            preferred_required = localdata.getVar('PREFERRED_VERSIONS_ARE_REQUIRED', True)
+            if preferred_required and preferred_required not in ("0", "False"):
+                bb.fatal("PREFERRED_VERSIONS_ARE_REQUIRED is set, preferred version not found for %s" % pn)
         else:
             logger.debug(1, "selecting %s as PREFERRED_VERSION %s of package %s%s", preferred_file, pv_str, pn, itemstr)


For completeness; I also thought that another, perhaps more appropriate way to fulfill this feature gap could maybe be to add a new REQUIRED_VERSION_packagename = "<version>" (in parallel to existing PREFERRED_VERSION_packagename = "<version>") support in bitbake. But this I didn't have the time to look further into currently.
Comment 1 Charles Davies 2021-02-14 11:05:43 UTC
My initial thoughts on this are the following; the reporter's first suggestion of PREFERRED_VERSIONS_ARE_REQUIRED has too much of a global effect on the PREFERRED_VERSION behaviour.

The solution that I would propose is to follow the reporter's second suggestion and introduce a REQUIRED_VERSION_<packagename> alongside PREFERRED_VERSION_<packagename>. This solution allows for a party to enforce a particular version of a package if they desire whilst still maintaining the behaviour (and usefulness) of PREFERRED_VERSION.

As for the reporter's requirement to fail quickly, on initial recipe parsing, changes to the area of code (bitbake/lib/bb/providers.py) the reporter mentions would have this effect.

Going forward I will add the necessary code to facilitate the use of REQUIRED_VERSION if this is deemed to be a suitable solution?
Comment 2 Randy MacLeod 2021-02-16 21:37:14 UTC
Hi Charles,
Since no one has replied, please send a comment or patch to the list:
  https://lists.openembedded.org/g/bitbake-devel
Thanks,
../Randy
Comment 3 Charles Davies 2021-02-17 00:33:40 UTC
Hi Randy,

I have gone ahead and started to work on a patch which adds a REQUIRED_VERSION variable and will submit patch when ready to mailing list. This will be a good starting point for comments regarding the best approach to solve this bug.

Kind Regards,

Charles