Bug 8699

Summary: offer a way to use sstate-object hashes as dependencies of builds/packages
Product: [Build System, Metadata & Runtime] BitBake Reporter: Gustavo Sverzut Barbieri <barbieri>
Component: bitbakeAssignee: Unassigned <unassigned>
Status: RESOLVED OBSOLETE QA Contact: Apoorv <apoorv.sangal>
Severity: enhancement    
Priority: Medium CC: alexandru.c.georgescu, jose.perez.carranza, joshuagloe, poky.bs.watcher, poky.watcher, randy.macleod, richard.purdie
Version: 5.99   
Target Milestone: Future   
Hardware: All   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Gustavo Sverzut Barbieri 2015-11-18 00:43:13 UTC
Yocto offers a powerful feature that allows one to easily modify how the package is built. It is simple to change CFLAGS/LDFLAGS, or to enable/disable a compile-time featue with PACKAGECONFIG or IMAGE/DISTRO_FEATURES.

However, these can result in incompatible packages. Say my application uses curl to access a server over SSL/IPv6, there is no way to get an libcurl-ssl-ipv6-7.44.rpm, it will only output libcurl-7.44.rpm. Thus a change to configuration would produce a new curl build that breaks my application.

Gentoo had the exact same problem ages ago, they solved by allowing depending on packages+use-flags. But it's all source-based, then it's okay.

However, if someone is deploying an Yocto-generated binary distro, there is no way to know if we're going to break an exiting application or not. This is a particular worry for 3rd party applications (ISV). Then depending solely on the package name+version is not enough.

My proposal is to have Yocto (likely via bitbake) to output the sstate-object hashes that were created with a given SDK/Image, and for individual packages (ie: applications) to list those that it used during build.

Then people could validate if the set of required hashes is contained into a given image before installing (an extra validation for those that care).

This would make the dependency much more sensitive.
Comment 1 Joshua Lock 2016-03-29 16:19:56 UTC
We have a possible solution for this issue already, perhaps it's sufficient?

We can generate a locked-sigs include file with:

bitbake my-image-name -S none

which produces locked-sigs.inc in the build directory.

If you then include that file, say in local.conf:

include ../locked-sigs.inc

Future builds where the sstate signature of a package has changed will result in an error. A snipped example from a test here:

The util-linux:do_build sig (4847af183e8a78714aee4d3d3e1813c0) changed, use locked sig 14e39f1ab2f95fad5cdfacaa8d4ea6e1 to instead
The libpam:do_build sig (caf45d5c74b4709c05de993756fb94cb) changed, use locked sig a9254c62e6fa42809c349ce9f7a7ba3f to instead
The kbd:do_build sig (681a3d1fc829482a371c39bd62b2aa2f) changed, use locked sig 3bcb74e6b5962cb1511d59cc35c8c09f to instead
The e2fsprogs:do_packagedata sig (7e5195dbf24226853edebfc80d098ee4) changed, use locked sig ae2103a61fd293c172da6282030d02cd to instead
The libcgroup:do_build sig (d3e0c508f0a445fa26645854544c21bf) changed, use locked sig 22e166edb793e1bd27f0f8a052e81b26 to instead
The dbus:do_build sig (9f4a2c65cbd47bb010cd0334b94b136b) changed, use locked sig a9d13838c176c7084490237cfca15242 to instead
The e2fsprogs:do_build sig (96bd6fddcc93694d4e8c27e0bff28d35) changed, use locked sig d72cda97410bc402caa25d00c0f289f8 to instead
The systemd:do_build sig (d9898b810c9f0ea22a2835d9ba7a1f29) changed, use locked sig a8feaceacbdefb49757fad6646181193 to instead

Additiaonlly, we can change the error (which prevents a build) to a warning with:

SIGGEN_LOCKEDSIGS_CHECK_LEVEL="warn"

I believe this provides a solution to the request to emit the sstate hashes (the locked-sigs include) and also demonstrates extra mechanisms we have already built around that.

At the very least we need to make some documentation updates to better demonstrate this functionality.
Comment 2 Gustavo Sverzut Barbieri 2016-03-29 23:18:39 UTC
looks like that is the building block, but we'd need to work on the user experience, not just document that.

For instance always generate the locked-sigs.inc, list just the packages that fail the signatures (not the actual signature), ask for confirmation and if confirmed then it would update locked-sigs.inc automatically and proceed. Something like:

    $ bitbake myimage-with-changes
    The following packages will be rebuilt:
       util-linux
       libpam
       kbd
    Do you want to continue [n/Y]?

Ideally we if we could pin-point why it happened, we could use a "--verbose" switch to produce something like:

    $ bitbake myimage-with-changes --verbose
    The following packages will be rebuilt:
       util-linux (CFLAGS: added -Wl,--gc-sections)
       libpam (PACKAGECONFIG: added xpto)
       kbd (changed dependency libXPTO)
    Do you want to continue [n/Y]?

Answering "Y" or a "--no-confirm" switch would proceed with build and would update the lock-sigs.
Comment 3 Gustavo Sverzut Barbieri 2016-04-12 19:40:52 UTC
it seems this is related to: https://bugzilla.yoctoproject.org/show_bug.cgi?id=6428
Comment 4 Joshua Lock 2016-12-06 16:13:38 UTC
Resetting assignee, importance and status so that this can be reassessed by the triage team.
Comment 5 Armin Kuster 2019-12-22 20:36:29 UTC
isn't this reusable sstate?

we are in the middle of finishing this up are we not?
Comment 6 Randy MacLeod 2020-01-30 16:26:36 UTC
Implementation of bblock/bblunlock tool is prefered. A defect for that exists.