Bug 14344

Summary: bitbake/OE "latches" sstate even with taskdep changes
Product: [Build System, Metadata & Runtime] OE-Core Reporter: jandryuk
Component: kernelAssignee: Bruce Ashfield <bruce.ashfield>
Status: RESOLVED NOTABUG QA Contact:
Severity: normal    
Priority: Undecided CC: randy.macleod, tom.zanussi
Version: unspecified   
Target Milestone: ---   
Hardware: x86   
OS: x86_64   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description jandryuk 2021-04-14 15:04:27 UTC
Does OE/Bitbake "latch" its decision to use sstate and not re-evaluate if the dependent taskdeps change?

We have builds using sstate, externalsrc and rm_work.  We build signed kernel modules letting Linux auto generate a key.  This key is stored in STAGING_KERNEL_BUILDDIR via do_shared_workdir and it is used to sign out-of-tree, externalsrc modules.

Sometimes the externalsrc modules are rebuilt and don't have a matching key, so they cannot load.

The scenario seems to be:
virtual/kernel (linux-openxt) is selected from sstate
argo-module needs to be rebuilt since exernalsrc and MACHINE changed
argo-module -> depends on make-mod-scripts -> virtual/kernel:do_shared_workdir -> virtual/kernel:do_compile

The kernel recompiles and generates a new signing key, which is used for argo-module.  However, the newly re-compiled kernel is not packaged, so the old one from sstate is re-used.  This leads to the signature mismatch.

We added the following to linux-openxt to (try to) prevent such a scenario:
def get_signing_key(d):
    path = os.path.join(d.getVar("B"),"certs","signing_key.x509")
    return path + ":" + str(os.path.exists(path))

do_shared_workdir[file-checksums] = "${@get_signing_key(d)}"

However, the OE/bitbake decision to use sstate for virtual/kernel seems to be "latched" early on and not re-evaluated even though the task hashes for virtual/kernel changed.  Is this expected?

Thanks,
Jason
Comment 1 Randy MacLeod 2021-04-15 14:37:27 UTC
We have to resolve task hashes in advance of builds, ie. at parse time.
So we don't support changing task hashes part way through the build.