Bug 14774

Summary: Fatal Error when running bitbake --dump-signatures=printdiff in an initramfs multiconfig environment
Product: [Build System, Metadata & Runtime] BitBake Reporter: Gregory Lumen <gregorylumen>
Component: bitbakeAssignee: Richard Purdie <richard.purdie>
Status: RESOLVED FIXED QA Contact:
Severity: minor    
Priority: Medium+ CC: gregorylumen, poky.bs.watcher, poky.watcher, randy.macleod
Version: unspecified   
Target Milestone: 4.1 M1   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Gregory Lumen 2022-04-05 23:37:08 UTC
I am building a multiconfig with an initramfs config alongside the normal device config.

Each config has a separate TMPDIR.

The device config includes:
  INITRAMFS_IMAGE_BUNDLE = "1"
  INITRAMFS_IMAGE = "core-image-tiny-initramfs"`

Where core-image-tiny-initramfs is built as part of the initramfs config.

When I attempt to run `bitbake --dump-signatures=printdiff device-image` it hits the following fatal error:

  ERROR: Can't find a task we're supposed to have written out? (hash: <hash>)?

Where the hash corresponds to the signature for the `core-image-tiny-initramfs:do_rootfs` task.

Based on my limited debugging so far, it appears that `print_diffscenetasks` (poky\bitbake\lib\bb\runqueue.py) is searching under the TMPDIR for the device config, yet the stampsfile is being correctly generated under the TMPDIR for the initramfs config.
Comment 1 Gregory Lumen 2022-04-06 16:45:32 UTC
The following patch appears to at least partially fix the issue (though I'll admit to meddling with forces I do not understand):



From 2b612ffb6b0378a9e1fb0fe0bde8eab07905431d Mon Sep 17 00:00:00 2001
From: Gregory Lumen <gregorylumen@microsoft.com>
Date: Wed, 6 Apr 2022 16:33:09 +0000
Subject: [PATCH] Runqueue: Attempting mc support for write_diffscenetasks

---
 bitbake/lib/bb/runqueue.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index a4bcab64d0..c06f32e5bf 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -1695,7 +1695,7 @@ class RunQueue:
             (mc, fn, taskname, taskfn) = split_tid_mcfn(tid)
             pn = self.rqdata.dataCaches[mc].pkg_fn[taskfn]
             h = self.rqdata.runtaskentries[tid].hash
-            matches = bb.siggen.find_siginfo(pn, taskname, [], self.cfgData)
+            matches = bb.siggen.find_siginfo(pn, taskname, [], self.cooker.databuilder.mcdata[mc])
             match = None
             for m in matches:
                 if h in m:
-- 
2.25.1
Comment 2 Gregory Lumen 2022-04-07 16:53:00 UTC
I also found the following patch necessary before dump-signatures could successfully locate some of the SSTATE siginfo files needed for comparison:

From 2d89705c97fedc54b59a9ddedeabe19341901a14 Mon Sep 17 00:00:00 2001
From: Gregory Lumen <gregorylumen@microsoft.com>
Date: Wed, 6 Apr 2022 21:16:59 +0000
Subject: [PATCH] Fixing find_siginfo

---
 bitbake/lib/bb/runqueue.py | 2 ++
 meta/lib/oe/sstatesig.py   | 5 +++--
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index 9d6d44ae35..ff959abbb2 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -1708,6 +1708,8 @@ class RunQueue:
                 prevh = __find_sha256__.search(latestmatch).group(0)
                 output = bb.siggen.compare_sigfiles(latestmatch, match, recursecb)
                 bb.plain("\nTask %s:%s couldn't be used from the cache because:\n  We need hash %s, closest matching task was %s\n  " % (pn, taskname, h, prevh) + '\n  '.join(output))
+            else:
+                bb.plain("\nTask %s:%s not found in the cache" % (pn, taskname))
 
 
 class RunQueueExecute:
diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index c2e3e2f4f5..9768b00fc4 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -382,15 +382,16 @@ def find_siginfo(pn, taskname, taskhashlist, d):
             localdata.setVar('PV', '*')
             localdata.setVar('PR', '*')
             localdata.setVar('BB_TASKHASH', hashval)
+            localdata.setVar('SSTATE_CURRTASK', taskname[3:])
             swspec = localdata.getVar('SSTATE_SWSPEC')
             if taskname in ['do_fetch', 'do_unpack', 'do_patch', 'do_populate_lic', 'do_preconfigure'] and swspec:
                 localdata.setVar('SSTATE_PKGSPEC', '${SSTATE_SWSPEC}')
             elif pn.endswith('-native') or "-cross-" in pn or "-crosssdk-" in pn:
                 localdata.setVar('SSTATE_EXTRAPATH', "${NATIVELSBSTRING}/")
-            sstatename = taskname[3:]
-            filespec = '%s_%s.*.siginfo' % (localdata.getVar('SSTATE_PKG'), sstatename)
+            filespec = '%s.siginfo' % localdata.getVar('SSTATE_PKG')
 
             matchedfiles = glob.glob(filespec)
+
             for fullpath in matchedfiles:
                 actual_hashval = get_hashval(fullpath)
                 if actual_hashval in hashfiles:
-- 
2.25.1
Comment 3 Richard Purdie 2022-04-28 14:14:39 UTC
Thanks for these, they look like the correct fixes to me. Since I'd spent the time to check them, I've written up commit messages and sent the patches out for wider review/testing. Please feel free to submit patches like this on the mailing list in future!