Bug 16285

Summary: AB-INT: check-layer: unexplained changed signatures
Product: [QA/Testing] Test Plans/Suite Reporter: Mathieu Dubois-Briand <mathieu.dubois-briand>
Component: test-plans/suiteAssignee: Paul Barker <paul>
Status: NEW --- QA Contact:
Severity: normal    
Priority: Medium+ CC: paul, randy.macleod, richard.purdie, yoann.congal
Version: unspecified   
Target Milestone: 6.2 M2   
Hardware: x86   
OS: Multiple   
Whiteboard: AB-INT
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Mathieu Dubois-Briand 2026-05-15 08:49:59 UTC
We got a few failures of check-layer on both master-next and mathieu/master-next in the last days, and I can't understand what is the issue.

INFO: test_signatures (common.CommonCheckLayer.test_signatures)
INFO:  ... FAIL

Stdout:
Loading cache...done.
Loaded 1992 entries from dependency cache.
INFO: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/check-layer/build/layers/openembedded-core/scripts/lib/checklayer/cases/common.py", line 141, in test_signatures
    self.fail('Adding layer %s changed signatures.\n%s' % (self.tc.layer['name'], msg))
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Adding layer meta changed signatures.
59 signatures changed, initial differences (first hash before, second after):
   busybox:do_recipe_qa: 7179671a25cf50a259e667e565b959c78711c629ff4882216a332426f3d6206e -> ae5900e39ac275822744f241eefc7f0e707e43e03fd3107d82b5190e4a4e6da9
      bitbake-diffsigs --task busybox do_recipe_qa --signature 7179671a25cf50a259e667e565b959c78711c629ff4882216a332426f3d6206e ae5900e39ac275822744f241eefc7f0e707e43e03fd3107d82b5190e4a4e6da9
      NOTE: Reconnecting to bitbake server...


So first, I'm surprised as this is intermittent.
Second, I believe I'm missing a point here: does changed signatures when "meta" is added make any sense? I understand "meta" as oe-core base layer, but without it we don't have any recipe, so how can we have signature changes?
Comment 1 Mathieu Dubois-Briand 2026-05-15 08:53:24 UTC
check-layer alma8-vk-1 mathieu/master-next completed at 2026-05-11 08:32:38+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/3797/steps/13/logs/stdio

check-layer ubuntu2204-vk-1 master-next completed at 2026-05-13 14:11:14+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/3811/steps/12/logs/stdio

check-layer debian13-vk-2 master-next completed at 2026-05-14 00:57:06+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/3814/steps/12/logs/stdio

check-layer debian12-vk-3 mathieu/master-next completed at 2026-05-14 19:36:17+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/3819/steps/12/logs/stdio
Comment 2 Mathieu Dubois-Briand 2026-05-16 12:20:54 UTC
check-layer ubuntu2404-vk-3 mathieu/master-next completed at 2026-05-15 16:55:37+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/3839/steps/12/logs/stdio
Comment 3 Richard Purdie 2026-05-19 08:33:45 UTC
I added some debug code to master-next doing two things, firstly printing the exact signature differences and secondly preserving copies of all the locked sigs files. The build with the debug code failed here:

https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/3846

What is interesting is that there are just two signature differences listed:

busybox:do_build changed 24e50a9a588e82050ef54ba288ac26e89b4b87e987685daacb06eb0a9dac1a3d -> c51657bb62a756c888d3acd1a5c6d9314cdc9aabc5e3c39ae1e717b73082bef5
busybox:do_populate_lic changed 4d27231de439cb257cf4ab552f5930f9fa9cd9ea33bbf9439af700c701f5cdc5 -> 363d7616636e03eb7bbfe2fae8da727cd16ca7610a8ee9bb2b937db36e6dbf65

do_build depends on do_populate_lic so that is expected. What is interesting is the do_populate_lic changes alone, no change in do_fetch.

The diffsigs output is therefore misleading as not all those signatures actually changed, just do_populate_lic.

Looking on disk at the failed build directory in ~/yocto-worker/check-layer/build/build/tmp/stamps/x86-64-v3-oe-linux/busybox/

we see:

-rw-rw-r--   1 pokybuild pokybuild 12608 May 19 00:19 1.38.0.do_populate_lic.sigdata.363d7616636e03eb7bbfe2fae8da727cd16ca7610a8ee9bb2b937db36e6dbf65
-rw-rw-r--   1 pokybuild pokybuild 12558 May 18 23:07 1.38.0.do_populate_lic.sigdata.4d27231de439cb257cf4ab552f5930f9fa9cd9ea33bbf9439af700c701f5cdc5

decompressing these with zstd, the piping through jq to add line breaks, we can see:

@@ -478,7 +478,7 @@
   },
   "taskhash": "4d27231de439cb257cf4ab552f5930f9fa9cd9ea33bbf9439af700c701f5cdc5",
   "taskhash_ignore_tasks": null,
-  "unihash": "363d7616636e03eb7bbfe2fae8da727cd16ca7610a8ee9bb2b937db36e6dbf65",
+  "unihash": "4d27231de439cb257cf4ab552f5930f9fa9cd9ea33bbf9439af700c701f5cdc5",
   "varvals": {
     "BP": "${BPN}-${PV}",
     "BPN": "${@oe.utils.prune_suffix(d.getVar('PN'), d.getVar('SPECIAL_PKGSUFFIX').split(), d)}",

i.e. the taskhashes were the same but somehow a unihash match up was found.

Given unihashes should be disabled in check-layer, this looks like the cause of the issues. The question therefore becomes, how is unihash somehow getting briefly enabled during the tests?
Comment 4 Yoann Congal 2026-05-19 21:51:20 UTC
check-layer opensuse160-vk-1 wrynose completed at 2026-05-19 16:38:08+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/3851
Comment 5 Antonin Godard 2026-05-22 13:49:42 UTC
check-layer ubuntu2404-vk-1 master-next&master completed at 2026-05-19 00:23:39+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/3846/steps/12/logs/stdio

check-layer opensuse156-vk-1 master completed at 2026-05-19 02:24:19+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/3847/steps/13/logs/stdio
Comment 6 Mathieu Dubois-Briand 2026-06-16 07:23:49 UTC
check-layer debian12-vk-2 mathieu/master-next-tests&mathieu/master-next completed at 2026-06-15 09:24:13+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/3997/steps/12/logs/stdio

check-layer alma8-vk-2 master&master-next completed at 2026-06-12 14:55:38+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/3983/steps/13/logs/stdio
Comment 7 Yoann Congal 2026-06-19 07:22:23 UTC
check-layer debian12-vk-4 contrib/ycongal/wrynose-nut completed at 2026-06-18 17:21:02+00:00
https://autobuilder.yoctoproject.org/valkyrie/?#/builders/27/builds/4025
Comment 8 Yoann Congal 2026-07-04 21:28:13 UTC
check-layer alma8-vk-2 wrynose completed at 2026-07-04 14:25:01+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/4138
Comment 9 Mathieu Dubois-Briand 2026-07-08 12:29:08 UTC
check-layer ubuntu2204-vk-3 master&master-next completed at 2026-07-06 22:53:23+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/4158/steps/12/logs/stdio

check-layer debian12-vk-5 master&master-next completed at 2026-07-07 12:40:30+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/4162/steps/12/logs/stdio
Comment 10 Mathieu Dubois-Briand 2026-07-21 09:09:04 UTC
check-layer alma8-vk-2 stable/2.18-nut&stable/wrynose-nut completed at 2026-07-04 14:25:01+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/4138/steps/13/logs/stdio
Comment 11 Mathieu Dubois-Briand 2026-07-22 15:09:00 UTC
check-layer debian11-vk-1 mathieu/master-next completed at 2026-07-21 19:23:20+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/4247/steps/13/logs/stdio
Comment 12 Mathieu Dubois-Briand 2026-08-20 05:34:35 UTC
check-layer alma10-vk-1 mathieu/master-next completed at 2026-08-19 10:23:53+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/4494/steps/14/logs/stdio

This was on a branch that failed to build, but looks unrelated to these failures.
Comment 13 Mathieu Dubois-Briand 2026-08-27 06:21:11 UTC
check-layer rocky10-vk-1 mathieu/master-next-tests&mathieu/master-next completed at 2026-08-26 19:56:36+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/4563/steps/14/logs/stdio
Comment 14 Yoann Congal 2026-10-19 06:47:15 UTC
check-layer ubuntu2204-vk-1 wrynose completed at 2026-10-18 23:59:44+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/4917


INFO: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/check-layer/build/layers/openembedded-core/scripts/lib/checklayer/cases/common.py", line 141, in test_signatures
    self.fail('Adding layer %s changed signatures.\n%s' % (self.tc.layer['name'], msg))
AssertionError: Adding layer meta changed signatures.
59 signatures changed, initial differences (first hash before, second after):
   busybox:do_recipe_qa: b7d75a012f50702647f54f4f2ca302a78368e9582af8160a91d07bd622a54e04 -> ae5900e39ac275822744f241eefc7f0e707e43e03fd3107d82b5190e4a4e6da9
      bitbake-diffsigs --task busybox do_recipe_qa --signature b7d75a012f50702647f54f4f2ca302a78368e9582af8160a91d07bd622a54e04 ae5900e39ac275822744f241eefc7f0e707e43e03fd3107d82b5190e4a4e6da9
      NOTE: Reconnecting to bitbake server...
Comment 15 Mathieu Dubois-Briand 2026-10-19 07:36:03 UTC
check-layer debian11-vk-3 mathieu/master-next completed at 2026-10-17 08:57:36+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/4911/steps/15/logs/stdio
Comment 16 Mathieu Dubois-Briand 2026-10-26 10:01:27 UTC
check-layer stream9-vk-1 master completed at 2026-10-25 02:01:38+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/4961/steps/15/logs/stdio
Comment 17 Mathieu Dubois-Briand 2026-11-02 10:46:47 UTC
A different but similar error now:
check-layer rocky8-vk-1 mathieu/master-next-tests&mathieu/master-next completed at 2026-10-27 10:44:23+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/4970/steps/15/logs/stdio

check-layer fedora45-vk-1 master completed at 2026-10-28 01:42:58+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/4974/steps/14/logs/stdio
Comment 18 Yoann Congal 2026-11-03 09:08:23 UTC
check-layer debian11-vk-1 blacksail completed at 2026-11-02 23:52:39+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/5005

| basehash changed from 63f194f6592c45ecc0fea61c19aaf35d00f6450266d773f00def99ac3a38f6c4 to 73cd7f03b0b23f4176562e05b09f3365d2e451187cec3143a0b0883726d93676
| Variable SRC_URI value changed:
| "https://busybox.net/downloads/busybox-${PV}.tar.bz2;name=tarball            [...]            [-file://sha_accel.cfg-]"

Note: sha_accel.cfg is added via a x86_64 override
| meta/recipes-core/busybox/busybox_1.38.0.bb
| 65:SRC_URI:append:x86-64 = " file://sha_accel.cfg"
Comment 19 Mathieu Dubois-Briand 2026-11-09 08:53:56 UTC
check-layer ubuntu2404-vk-3 master completed at 2026-11-08 01:44:13+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/5048/steps/18/logs/stdio
Comment 20 Mathieu Dubois-Briand 2026-11-11 15:51:24 UTC
check-layer rocky10-vk-1 master completed at 2026-11-11 02:09:14+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/27/builds/5064/steps/14/logs/stdio