Bug 15880 - AB-INT: barebox: fixdep: error opening file
Summary: AB-INT: barebox: fixdep: error opening file
Status: RESOLVED WORKSFORME
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 5.99
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 5.3 M4
Assignee: Enrico Jorns
QA Contact:
URL:
Whiteboard: AB-INT
Depends on:
Blocks:
 
Reported: 2025-05-28 07:10 UTC by Mathieu Dubois-Briand
Modified: 2026-01-22 16:16 UTC (History)
5 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mathieu Dubois-Briand 2025-05-28 07:10:56 UTC
Maybe a race condition in barebox Makefiles?

2025-05-28 01:48:07,246 - oe-selftest - INFO - barebox.BareboxTest.test_boot_barebox (subunit.RemotedTestCase)
2025-05-28 01:48:07,248 - oe-selftest - INFO -  ... FAIL
...
ERROR: barebox-2025.04.0-r0 do_compile: oe_runmake failed
...
| fixdep: error opening file: arch/arm/dts/.imx8mq-zii-ultra-rmb3.dtb.pbl.o.d: No such file or directory
| make[2]: *** [/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1221065/tmp/work/qemuarm64-poky-linux/barebox/2025.04.0/barebox-2025.04.0/scripts/Makefile.build:187: arch/arm/dts/imx8mq-zii-ultra-rmb3.dtb.pbl.o] Error 2
| make[2]: *** Deleting file 'arch/arm/dts/imx8mq-zii-ultra-rmb3.dtb.pbl.o'
| make[1]: *** [/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1221065/tmp/work/qemuarm64-poky-linux/barebox/2025.04.0/barebox-2025.04.0/Makefile:1057: arch/arm/dts] Error 2
| make[1]: *** Waiting for unfinished jobs....
Comment 1 Mathieu Dubois-Briand 2025-05-28 07:21:43 UTC
oe-selftest-armhost ubuntu2204-vk-arm2 master completed at 2025-05-28T05:27:08Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1787/steps/14/logs/stdio
Comment 2 Randy MacLeod 2025-05-29 14:40:30 UTC
Seems like  race in the Makefile...
Comment 3 Enrico Jorns 2025-06-02 08:51:19 UTC
Oh, haven't seen this before, thank you for notifying!

At least with latest master I wasn't able to reproduce it, yet. But I've contacted the maintainers and we'll try to identify what's going on.

Did this happen only once so far? Or multiple times?
Comment 4 Randy MacLeod 2025-06-02 17:48:51 UTC
Enrico,

It's good that you notified upstream. Hopefully they can see and fix the race.

I think it happened only once so far but Mathieu will correct me if I'm wrong.

This is known as an "AB-INT" / "auto-builder intermittent" bug - fun, eh!

The YP autobuilder runs 2 or 3 builds at a time that are either building code or running various tests so the load can be very high. You can try using:
$ stress -t TIMEOUT_TO_SAVE_ME -c <2x v cores> 
or similar in an overnight stress tests while building and cleanall-ing your recipe in a tight loop.

The smarter thing to do is to study the Makefile and insert a delay 
to make the rare race be deterministic. Smarter and harder but it's a very
useful skill to develop.
Comment 5 Mathieu Dubois-Briand 2025-06-03 07:45:37 UTC
> I think it happened only once so far but Mathieu will correct me if I'm wrong.

Yes, only happened once so far.
Comment 6 Ross Burton 2025-06-03 09:11:21 UTC
This looks like a missing makefile dependency to me, which modern GNU Make has options which can help.  Copy this class and inherit shufflemake in the recipe and make will randomly re-order task execution (but still keeping the dependencies).
Comment 7 Randy MacLeod 2025-06-03 14:54:21 UTC
Nice, I didn't know about that feature. It is only ~ 2 years old...

make.git on master
❯ git log --oneline -1 621d3196f
621d3196 [SV 62100] Add '--shuffle' option support

❯ git tag --contains 621d3196f
4.3.90
4.3.91
4.3.92
4.4
4.4.0.90
4.4.0.91
4.4.1

There's also a commandline option: --shuffle
https://www.gnu.org/software/make/manual/make.html#index-_002d_002dshuffle
Comment 8 Ross Burton 2025-06-03 14:55:40 UTC
Yeah, two years old is too new for a host dependency, thus the class to pull in the native dependencies and pass the option globally.
Comment 9 Randy MacLeod 2025-08-21 14:51:54 UTC
Enrico, Can you try to reproduce this using the new make features?
Comment 10 Ross Burton 2025-08-21 14:54:08 UTC
I failed to link to the class I was talking about:

https://github.com/rossburton/meta-ross/blob/master/classes/shufflemake.bbclass

That's just:

BASEDEPENDS:append:class-target = ' make-native'
EXTRA_OEMAKE:append:class-target = " --shuffle"
Comment 11 Randy MacLeod 2025-08-21 14:59:15 UTC
Thanks for atttaching the patch.
Could you send the patch to the email list ?
https://docs.yoctoproject.org/contributor-guide/index.html

Please add a test if possible, and ensure that you have a good commit log and Signed-off-by line.
Comment 12 Ahmad Fatoum 2025-09-01 15:54:08 UTC
There is now a fix for this issue on the barebox mailing list:
https://lore.kernel.org/barebox/20250828093135.1959085-1-s.hauer@pengutronix.de

It's expected that this will be part of this month's release.
Given that this seems to have broken the Yocto autobuilder only once over many months, I'd suggest just to wait out the two~ weeks or so until there is a v2025.09.0 release.

While make --shuffle was not used to fix this issue, it may be useful to early detect such issues before downstreams run into them.
I used the occasion of Debian Trixie shipped a new enough GNU make to submit patches against upstream CI[1] that run make for the 87 defconfigs that are built on every push with --shuffle.

[1]: https://lore.kernel.org/barebox/20250901153852.2554758-1-a.fatoum@pengutronix.de/
Comment 13 Ahmad Fatoum 2025-10-13 08:38:34 UTC
The issue is now fixed in v2025.09.0. I assume Enrico will submit a version bump or should I?

I want to extend my thanks for the tip around make --shuffle. All our CI builds now use it and just a few days ago it helped eliminate a new build breakage, but this time before it made it into a release:
https://github.com/barebox/barebox/commit/030f0acc230

:)
Comment 14 Randy MacLeod 2026-01-22 16:16:08 UTC
Not seen in > 6 months.
barebox is updated: 2025.12.0
https://git.openembedded.org/openembedded-core/log/?qt=grep&q=barebox