Bug 15655

Summary: reproducibility test does not display actual error log in case of build failure
Product: [QA/Testing] Functional (self) Testing Reporter: Yoann Congal <yoann.congal>
Component: oe-selftestAssignee: Yoann Congal <yoann.congal>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: randy.macleod, richard.purdie
Version: unspecified   
Target Milestone: 5.2 M2   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Yoann Congal 2024-11-20 23:20:42 UTC 
[0] reworked reproducibility test output in case of a build failure. But this new display may omit error messages which is hard to debug.

For example: https://valkyrie.yoctoproject.org/#/builders/37/builds/500/steps/12/logs/stdio
> 2024-11-20 20:19:58,394 - oe-selftest - INFO -     NOTE: Tasks Summary: Attempted 21515 tasks of which 3874 didn't need to be rerun and all succeeded.
> 2024-11-20 20:19:58,394 - oe-selftest - INFO -     
> 2024-11-20 20:19:58,395 - oe-selftest - INFO -     Summary: There were 3 ERROR messages, returning a non-zero exit code.
The ERROR messages are not displayed.

In the full log [1]:
ERROR: tzcode-native-2024b-r0 do_populate_sysroot_setscene: Error executing a python function in exec_func_python() autogenerated:
WARNING: Logfile for failed setscene task is /srv/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB-extended/tmp/work/x86_64-linux/tzcode-native/2024b/temp/log.do_populate_sysroot_setscene.1543184
WARNING: Setscene task (/srv/pokybuild/yocto-worker/reproducible/build/meta/recipes-extended/timezone/tzcode-native.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead
ERROR: expect-native-5.45.4-r0 do_populate_sysroot_setscene: Error executing a python function in exec_func_python() autogenerated:
WARNING: Logfile for failed setscene task is /srv/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB-extended/tmp/work/x86_64-linux/expect-native/5.45.4/temp/log.do_populate_sysroot_setscene.1543937
WARNING: Setscene task (virtual:native:/srv/pokybuild/yocto-worker/reproducible/build/meta/recipes-devtools/expect/expect_5.45.4.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead
ERROR: python3-cryptography-native-43.0.1-r0 do_populate_sysroot_setscene: Error executing a python function in exec_func_python() autogenerated:
WARNING: Logfile for failed setscene task is /srv/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB-extended/tmp/work/x86_64-linux/python3-cryptography-native/43.0.1/temp/log.do_populate_sysroot_setscene.1549616
WARNING: Setscene task (virtual:native:/srv/pokybuild/yocto-worker/reproducible/build/meta/recipes-devtools/python/python3-cryptography_43.0.1.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead

setscene tasks failure (from an unrelated connection timeout issue) make bitbake return an error code but the task is not listed in failure since the "real"(not setscene) task succeeded.

[0]: https://git.yoctoproject.org/poky/commit/?id=5a9a5ad6a7bec5d07ed0d1efc0e62919498d5dbf
[1]: https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20241120-5c59kuil/bitbake-reproducibleB-extended.log
Comment 1 Yoann Congal 2024-11-20 23:24:58 UTC 
(CC'ing Richard since he reported it)

@Richard, any advice on how to handle this?
My first thought is to extract lines matching "^ERROR:.*" from logs and display those after the normal logs. Is that a good idea?
Comment 2 Randy MacLeod 2024-11-21 15:44:58 UTC 
Yoann to add a message to point the full log.
Also dump the error with ~10 lines of context around the error.
Comment 3 Yoann Congal 2025-01-15 23:03:06 UTC 
Fix merged:
selftest/reproducible: Print full log path on failure - openembedded-core - OpenEmbedded Core layer
https://git.openembedded.org/openembedded-core/commit/?id=1c2287849a2742445d80c81d6a1c6c3e3668765e
selftest/reproducible: Print error context on failure - openembedded-core - OpenEmbedded Core layer
https://git.openembedded.org/openembedded-core/commit/?id=b3bc820e58b65a976e0abce85fcdffd067697ecf