Bug 15798

Summary: reproducible: in case of an error, the saved log does not contain detailed task log
Product: [QA/Testing] Functional (self) Testing Reporter: Yoann Congal <yoann.congal>
Component: oe-selftestAssignee: Yoann Congal <yoann.congal>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: randy.macleod
Version: unspecified   
Target Milestone: 5.3 M1   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Yoann Congal 2025-03-16 00:12:36 UTC 
For example: in [0], nodejs fails to build with:
  2025-03-15 07:51:22,878 - oe-selftest - INFO -       ERROR: Logfile of failure stored in: /srv/pokybuild/yocto-worker/reproducible-meta-oe/build/build/build-st-meta-oe/build-st/reproducibleA/tmp/work/core2-64-poky-linux/nodejs/22.13.1/temp/log.do_compile.2854302
  2025-03-15 07:51:22,878 - oe-selftest - INFO -       Log data follows:
  2025-03-15 07:51:22,878 - oe-selftest - INFO -       | DEBUG: Executing python function set_gyp_variables
[...]
  2025-03-15 07:51:22,879 - oe-selftest - INFO -       | ERROR: oe_runmake failed
  2025-03-15 07:51:22,879 - oe-selftest - INFO -       | WARNING: exit code 1 from a shell command.
  2025-03-15 07:51:22,879 - oe-selftest - INFO -       NOTE: recipe nodejs-22.13.1-r0: task do_compile: Failed
  2025-03-15 07:51:22,879 - oe-selftest - INFO -       ERROR: Task (/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/meta-openembedded/meta-oe/recipes-devtools/nodejs/nodejs_22.13.1.bb:do_compile) failed with exit code '1'

But the save logs[1] only contains:
  ERROR: Logfile of failure stored in: /srv/pokybuild/yocto-worker/reproducible-meta-oe/build/build/build-st-meta-oe/build-st/reproducibleA/tmp/work/core2-64-poky-linux/nodejs/22.13.1/temp/log.do_compile.2854302
  NOTE: recipe nodejs-22.13.1-r0: task do_compile: Failed
  ERROR: Task (/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/meta-openembedded/meta-oe/recipes-devtools/nodejs/nodejs_22.13.1.bb:do_compile) failed with exit code '1'

The "Log data follows" is missing and contains precious information.

[0]: https://autobuilder.yoctoproject.org/valkyrie/#/builders/87/builds/36/steps/32/logs/stdio
[1]: https://valkyrie.yocto.io/pub/repro-fail-openembedded/meta-oe/oe-reproducible-20250315-8_7ama2q/bitbake-reproducibleA.log
Comment 1 Yoann Congal 2025-05-11 08:59:20 UTC 
Suspicious code is in https://git.yoctoproject.org/poky/tree/bitbake/lib/bb/ui/knotty.py#n354:
def print_event_log(event, includelogs, loglines, termfilter):
...
        bb.error("Logfile of failure stored in: %s" % logfile)
        if includelogs and not event.errprinted:
            print("Log data follows:")
...
                for line in lines:
                    print(line)

=> print_event_log prints on stdout bypassing the log framework. Refactor this to use the log framework instead?

Note: the commit introducing this function is:
---
commit	24879df071d4803db3d39ae1d5cad852daa92f28
Author :	Paul Eggleton <paul.eggleton@linux.intel.com>  Thu Aug 31 01:30:44 2017
tinfoil: ensure log lines get printed when tasks fail
If a task fails during build_targets(), we need to print out the log
lines as knotty does or the user will be missing information about the
failure.

(This should get some deeper refactoring, but now isn't the time for
that.)
---
Maybe now is the time for that? ;)
Comment 2 Yoann Congal 2025-05-14 21:32:20 UTC 
Patch sent:
[PATCH] knotty: pass failed task logs through the log infrastructure
https://lists.openembedded.org/g/bitbake-devel/message/17611