Bug 15798 - reproducible: in case of an error, the saved log does not contain detailed task log
Summary: reproducible: in case of an error, the saved log does not contain detailed ta...
Status: RESOLVED FIXED
Alias: None
Product: Functional (self) Testing
Classification: QA/Testing
Component: oe-selftest (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 5.3 M1
Assignee: Yoann Congal
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2025-03-16 00:12 UTC by Yoann Congal
Modified: 2025-05-18 16:44 UTC (History)
1 user (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Yoann Congal 2025-03-16 00:12:36 UTC 
For example: in [0], nodejs fails to build with:
  2025-03-15 07:51:22,878 - oe-selftest - INFO -       ERROR: Logfile of failure stored in: /srv/pokybuild/yocto-worker/reproducible-meta-oe/build/build/build-st-meta-oe/build-st/reproducibleA/tmp/work/core2-64-poky-linux/nodejs/22.13.1/temp/log.do_compile.2854302
  2025-03-15 07:51:22,878 - oe-selftest - INFO -       Log data follows:
  2025-03-15 07:51:22,878 - oe-selftest - INFO -       | DEBUG: Executing python function set_gyp_variables
[...]
  2025-03-15 07:51:22,879 - oe-selftest - INFO -       | ERROR: oe_runmake failed
  2025-03-15 07:51:22,879 - oe-selftest - INFO -       | WARNING: exit code 1 from a shell command.
  2025-03-15 07:51:22,879 - oe-selftest - INFO -       NOTE: recipe nodejs-22.13.1-r0: task do_compile: Failed
  2025-03-15 07:51:22,879 - oe-selftest - INFO -       ERROR: Task (/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/meta-openembedded/meta-oe/recipes-devtools/nodejs/nodejs_22.13.1.bb:do_compile) failed with exit code '1'

But the save logs[1] only contains:
  ERROR: Logfile of failure stored in: /srv/pokybuild/yocto-worker/reproducible-meta-oe/build/build/build-st-meta-oe/build-st/reproducibleA/tmp/work/core2-64-poky-linux/nodejs/22.13.1/temp/log.do_compile.2854302
  NOTE: recipe nodejs-22.13.1-r0: task do_compile: Failed
  ERROR: Task (/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/meta-openembedded/meta-oe/recipes-devtools/nodejs/nodejs_22.13.1.bb:do_compile) failed with exit code '1'

The "Log data follows" is missing and contains precious information.

[0]: https://autobuilder.yoctoproject.org/valkyrie/#/builders/87/builds/36/steps/32/logs/stdio
[1]: https://valkyrie.yocto.io/pub/repro-fail-openembedded/meta-oe/oe-reproducible-20250315-8_7ama2q/bitbake-reproducibleA.log
Comment 1 Yoann Congal 2025-05-11 08:59:20 UTC 
Suspicious code is in https://git.yoctoproject.org/poky/tree/bitbake/lib/bb/ui/knotty.py#n354:
def print_event_log(event, includelogs, loglines, termfilter):
...
        bb.error("Logfile of failure stored in: %s" % logfile)
        if includelogs and not event.errprinted:
            print("Log data follows:")
...
                for line in lines:
                    print(line)

=> print_event_log prints on stdout bypassing the log framework. Refactor this to use the log framework instead?

Note: the commit introducing this function is:
---
commit	24879df071d4803db3d39ae1d5cad852daa92f28
Author :	Paul Eggleton <paul.eggleton@linux.intel.com>  Thu Aug 31 01:30:44 2017
tinfoil: ensure log lines get printed when tasks fail
If a task fails during build_targets(), we need to print out the log
lines as knotty does or the user will be missing information about the
failure.

(This should get some deeper refactoring, but now isn't the time for
that.)
---
Maybe now is the time for that? ;)
Comment 2 Yoann Congal 2025-05-14 21:32:20 UTC 
Patch sent:
[PATCH] knotty: pass failed task logs through the log infrastructure
https://lists.openembedded.org/g/bitbake-devel/message/17611