Bug 13212 - image_qa task is always triggered with rm_work class enabled
Summary: image_qa task is always triggered with rm_work class enabled
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: oe-core other (show other bugs)
Version: 3.0.1
Hardware: All Multiple
: Medium+ normal
Target Milestone: 3.0.1
Assignee: Anuj Mittal
QA Contact:
URL:
Whiteboard: with backport
Depends on:
Blocks:
 
Reported: 2019-02-28 16:24 UTC by Romuald Jeanne
Modified: 2019-11-19 00:54 UTC (History)
2 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Romuald Jeanne 2019-02-28 16:24:33 UTC
Preconditions/Environment
-------------------------
Setup default Yocto configuration on THUD release:
DISTRO=nodistro
MACHINE=(any machine)

Add in 'local.conf' file
INHERIT += "rm_work"

Triggering Action/Cause
-----------------------
Once bitbake environment enabled, build the core-image-base several time in a row:
bitbake core-image-base
bitbake core-image-base
bitbake core-image-base

Expectation
-----------
The first build should last for a few hours (depending on your network and host).
Then following builds for the same image should not trigger any specific tasks as the image has been already built.

Actual Result
-------------
After the first build, the other builds trigger each time the 'image_qa' task.

Reproducibility
---------------
1/1

Pre-Analysis:
-------------
In 'rm_work.bbclass' class we should also manage the do_image_qa_setscene and do_image_qa tasks in the same way do_image_complete_setscene and do_image_complete tasks are handled.

I suggest to add in 'rm_work.bbclass' file,
line 66 (just after the switch '*do_image_complete*)' :

 *do_image_qa_setscene*)
     # Ensure we don't 'stack' setscene extensions to this stamp with the section below
     i=dummy
     ;;
 *do_image_qa*)
     # Promote do_image_complete stamps to setscene versions (ahead of *do_image* below)
     mv $i `echo $i | sed -e "s#do_image_qa#do_image_qa_setscene#"`
     i=dummy
     ;;
Comment 1 Joshua Watt 2019-03-07 16:43:33 UTC
If you are not using poky to get bitbake, could you try bringing up bitbake to the latest version? There is a known bug where newer versions of oe-core don't work quite right with older versions of bitbake and it can cause tasks to re-run.
Comment 2 Romuald Jeanne 2019-03-12 10:48:13 UTC
I'm using the Poky release thud-20.0.1 (or yocto-2.6.1)
SHA1: 1cab405d88149fd63322a867c6adb4a80ba68db3

If you look at the 'log.task_order' file for core-image-base build:

* without enabling 'rm_work' class, I have the following log:

do_prepare_recipe_sysroot (24001): log.do_prepare_recipe_sysroot.24001
do_rootfs (24046): log.do_rootfs.24046
do_image_qa (5884): log.do_image_qa.5884
do_image (5888): log.do_image.5888
do_image_stmultiubi (5928): log.do_image_stmultiubi.5928
do_image_tar (5930): log.do_image_tar.5930
do_image_ext4 (5932): log.do_image_ext4.5932
do_image_complete (5982): log.do_image_complete.5982
do_populate_lic_deploy (6110): log.do_populate_lic_deploy.6110

If I launch again the same image build, nothing is added to the 'log.task_order' as there is no task scheduled.

* with 'rm_work' class enabled, I just launch again the same build, and I have the 'log.task_order' file populated with these additional lines:

do_image_complete (4472): log.do_image_complete.4472
do_populate_lic_deploy (4762): log.do_populate_lic_deploy.4762
do_rm_work (4765): log.do_rm_work.4765

Again I launch the same build, and I got the follwoing update:

do_prepare_recipe_sysroot (5027): log.do_prepare_recipe_sysroot.5027
do_rootfs (5082): log.do_rootfs.5082
do_image_qa (13375): log.do_image_qa.13375
do_populate_lic_deploy (13379): log.do_populate_lic_deploy.13379
do_rm_work (13381): log.do_rm_work.13381

And again:

do_prepare_recipe_sysroot (13520): log.do_prepare_recipe_sysroot.13520
do_rootfs (13534): log.do_rootfs.13534
do_image_qa (21395): log.do_image_qa.21395
do_populate_lic_deploy (21399): log.do_populate_lic_deploy.21399
do_rm_work (21401): log.do_rm_work.21401

So when 'rm_work' class is enabled, I have the 'do_image_qa' task that is triggered, while it should not.

And as soon as I apply the correction I've suggested in 'rm_work' class to manage the 'do_image_qa' task in the same way 'do_image_complete' task is handled, then I can get 'log.task_order' file updated in this way:

do_prepare_recipe_sysroot (25833): log.do_prepare_recipe_sysroot.25833
do_rootfs (26144): log.do_rootfs.26144
do_image_qa (1775): log.do_image_qa.1775
do_populate_lic_deploy (1800): log.do_populate_lic_deploy.1800
do_rm_work (1812): log.do_rm_work.1812
[ new build ]
do_populate_lic_deploy (1993): log.do_populate_lic_deploy.1993
do_rm_work (2001): log.do_rm_work.2001
[ new build ]
do_populate_lic_deploy (2124): log.do_populate_lic_deploy.2124
do_rm_work (2131): log.do_rm_work.2131
[ new build ]
do_populate_lic_deploy (7512): log.do_populate_lic_deploy.7512
do_rm_work (7578): log.do_rm_work.7578

I hope I'm clear enought.
Comment 3 Armin Kuster 2019-10-13 19:10:32 UTC
looking at the current rm_work.bbclass in master, I don't see anything like these changes so I suspect this will affect master too.
Comment 4 Jacob Kroon 2019-10-13 19:57:21 UTC
FWIW: I've seen this bug for years using OE. To me it has not been annoying enough for me to complain loudly, since the rootfs tasks that are re-run each time don't take a huge amount of time. But it would be great if this could be fixed. I did try patching rm_work.bbclass according to the bug-report, and it does seem to solve the problem with re-running tasks.
Comment 5 Armin Kuster 2019-10-13 20:31:47 UTC
(In reply to comment #4)
> FWIW: I've seen this bug for years using OE. To me it has not been annoying
> enough for me to complain loudly, since the rootfs tasks that are re-run
> each time don't take a huge amount of time. But it would be great if this
> could be fixed. I did try patching rm_work.bbclass according to the
> bug-report, and it does seem to solve the problem with re-running tasks.

I will change the milestone to 3.1 with backport and see what the Bug triage team thinks of it.

thanks for the info.
Comment 6 Jacob Kroon 2019-11-08 06:30:14 UTC
Fix merged to master:

commit 3d3bd7952cf4bf5c94e4d18b45a604b95b69e8c3
Author: Jacob Kroon <jacob.kroon@gmail.com>
Date:   Wed Nov 6 13:34:58 2019 +0100

    rm_work: Promote do_image_qa stamps to setscene versions
    
    [ YOCTO #13212 ]
    
    Suggested-by: Romuald Jeanne <romuald.jeanne@st.com>
    Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Comment 7 Armin Kuster 2019-11-18 23:56:30 UTC
assigning to anuj to backport for Zeus
Comment 8 Anuj Mittal 2019-11-19 00:54:32 UTC
(In reply to comment #7)
> assigning to anuj to backport for Zeus

This is in zeus already:

https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=zeus&id=4ad64a5cf5a4ce03185120cc6e27a0b985f3244c

Closing.