Bug 12365 - glibc-initial do_fetch failed if rm_work is enabled
Summary: glibc-initial do_fetch failed if rm_work is enabled
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: configuration (show other bugs)
Version: 2.4
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 2.1.4
Assignee: Richard Purdie
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2017-11-20 06:02 UTC by Mingli Yu
Modified: 2017-12-21 15:28 UTC (History)
1 user (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mingli Yu 2017-11-20 06:02:37 UTC
Git rev: master/a17f3ec910366e9e7551fa24fbc07929b9584341

Steps:
1. Download the poky source and set the environment
2. Add INHERIT += "rm_work" in conf/local.conf
3. Run "bitbake glibc-initial:do_fetch glibc-initial:do_rm_work"

It sometimes failed as do_rm_work is a floating task when add the logic "deltask do_build" for glibc-initial in below commit. 

http://cgit.openembedded.org/openembedded-core/commit/?id=04c053d42ab05f77b2d1ca93a0fabae44073d57e
=====================================================================
author	Richard Purdie <richard.purdie@linuxfoundation.org>	2017-04-05 17:27:20 +0100
committer	Richard Purdie <richard.purdie@linuxfoundation.org>	2017-04-08 22:48:05 +0100
commit	04c053d42ab05f77b2d1ca93a0fabae44073d57e (patch)
tree	58227aa83a6f568947415101969bc61cd14768b0
parent	ef16288f3e1e801e5c763ed49fad8eb0c885183e (diff)
download	openembedded-core-04c053d42ab05f77b2d1ca93a0fabae44073d57e.zip
openembedded-core-04c053d42ab05f77b2d1ca93a0fabae44073d57e.tar.gz
openembedded-core-04c053d42ab05f77b2d1ca93a0fabae44073d57e.tar.bz2
glibc/gcc/libgcc-initial: Delete do_build tasks for -initial
We've had a lot of users running into RSS issues where -initial recipes
were being installed into sysroots alongside their counterparts and
causing overlapping files issues.

In general this was through do_build dependencies. Such dependencies are
bad in general and I'd encourage people to compare the taskgraphs with
using a more specific dependency like do_populate_sysroot, do_image_complete
or do_deploy as often the more specific dependency will result in a much
cleaner build.
Regardless, we don't want -initial dependencies getting in the way like
this and there are cases a do_build dependency could make sense.

Deleting the do_build task in these cases makes sense since this is not
a build "endpoint" we'd ever want a user to use, its a behind the scenes
piece of bootstrappping.

Unfortunately to make this work, we need a newer bitbake version which
has a bitbake bug fixed.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
=====================================================================

Previously, do_rm_work depends on the dependencies which do_build depends on as below logic in meta/classes/rm_work.bbclass
=====================================================================
# Determine what do_build depends upon, without including do_build
    # itself or our own special do_rm_work_all.
    deps = set(bb.build.preceedtask('do_build', True, d)) 
    deps.difference_update(('do_build', 'do_rm_work_all'))

    if pn in excludes:
        d.delVarFlag('rm_work_rootfs', 'cleandirs')
        d.delVarFlag('rm_work_populatesdk', 'cleandirs')
    else:
        # Inject do_rm_work into the tasks of the current recipe such that do_build
        # depends on it and that it runs after all other tasks that block do_build,
        # i.e. after all work on the current recipe is done. The reason for taking
        # this approach instead of making do_rm_work depend on do_build is that
        # do_build inherits additional runtime dependencies on
        # other recipes and thus will typically run much later than completion of
        # work in the recipe itself.
        # In practice, addtask() here merely updates the dependencies.
        bb.build.addtask('do_rm_work', 'do_build', ' '.join(deps), d)
=====================================================================

When delete do_build task, the dependencies for do_rm_work miss, and do_rm_work becomes a floating task. And it can result in a race condition between do_fetch and do_rm_work.

The error message as below:
NOTE: recipe glibc-initial-2.26-r0: task do_rm_work: Started
NOTE: recipe glibc-initial-2.26-r0: task do_fetch: Started
ERROR: glibc-initial-2.26-r0 do_fetch: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:extend_recipe_sysroot(d)
     0003:
File: '/buildarea/build1/layers/oe-core/meta/classes/staging.bbclass', lineno: 410, function: extend_recipe_sysroot     0406:    manifests = {}
     0407:    # All files that we're going to be installing, to find conflicts.
     0408:    fileset = {}
     0409:
 *** 0410:    for f in os.listdir(depdir):
     0411:        if not f.endswith(".complete"):
     0412:            continue
     0413:        f = depdir + "/" + f
     0414:        if os.path.islink(f) and not os.path.exists(f):
Exception: FileNotFoundError: [Errno 2] No such file or directory: '/buildarea/build1/build/tmp-glibc/work/i586-oe-linux/glibc-initial/2.26-r0/recipe-sysroot-native/installeddeps'

ERROR: glibc-initial-2.26-r0 do_fetch: Function failed: extend_recipe_sysroot
NOTE: recipe glibc-initial-2.26-r0: task do_rm_work: Succeeded
ERROR: Logfile of failure stored in: /buildarea/build1/build/tmp-glibc/work/i586-oe-linux/glibc-initial/2.26-r0/temp/log.do_fetch.3330
NOTE: recipe glibc-initial-2.26-r0: task do_fetch: Failed