| Summary: | BitBake default 'dirs' in exec_func (${B}) causes build failures | ||
|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] BitBake | Reporter: | Christopher Larson <kergoth> |
| Component: | bitbake | Assignee: | Ross Burton <ross.burton> |
| Status: | VERIFIED FIXED | QA Contact: | Jose Perez C <jose.perez.carranza> |
| Severity: | enhancement | ||
| Priority: | Medium+ | CC: | alexandru.c.georgescu, jose.perez.carranza, poky.bs.watcher, poky.watcher, richard.purdie, ross.burton, sgw |
| Version: | unspecified | ||
| Target Milestone: | 2.2 M2 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | Yes (doc changes required) | |
Discovered that this (via buildhistory_commit) is the reason that a do_unpack causes $(S) to be created.
Decided to comment out the mkdir of ${B} to see what broke.
BOOM.
The assignment to adir of $(B) means that every Python function is chdir()'d into $(B), which doesn't exist. Comment out the assignment and stuff is building now... although how much of those Python functions are making an assumption about cwd I don't know.
Status: poky-contrib:ross/dirs has a patch to bitbake to stop his behaviour, and fixes for the problems it causes are in oe-core as they're found. Too much fallout for M2 this late, moving to 2.2. Fixup patches for oe-core sent, the bitbake patch just needs an AB run. poky-contrib:ross/dirs has some fixes to oe-core and a bitbake patch to solve this. bitbake watchdog -c populate_lic
bitbake watchdog -c clean
bitbake watchdog
Errors since the current working directory no longer exists.
ERROR: watchdog-5.15-r0 do_populate_lic_setscene: Error executing a python function in exec_python_func() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:do_populate_lic_setscene(d)
0003:
File: '/media/build1/poky/meta/classes/license.bbclass', lineno: 674, function: do_populate_lic_setscene
0670:
0671:do_populate_lic_setscene[dirs] = "${LICSSTATEDIR}/${PN}"
0672:do_populate_lic_setscene[cleandirs] = "${LICSSTATEDIR}"
0673:python do_populate_lic_setscene () {
*** 0674: sstate_setscene(d)
0675:}
0676:addtask do_populate_lic_setscene
File: '/media/build1/poky/meta/classes/sstate.bbclass', lineno: 642, function: sstate_setscene
0638: break
0639:
0640:def sstate_setscene(d):
0641: shared_state = sstate_state_fromvars(d)
*** 0642: accelerate = sstate_installpkg(shared_state, d)
0643: if not accelerate:
0644: raise bb.build.FuncFailed("No suitable staging package found")
0645:
0646:python sstate_task_prefunc () {
File: '/media/build1/poky/meta/classes/sstate.bbclass', lineno: 320, function: sstate_installpkg
0316:
0317: for state in ss['dirs']:
0318: prepdir(state[1])
0319: os.rename(sstateinst + state[0], state[1])
*** 0320: sstate_install(ss, d)
0321:
0322: for plain in ss['plaindirs']:
0323: workdir = d.getVar('WORKDIR', True)
0324: src = sstateinst + "/" + plain.replace(workdir, '')
File: '/media/build1/poky/meta/classes/sstate.bbclass', lineno: 272, function: sstate_install
0268: oe.path.copyhardlinktree(state[1], state[2])
0269:
0270: for postinst in (d.getVar('SSTATEPOSTINSTFUNCS', True) or '').split():
0271: # All hooks should run in the SSTATE_INSTDIR
*** 0272: bb.build.exec_func(postinst, d, (sstateinst,))
0273:
0274: for lock in locks:
0275: bb.utils.unlockfile(lock)
0276:
File: '/media/build1/poky/bitbake/lib/bb/build.py', lineno: 171, function: exec_func
0167:#
0168:def exec_func(func, d, dirs = None, pythonexception=False):
0169: """Execute a BB 'function'"""
0170:
*** 0171: oldcwd = os.getcwd()
0172: body = d.getVar(func, False)
0173: if not body:
0174: if body is None:
0175: logger.warning("Function %s doesn't exist", func)
Exception: FileNotFoundError: [Errno 2] No such file or directory
ERROR: watchdog-5.15-r0 do_populate_lic_setscene: Function failed: do_populate_lic_setscene
ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp6/work/core2-64-poky-linux-musl/watchdog/5.15-r0/temp/log.do_populate_lic_setscene.71045
Merged in bitbake 67a7b8b021badc17d8fdf447c250e79d291e75f7. Verified on master commit: 6b66e9317f4ec3a69f98f29836aafa35b52f3fc7 |
lib/bb/build.py:exec_func reads: 162: if dirs: [...] 166: else 167: adir = data.getVar('B', d, 1) 168: bb.utils.mkdirhier(adir) This means that execution of any function which does not have the 'dirs' flag set will cause ${B} to be used, and created. This can cause unforeseen complications. So far, at Mentor, we've seen two different build failures caused by this. First, copyleft_compliance's task can run before do_unpack, causing ${B}, which is ${S} by default, to be created ahead of time, and the git unpack can fail if the destination already exists. Second, do_rootfs currently adds ${S} to its cleandirs (unnecessarily, as it doesn't use it), and sstate_create_package doesn't have a 'dirs' flag set. As a result, it's possible for these two tasks to race, and sstate_create_package can try to cd into a path which do_rootfs's cleandirs is removing. One workaround is to set dirs for sstate_create_package, another is to remove ${S} from do_rootfs cleandirs. BitBake should not have any knowledge of ${S} or ${B}, so this default in exec_func is questionable at best. I believe that we should pursue removing this logic, but we will have to be careful about breaking any functions which could be relying on this default.