4634 2013-06-10 18:42:54 +0000 BitBake default 'dirs' in exec_func (${B}) causes build failures 2016-08-12 20:25:30 +0000 1 1 7 Build System, Metadata & Runtime BitBake bitbake unspecified x86 Multiple VERIFIED FIXED Medium+ enhancement 2.2 M2 1 kergoth ross.burton alexandru.c.georgescu jose.perez.carranza poky.bs.watcher poky.watcher richard.purdie ross.burton sgw jose.perez.carranza --- --- Yes (doc changes required) oldest_to_newest 33731 0 kergoth 2013-06-10 18:42:54 +0000 lib/bb/build.py:exec_func reads: 162: if dirs: [...] 166: else 167: adir = data.getVar('B', d, 1) 168: bb.utils.mkdirhier(adir) This means that execution of any function which does not have the 'dirs' flag set will cause ${B} to be used, and created. This can cause unforeseen complications. So far, at Mentor, we've seen two different build failures caused by this. First, copyleft_compliance's task can run before do_unpack, causing ${B}, which is ${S} by default, to be created ahead of time, and the git unpack can fail if the destination already exists. Second, do_rootfs currently adds ${S} to its cleandirs (unnecessarily, as it doesn't use it), and sstate_create_package doesn't have a 'dirs' flag set. As a result, it's possible for these two tasks to race, and sstate_create_package can try to cd into a path which do_rootfs's cleandirs is removing. One workaround is to set dirs for sstate_create_package, another is to remove ${S} from do_rootfs cleandirs. BitBake should not have any knowledge of ${S} or ${B}, so this default in exec_func is questionable at best. I believe that we should pursue removing this logic, but we will have to be careful about breaking any functions which could be relying on this default. 52195 1 ross.burton 2015-07-07 20:24:24 +0000 Discovered that this (via buildhistory_commit) is the reason that a do_unpack causes $(S) to be created. Decided to comment out the mkdir of ${B} to see what broke. BOOM. The assignment to adir of $(B) means that every Python function is chdir()'d into $(B), which doesn't exist. Comment out the assignment and stuff is building now... although how much of those Python functions are making an assumption about cwd I don't know. 52471 2 ross.burton 2015-07-17 16:35:13 +0000 Status: poky-contrib:ross/dirs has a patch to bitbake to stop his behaviour, and fixes for the problems it causes are in oe-core as they're found. 58661 3 ross.burton 2016-02-05 16:31:56 +0000 Too much fallout for M2 this late, moving to 2.2. 62955 4 ross.burton 2016-06-14 13:02:17 +0000 Fixup patches for oe-core sent, the bitbake patch just needs an AB run. 62956 5 ross.burton 2016-06-14 13:34:36 +0000 poky-contrib:ross/dirs has some fixes to oe-core and a bitbake patch to solve this. 63012 6 richard.purdie 2016-06-16 10:19:36 +0000 bitbake watchdog -c populate_lic bitbake watchdog -c clean bitbake watchdog Errors since the current working directory no longer exists. ERROR: watchdog-5.15-r0 do_populate_lic_setscene: Error executing a python function in exec_python_func() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:do_populate_lic_setscene(d) 0003: File: '/media/build1/poky/meta/classes/license.bbclass', lineno: 674, function: do_populate_lic_setscene 0670: 0671:do_populate_lic_setscene[dirs] = "${LICSSTATEDIR}/${PN}" 0672:do_populate_lic_setscene[cleandirs] = "${LICSSTATEDIR}" 0673:python do_populate_lic_setscene () { *** 0674: sstate_setscene(d) 0675:} 0676:addtask do_populate_lic_setscene File: '/media/build1/poky/meta/classes/sstate.bbclass', lineno: 642, function: sstate_setscene 0638: break 0639: 0640:def sstate_setscene(d): 0641: shared_state = sstate_state_fromvars(d) *** 0642: accelerate = sstate_installpkg(shared_state, d) 0643: if not accelerate: 0644: raise bb.build.FuncFailed("No suitable staging package found") 0645: 0646:python sstate_task_prefunc () { File: '/media/build1/poky/meta/classes/sstate.bbclass', lineno: 320, function: sstate_installpkg 0316: 0317: for state in ss['dirs']: 0318: prepdir(state[1]) 0319: os.rename(sstateinst + state[0], state[1]) *** 0320: sstate_install(ss, d) 0321: 0322: for plain in ss['plaindirs']: 0323: workdir = d.getVar('WORKDIR', True) 0324: src = sstateinst + "/" + plain.replace(workdir, '') File: '/media/build1/poky/meta/classes/sstate.bbclass', lineno: 272, function: sstate_install 0268: oe.path.copyhardlinktree(state[1], state[2]) 0269: 0270: for postinst in (d.getVar('SSTATEPOSTINSTFUNCS', True) or '').split(): 0271: # All hooks should run in the SSTATE_INSTDIR *** 0272: bb.build.exec_func(postinst, d, (sstateinst,)) 0273: 0274: for lock in locks: 0275: bb.utils.unlockfile(lock) 0276: File: '/media/build1/poky/bitbake/lib/bb/build.py', lineno: 171, function: exec_func 0167:# 0168:def exec_func(func, d, dirs = None, pythonexception=False): 0169: """Execute a BB 'function'""" 0170: *** 0171: oldcwd = os.getcwd() 0172: body = d.getVar(func, False) 0173: if not body: 0174: if body is None: 0175: logger.warning("Function %s doesn't exist", func) Exception: FileNotFoundError: [Errno 2] No such file or directory ERROR: watchdog-5.15-r0 do_populate_lic_setscene: Function failed: do_populate_lic_setscene ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp6/work/core2-64-poky-linux-musl/watchdog/5.15-r0/temp/log.do_populate_lic_setscene.71045 64219 7 ross.burton 2016-07-20 13:40:24 +0000 Merged in bitbake 67a7b8b021badc17d8fdf447c250e79d291e75f7. 65149 8 jose.perez.carranza 2016-08-12 20:25:30 +0000 Verified on master commit: 6b66e9317f4ec3a69f98f29836aafa35b52f3fc7