13709 2019-12-16 14:40:36 +0000 Buildhistory race when building sdk and eSDK in parallel 2020-07-25 07:14:30 +0000 1 1 7 Build System, Metadata & Runtime OE-Core core 3.0 x86 Multiple RESOLVED FIXED autobuilder NEWCOMER AB-INT Medium+ normal 3.2 M2 1 richard.purdie chee.yang.lee akuster matthew.zeng meta.mr.watcher meta.watcher randy.macleod tim.orling --- --- No (bug/feature does not impact docs) oldest_to_newest 85837 0 richard.purdie 2019-12-16 14:40:36 +0000 https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/1356 Looks like a buildhistory race when sdk and eSDK are running in parallel: NOTE: recipe core-image-sato-1.0-r0: task do_sdk_depends: Succeeded NOTE: Running task 8949 of 8949 (/home/pokybuild/yocto-worker/qemux86-64/build/meta/recipes-sato/images/core-image-sato.bb:do_populate_sdk_ext) NOTE: recipe core-image-sato-1.0-r0: task do_populate_sdk_ext: Started ERROR: core-image-sato-1.0-r0 do_populate_sdk: Error executing a python function in exec_python_func() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:do_populate_sdk(d) 0003: File: '/home/pokybuild/yocto-worker/qemux86-64/build/meta/classes/populate_sdk_base.bbclass', lineno: 169, function: do_populate_sdk 0165: 0166: populate_sdk(d) 0167: 0168:fakeroot python do_populate_sdk() { *** 0169: populate_sdk_common(d) 0170:} 0171:SSTATETASKS += "do_populate_sdk" 0172:SSTATE_SKIP_CREATION_task-populate-sdk = '1' 0173:do_populate_sdk[cleandirs] = "${SDKDEPLOYDIR}" File: '/home/pokybuild/yocto-worker/qemux86-64/build/meta/classes/populate_sdk_base.bbclass', lineno: 166, function: populate_sdk_common 0162: manifest_type=Manifest.MANIFEST_TYPE_SDK_HOST) 0163: create_manifest(d, manifest_dir=d.getVar('SDK_DIR'), 0164: manifest_type=Manifest.MANIFEST_TYPE_SDK_TARGET) 0165: *** 0166: populate_sdk(d) 0167: 0168:fakeroot python do_populate_sdk() { 0169: populate_sdk_common(d) 0170:} File: '/home/pokybuild/yocto-worker/qemux86-64/build/meta/lib/oe/sdk.py', lineno: 413, function: populate_sdk 0409: env_bkp = os.environ.copy() 0410: 0411: img_type = d.getVar('IMAGE_PKGTYPE') 0412: if img_type == "rpm": *** 0413: RpmSdk(d, manifest_dir).populate() 0414: elif img_type == "ipk": 0415: OpkgSdk(d, manifest_dir).populate() 0416: elif img_type == "deb": 0417: DpkgSdk(d, manifest_dir).populate() File: '/home/pokybuild/yocto-worker/qemux86-64/build/meta/lib/oe/sdk.py', lineno: 42, function: populate 0038: def populate(self): 0039: self.mkdirhier(self.sdk_output) 0040: 0041: # call backend dependent implementation *** 0042: self._populate() 0043: 0044: # Don't ship any libGL in the SDK 0045: self.remove(os.path.join(self.sdk_output, self.sdk_native_path, 0046: self.d.getVar('libdir_nativesdk').strip('/'), File: '/home/pokybuild/yocto-worker/qemux86-64/build/meta/lib/oe/sdk.py', lineno: 182, function: _populate 0178: self.install_locales(self.host_pm) 0179: 0180: self.host_pm.run_intercepts(populate_sdk='host') 0181: *** 0182: execute_pre_post_process(self.d, self.d.getVar("POPULATE_SDK_POST_HOST_COMMAND")) 0183: 0184: if not bb.utils.contains("SDKIMAGE_FEATURES", "package-management", True, False, self.d): 0185: self.host_pm.remove_packaging_data() 0186: File: '/home/pokybuild/yocto-worker/qemux86-64/build/meta/lib/oe/utils.py', lineno: 260, function: execute_pre_post_process 0256: for cmd in cmds.strip().split(';'): 0257: cmd = cmd.strip() 0258: if cmd != '': 0259: bb.note("Executing %s ..." % cmd) *** 0260: bb.build.exec_func(cmd, d) 0261: 0262:# For each item in items, call the function 'target' with item as the first 0263:# argument, extraargs as the other arguments and handle any exceptions in the 0264:# parent thread File: '/home/pokybuild/yocto-worker/qemux86-64/build/bitbake/lib/bb/build.py', lineno: 250, function: exec_func 0246: with bb.utils.fileslocked(lockfiles): 0247: if ispython: 0248: exec_func_python(func, d, runfile, cwd=adir) 0249: else: *** 0250: exec_func_shell(func, d, runfile, cwd=adir) 0251: 0252: try: 0253: curcwd = os.getcwd() 0254: except: File: '/home/pokybuild/yocto-worker/qemux86-64/build/bitbake/lib/bb/build.py', lineno: 451, function: exec_func_shell 0447: with open(fifopath, 'r+b', buffering=0) as fifo: 0448: try: 0449: bb.debug(2, "Executing shell function %s" % func) 0450: with open(os.devnull, 'r+') as stdin, logfile: *** 0451: bb.process.run(cmd, shell=False, stdin=stdin, log=logfile, extrafiles=[(fifo,readfifo)]) 0452: finally: 0453: os.unlink(fifopath) 0454: 0455: bb.debug(2, "Shell function %s finished" % func) File: '/home/pokybuild/yocto-worker/qemux86-64/build/bitbake/lib/bb/process.py', lineno: 182, function: run 0178: if not stderr is None: 0179: stderr = stderr.decode("utf-8") 0180: 0181: if pipe.returncode != 0: *** 0182: raise ExecutionError(cmd, pipe.returncode, stdout, stderr) 0183: return stdout, stderr Exception: bb.process.ExecutionError: Execution of '/home/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/temp/run.buildhistory_get_sdk_installed_host.9350' failed with exit code 1: rm: cannot remove ‘/home/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/bh_installed_pkgs.txt’: No such file or directory WARNING: exit code 1 from a shell command. ERROR: Logfile of failure stored in: /home/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/temp/log.do_populate_sdk.9350 NOTE: recipe core-image-sato-1.0-r0: task do_populate_sdk: Failed ERROR: Task (/home/pokybuild/yocto-worker/qemux86-64/build/meta/recipes-sato/images/core-image-sato.bb:do_populate_sdk) failed with exit code '1' NOTE: recipe core-image-sato-1.0-r0: task do_populate_sdk_ext: Succeeded 86778 1 richard.purdie 2020-03-23 10:29:07 +0000 Happened again: https://autobuilder.yoctoproject.org/typhoon/#/builders/59/builds/1713 87656 2 richard.purdie 2020-06-28 23:32:39 +0000 https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/2099 87662 3 richard.purdie 2020-06-30 07:06:55 +0000 https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/2092/steps/8/logs/step2b 87814 4 chee.yang.lee 2020-07-21 07:34:18 +0000 for eSDK, buildhistory are by default disabled in SDK_INHERIT_BLACKLIST. it seems like in these build eSDK do INHERIT buildhistory, is this expected? 87823 5 richard.purdie 2020-07-22 14:42:53 +0000 (In reply to comment #4) > for eSDK, buildhistory are by default disabled in SDK_INHERIT_BLACKLIST. > it seems like in these build eSDK do INHERIT buildhistory, is this expected? Those would be applied when the eSDK is running but not while eSDK itself is being built. The error happens when the eSDK is being built at the same time as an SDK. 87833 6 richard.purdie 2020-07-25 07:14:30 +0000 Thanks for the fix!: http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=4eef15704a58f65514d7e7b43d8abe3b3f78fcca