13404 2019-06-18 11:07:35 +0000 'devtool upgrade' tracebacks when a recipe includes another recipe with PV in the name 2019-07-03 20:27:48 +0000 1 1 7 Build System, Metadata & Runtime OE-Core Scripts and Tools unspecified x86 Multiple RESOLVED FIXED Medium+ normal 2.8 M2 1 alex.kanavin bluelightning ross.burton --- --- No (bug/feature does not impact docs) oldest_to_newest 84232 0 alex.kanavin 2019-06-18 11:07:35 +0000 Devtool is not able to handle a situation when a recipe to be upgraded includes another recipe with $PV in its filename (in this case the included recipe filename should be renamed but isn't). How to reproduce: With poky at http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=111b7173fee69fe8b01ec2fa6999713841326537 issue 'devtool upgrade libnewt-python' This will result in Traceback (most recent call last): File "/home/alexander/development/poky/scripts/devtool", line 334, in <module> ret = main() File "/home/alexander/development/poky/scripts/devtool", line 321, in main ret = args.func(args, config, basepath, workspace) File "/home/alexander/development/poky/scripts/lib/devtool/upgrade.py", line 551, in upgrade rf, copied = _create_new_recipe(args.version, md5, sha256, args.srcrev, srcbranch, srcsubdir1, srcsubdir2, config.workspace_path, tinfoil, rd, license_diff, new_licenses) File "/home/alexander/development/poky/scripts/lib/devtool/upgrade.py", line 419, in _create_new_recipe rd = tinfoil.parse_recipe_file(fullpath, False) File "/home/alexander/development/poky/bitbake/lib/bb/tinfoil.py", line 641, in parse_recipe_file dscon = self.run_command('parseRecipeFile', fn, appends, appendlist) File "/home/alexander/development/poky/bitbake/lib/bb/tinfoil.py", line 466, in run_command raise TinfoilCommandFailed(result[1]) bb.tinfoil.TinfoilCommandFailed: Traceback (most recent call last): File "/home/alexander/development/poky/bitbake/lib/bb/parse/parse_py/ConfHandler.py", line 94, in include_single_file bb.parse.handle(fn, data, True) File "/home/alexander/development/poky/bitbake/lib/bb/parse/__init__.py", line 107, in handle return h['handle'](fn, data, include) File "/home/alexander/development/poky/bitbake/lib/bb/parse/parse_py/BBHandler.py", line 120, in handle abs_fn = resolve_file(fn, d) File "/home/alexander/development/poky/bitbake/lib/bb/parse/__init__.py", line 125, in resolve_file raise IOError(errno.ENOENT, "file %s not found in %s" % (fn, bbpath)) FileNotFoundError: [Errno 2] file recipes-extended/newt/libnewt_0.52.21.bb not found in /home/alexander/development/poky/build/workspace:/home/alexander/development/poky/meta-poky:/home/alexander/development/poky/build:/home/alexander/development/poky/meta:/home/alexander/development/poky/meta-yocto-bsp:/home/alexander/development/meta-mingw During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/alexander/development/poky/bitbake/lib/bb/command.py", line 74, in runCommand result = command_method(self, commandline) File "/home/alexander/development/poky/bitbake/lib/bb/command.py", line 562, in parseRecipeFile envdata = parser.loadDataFull(fn, appendfiles) File "/home/alexander/development/poky/bitbake/lib/bb/cache.py", line 327, in loadDataFull bb_data = self.load_bbfile(virtualfn, appends, virtonly=True) File "/home/alexander/development/poky/bitbake/lib/bb/cache.py", line 340, in load_bbfile datastores = parse_recipe(bb_data, bbfile, appends, mc) File "/home/alexander/development/poky/bitbake/lib/bb/cache.py", line 303, in parse_recipe bb_data = bb.parse.handle(bbfile, bb_data) File "/home/alexander/development/poky/bitbake/lib/bb/parse/__init__.py", line 107, in handle return h['handle'](fn, data, include) File "/home/alexander/development/poky/bitbake/lib/bb/parse/parse_py/BBHandler.py", line 130, in handle statements.eval(d) File "/home/alexander/development/poky/bitbake/lib/bb/parse/ast.py", line 23, in eval statement.eval(data) File "/home/alexander/development/poky/bitbake/lib/bb/parse/ast.py", line 45, in eval bb.parse.ConfHandler.include(self.filename, s, self.lineno, data, "include required") File "/home/alexander/development/poky/bitbake/lib/bb/parse/parse_py/ConfHandler.py", line 71, in include include_single_file(parentfn, fn, lineno, data, error_out) File "/home/alexander/development/poky/bitbake/lib/bb/parse/parse_py/ConfHandler.py", line 98, in include_single_file raise ParseError("Could not %s file %s" % (error_out, fn), parentfn, lineno) bb.parse.ParseError: ParseError at /home/alexander/development/poky/build/workspace/recipes/libnewt-python/libnewt-python_0.52.21.bb:1: Could not include required file recipes-extended/newt/libnewt_0.52.21.bb and the workspace will be left in an inconsistent state: workspace/recipes will have the half-updated recipe that bitbake is unable to parse (due to missing include) but workspace/appends will be empty. Because of that, 'devtool reset' will not be able to remove the offending recipe from the workspace, and it has to be deleted by hand. 84243 1 alex.kanavin 2019-06-19 10:06:37 +0000 A related improvement would be to have a 'devtool reset --force-all' command, which would basically re-initialize the workspace to a completely blank state. This wouls allow recovery from similar tracebacks - even if this specific one is fixed, there may be more. 84356 2 bluelightning 2019-07-02 04:27:19 +0000 I've sent a patch to fix this so it doesn't leave files around after the error by default (as well as generally fixing error handling). I think automatically handling this scenario completely (i.e. upgrading the other recipe) would be overkill, since this is rather unusual. With regard to changing devtool reset, I'd prefer instead that we fix these kinds of situations as they come up rather than providing a tool to work around them. 84363 3 bluelightning 2019-07-03 20:27:48 +0000 Fix merged to master: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=b52d58b8bb81f5053b9fa8b79a137c37d6ef6fb3 (Note that the recipe originally triggering this was already changed to avoid the issue under http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=0d8357f9f38f529cc03ce2602d5209c9f8dfe878 ).