14070 2020-09-30 04:19:07 +0000 Devtool modify failed if recipe doesn't do git properly 2023-10-13 07:49:48 +0000 1 1 7 Build System, Metadata & Runtime OE-Core devtools / tool chain 3.1.2 x86 arm64 RESOLVED OBSOLETE Medium normal 4.3 1 prabin.ca ross.burton abdellatif.elkhlifi jstephan meta.mr.watcher meta.watcher randy.macleod richard.purdie ross.burton --- --- No (bug/feature does not impact docs) oldest_to_newest 88272 0 4733 prabin.ca 2020-09-30 04:19:07 +0000 Created attachment 4733 patch file to apply Hi All, I was trying "devtool modify" on one of my recipes (edk2-firmware.bb/bbappend for aarch64 platform). This recipe (both bb and bbappend) is not including any patches in SRC_URI. Even though its not using any patches, but devtool modify is failed at do_patch. How it can be failed at do_patch, since my recipes was not using any patches. As a hack, we can add do_patch[noexec]= "1" in recipes and bye pass the error. steps to replicate 1. repo init -u ssh://git@git.linaro.org/landing-teams/working/arm/arm-reference-platforms-manifest.git -b master -m corstone700.xml 2. repo sync -j8 3. cd layers/meta-arm 4. git apply edk2-firmware-crostone.diff , I have attached patch file 5. cd - export MACHINE=corstone700-fvp export DISTRO=poky-tiny . ./setup-environment 6. devtool modify edk2-firmware Error will be looks like below ERROR: Error executing a python function in exec_python_func() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:patch_task_postfunc(d) 0003: File: '/data_sdb/prabin/Abdel/build-poky-tiny/conf/../../layers/openembedded-core/meta/classes/patch.bbclass', lineno: 82, function: patch_task_postfunc 0078: stdout, _ = bb.process.run('git status --porcelain .', cwd=srcsubdir) 0079: if stdout: 0080: useroptions = [] 0081: oe.patch.GitApplyTree.gitCommandUserOptions(useroptions, d=d) *** 0082: bb.process.run('git add .; git %s commit -a -m "Committing changes from %s\n\n%s"' % (' '.join(useroptions), func, oe.patch.GitApplyTree.ignore_commit_prefix + ' - from %s' % func), cwd=srcsubdir) 0083:} 0084: 0085:def src_patches(d, all=False, expand=True): 0086: import oe.patch File: '/data_sdb/prabin/Abdel/bitbake/lib/bb/process.py', lineno: 182, function: run 0178: if not stderr is None: 0179: stderr = stderr.decode("utf-8") 0180: 0181: if pipe.returncode != 0: *** 0182: raise ExecutionError(cmd, pipe.returncode, stdout, stderr) 0183: return stdout, stderr Exception: bb.process.ExecutionError: Execution of 'git add .; git -c user.name="OpenEmbedded" -c user.email="oe.patch@oe" commit -a -m "Committing changes from do_patch %% ignore - from do_patch"' failed with exit code 1: On branch devtool Changes not staged for commit: modified: edk2 (untracked content) no changes added to commit ERROR: Logfile of failure stored in: /data_sdb/prabin/Abdel/build-poky-tiny/tmp-poky_tiny/work/corstone700_fvp-poky-linux-musleabi/edk2-firmware/gitAUTOINC+11247ae126_1a785d6f0c-r0/devtooltmp-_d4cpxiq/temp/log.do_patch.27583 NOTE: Tasks Summary: Attempted 93 tasks of which 90 didn't need to be rerun and 1 failed. 88369 1 ross.burton 2020-10-12 09:17:28 +0000 Replicated with meta-arm master and MACHINE=n1sdp. The problem is how edk2-firmware clones in a manner that devtool doesn't expect: modified: edk2 (untracked content) ${S} isn't in fact a git repository, but a directory that contains a git repository. We can fix this in meta-arm so that devtool works as expected, but it would be nice if devtool handled this case. Basically the situation is: S=${WORKDIR}/git SRC_URI=git://foo.com;destsuffix=${S}/foo WORKDIR/git is just a normal directory containing foo, a git clone. 88434 2 ross.burton 2020-10-20 04:00:47 +0000 Summary: devtool expects a recipe using the git fetcher to unpack a git repository in S. This is a fair assumption, and I think in this case devtool should just detect and abort cleanly. 88685 3 ross.burton 2020-11-27 10:10:34 +0000 Our original reproducer is now fixed with meta-arm ed66883f144d5c30ee081a08bb57391b0a749622. devtool not handling this situation nicely is still a valid issue. 93209 4 ross.burton 2022-04-28 15:10:49 +0000 Need to verify this is still a problem. 95440 5 randy.macleod 2023-04-21 18:45:46 +0000 Moved en masse from 4.2 to 4.3 by Randy. 96215 6 jstephan 2023-08-10 13:06:38 +0000 Hi, just tried to reproduce using: S=${WORKDIR}/git SRC_URI=git://foo.com;destsuffix=${S}/foo but it seems that it is not allowed anymore to use ${S} in SRC_URI. I now get the following error: bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher failure: There are recursive references in fetcher variables, likely through SRC_URI The variable dependency chain for the failure is: SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> BP -> FILESPATH 96331 7 richard.purdie 2023-08-31 14:57:59 +0000 (In reply to Julien Stephan from comment #6) > Hi, > just tried to reproduce using: > > S=${WORKDIR}/git > SRC_URI=git://foo.com;destsuffix=${S}/foo > > but it seems that it is not allowed anymore to use ${S} in SRC_URI. I now > get the following error: > > bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression > was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: > Fetcher failure: There are recursive references in fetcher variables, likely > through SRC_URI > > The variable dependency chain for the failure is: SRCPV -> PV -> WORKDIR -> > S -> SRC_URI -> SRCPV -> PV -> BP -> FILESPATH Correct, using ${S} in SRC_URI is a bad idea as we can need to query the source to work out PV and you get a circular dependency. You probably want a relative path rather than an absolute one. 96530 8 jstephan 2023-09-21 14:11:04 +0000 Hi Richard, I tried to reproduce using S=${WORKDIR}/git/foo SRC_URI=git://foo.com;destsuffix=git/foo but still not reproducing the error, so maybe this is not an issue anymore? 96698 9 jstephan 2023-10-13 07:49:48 +0000 Hi all, closing this bug as I cannot reproduce it. 4733 2020-09-30 04:19:07 +0000 2020-09-30 04:19:07 +0000 patch file to apply edk2-firmware-crostone.diff application/octet-stream 3785 prabin.ca ZGlmZiAtLWdpdCBhL21ldGEtYXJtLWJzcC9yZWNpcGVzLWJzcC91ZWZpL2VkazItZmlybXdhcmUt Y29yc3RvbmU3MDAuaW5jIGIvbWV0YS1hcm0tYnNwL3JlY2lwZXMtYnNwL3VlZmkvZWRrMi1maXJt d2FyZS1jb3JzdG9uZTcwMC5pbmMKbmV3IGZpbGUgbW9kZSAxMDA2NDQKaW5kZXggMDAwMDAwMC4u ZjIzODE4MwotLS0gL2Rldi9udWxsCisrKyBiL21ldGEtYXJtLWJzcC9yZWNpcGVzLWJzcC91ZWZp L2VkazItZmlybXdhcmUtY29yc3RvbmU3MDAuaW5jCkBAIC0wLDAgKzEsNjQgQEAKK1NSQ19VUkkg PSAiXAorICAgICAgICAgICBnaXQ6Ly8kVVNFUkBnZXJyaXQub3NzLmFybS5jb20vTW9yZWxsby91 ZWZpL2VkazI7bmFtZT1lZGsyO3Byb3RvY29sPXNzaDtkZXN0c3VmZml4PWdpdC9lZGsyO2JyYW5j aD10b3BpY3MveW9jdG8gXAorICAgICAgICAgICBnaXQ6Ly8kVVNFUkBnZXJyaXQub3NzLmFybS5j b20vTW9yZWxsby91ZWZpL2VkazItcGxhdGZvcm1zO3Byb3RvY29sPXNzaDtuYW1lPWVkazItcGxh dGZvcm1zO2Rlc3RzdWZmaXg9Z2l0L2VkazIvZWRrMi1wbGF0Zm9ybXM7YnJhbmNoPXRvcGljcy95 b2N0byBcCisiCisKKyMgVEFHUyBhbmQgY29tbWl0IElEIGZvciBtb3JlbGxvCitTUkNSRVZfZWRr MiAgICAgICAgICAgPSAiJHtBVVRPUkVWfSIKK1NSQ1JFVl9lZGsyLXBsYXRmb3JtcyA9ICIke0FV VE9SRVZ9IgorCitFREsyX0JVSUxEX1JFTEVBU0UgPSAiMCIKK0VESzJfUExBVEZPUk0gICAgICA9 ICJtb3JlbGxvZnZwIgorRURLMl9QTEFURk9STV9EU0MgID0gIk1vcmVsbG8vTW9yZWxsb1BsYXRm b3JtRnZwLmRzYyIKK0VESzJfQklOX05BTUUgICAgICA9ICJCTDMzX0FQX1VFRkkuZmQiCitFREsy X0FSQ0ggICAgICAgICAgPSAiQUFSQ0g2NCIKKworIyBsbHZtIGNyb3NzIHRvb2wgY2hhaW4gZGVw ZW5kcyBmb3IgYnVpbGRpbmcgdWVmaQorTExWTV9QQVRIID0gIiR7UkVDSVBFX1NZU1JPT1R9L291 dC9pbnN0YWxsL2xpbnV4LXg4Ni9jbGFuZy1hcm0vYmluIgorREVQRU5EUyArPSAiJHtAYmIudXRp bHMuY29udGFpbnMoJ1NUQUNLX0M2NCcsICcxJywgJ3ZpcnR1YWwvYWFyY2g2NC1jNjQtbGx2bScs ICcnLCBkKX0iCisKK1VFRklfQzY0ID0gIiR7QGJiLnV0aWxzLmNvbnRhaW5zKCdTVEFDS19DNjQn LCAnMScsICcxJywgJzAnLCBkKX0iCisKK0VESzJfQlVJTERfRkxBR1MgICArPSAiJHtAYmIudXRp bHMuY29udGFpbnMoJ1VFRklfQzY0JywgJzEnLCAnLUQgRU5BQkxFX01PUkVMTE9fQ0FQJywgJycs IGQpfSIKK0VESzJfQlVJTERfRkxBR1MgICArPSAiLUQgRklSTVdBUkVfVkVSPTExMjQ3YWUxMjYi CitUT09MX0NIQUlOICA9ICIke0BiYi51dGlscy5jb250YWlucygnVUVGSV9DNjQnLCAnMScsICdj bGFuZycsICdnY2MnLGQpfSIKK1BMQVRGT1JNX1RPT0xDSEFJTiA9ICIke0BiYi51dGlscy5jb250 YWlucygnVE9PTF9DSEFJTicsICdjbGFuZycsICdDTEFORzM4JywgJyR7R0NDX1ZFUn0nLCBkKX0i CisKK2V4cG9ydCBDTEFORzM4X0JJTj0gIiR7TExWTV9QQVRIfS8iCitMTFZNX1BSRUZJWCA9ICJs bHZtLSIKKyMgbGluYXJvLWdjYyBkZXBlbmRzCisKK0RFUEVORFMgKz0gIiR7QGJiLnV0aWxzLmNv bnRhaW5zKCdUT09MX0NIQUlOJywgJ2NsYW5nJywgJ3ZpcnR1YWwvYWFyY2g2NC1nbnUtbGluYXJv JywgJycsIGQpfSIKK0xJTkFST19QQVRIID0gIiR7QGJiLnV0aWxzLmNvbnRhaW5zKCdUT09MX0NI QUlOJywgJ2NsYW5nJywgJyR7UkVDSVBFX1NZU1JPT1R9L2djYy1saW5hcm8tNy41LjAtMjAxOS4x Mi14ODZfNjRfYWFyY2g2NC1saW51eC1nbnUvYmluJywgJycsIGQpfSIKKworaW5oZXJpdCBweXRo b24zbmF0aXZlCitERVBFTkRTICs9ICJweXRob24zLW5hdGl2ZSIKKworZG9fY29tcGlsZSgpIHsK KyAgICBzZWQgLWkgLWUgJ3M6LUkgXC5cLjotSSBcLlwuICR7QlVJTERfQ0ZMQUdTfSA6JyAke0VE S19UT09MU19QQVRIfS9Tb3VyY2UvQy9NYWtlZmlsZXMvaGVhZGVyLm1ha2VmaWxlCisgICAgc2Vk IC1pIC1lICdzOiAtbHV1aWQ6IC1sdXVpZCAke0JVSUxEX0xERkxBR1N9OmcnICR7RURLX1RPT0xT X1BBVEh9L1NvdXJjZS9DLyovR05VbWFrZWZpbGUKKworICAgICMgQ29weSB0aGUgdGVtcGxhdGVz IGFzIHdlIGRvbid0IHJ1biBlbnZzZXR1cAorICAgIGNwICR7RURLX1RPT0xTX1BBVEh9L0NvbmYv YnVpbGRfcnVsZS50ZW1wbGF0ZSAke1dPUktTUEFDRX0vQ29uZi9idWlsZF9ydWxlLnR4dAorICAg IGNwICR7RURLX1RPT0xTX1BBVEh9L0NvbmYvdG9vbHNfZGVmLnRlbXBsYXRlICR7V09SS1NQQUNF fS9Db25mL3Rvb2xzX2RlZi50eHQKKyAgICBjcCAke0VES19UT09MU19QQVRIfS9Db25mL3Rhcmdl dC50ZW1wbGF0ZSAke1dPUktTUEFDRX0vQ29uZi90YXJnZXQudHh0CisgICAgIyBCdWlsZCBiYXNl dG9vbHMKKyAgICBvZV9ydW5tYWtlIC1DICR7U30vZWRrMi9CYXNlVG9vbHMKKyAgICAke1BMQVRG T1JNX1RPT0xDSEFJTn1fJHtFREsyX0FSQ0h9X1BSRUZJWD0iJHtAYmIudXRpbHMuY29udGFpbnMo J1RPT0xfQ0hBSU4nLCAnY2xhbmcnLCAnJHtMTFZNX1BBVEh9LyR7TExWTV9QUkVGSVh9JywgJyR7 U1RBR0lOR19CSU5ESVJfVE9PTENIQUlOfS8ke1RBUkdFVF9QUkVGSVh9JywgZCl9IiBcCisgICAg UEFUSD0iJHtXT1JLU1BBQ0V9L2VkazIvQmFzZVRvb2xzL0Jpbi9MaW51eC14ODZfNjQ6JHtXT1JL U1BBQ0V9OiR7QlRPT0xTX1BBVEh9OiR7TElOQVJPX1BBVEh9OiRQQVRIIiBcCisgICAgIiR7U30v ZWRrMi9CYXNlVG9vbHMvQmluV3JhcHBlcnMvUG9zaXhMaWtlL2J1aWxkIiBcCisgICAgICAgLWEg IiR7RURLMl9BUkNIfSIgXAorICAgICAgIC1iICR7RURLMl9CVUlMRF9NT0RFfSBcCisgICAgICAg LXMgJHtFREsyX0JVSUxEX0ZMQUdTfSBcCisgICAgICAgLXQgJHtQTEFURk9STV9UT09MQ0hBSU59 IFwKKyAgICAgICAtcCAiJHtTfS9lZGsyL2VkazItcGxhdGZvcm1zL1BsYXRmb3JtL0FSTS8ke0VE SzJfUExBVEZPUk1fRFNDfSIKK30KKworZG9faW5zdGFsbCgpIHsKKyAgICBpbnN0YWxsIC1kICR7 RH0vZmlybXdhcmUKKyAgICBpbnN0YWxsICIke1N9L2VkazIvQnVpbGQvJHtFREsyX1BMQVRGT1JN fS8ke0VESzJfQlVJTERfTU9ERX1fJHtQTEFURk9STV9UT09MQ0hBSU59L0ZWLyR7RURLMl9CSU5f TkFNRX0iICIke0R9L2Zpcm13YXJlL3VlZmkuYmluIgorfQorCitDT01QQVRJQkxFX01BQ0hJTkUg PSAiY29yc3RvbmU3MDAtZnZwIgorCitQViA9ICJnaXQke1NSQ1BWfSIKZGlmZiAtLWdpdCBhL21l dGEtYXJtLWJzcC9yZWNpcGVzLWJzcC91ZWZpL2VkazItZmlybXdhcmVfZ2l0LmJiYXBwZW5kIGIv bWV0YS1hcm0tYnNwL3JlY2lwZXMtYnNwL3VlZmkvZWRrMi1maXJtd2FyZV9naXQuYmJhcHBlbmQK aW5kZXggYmVmNTRiMC4uY2EwOTFiOCAxMDA2NDQKLS0tIGEvbWV0YS1hcm0tYnNwL3JlY2lwZXMt YnNwL3VlZmkvZWRrMi1maXJtd2FyZV9naXQuYmJhcHBlbmQKKysrIGIvbWV0YS1hcm0tYnNwL3Jl Y2lwZXMtYnNwL3VlZmkvZWRrMi1maXJtd2FyZV9naXQuYmJhcHBlbmQKQEAgLTQsNSArNCw2IEBA IE1BQ0hJTkVfRURLMl9SRVFVSVJFID89ICIiCiAKIE1BQ0hJTkVfRURLMl9SRVFVSVJFX24xc2Rw ID0gImVkazItZmlybXdhcmUtbjFzZHAuaW5jIgogTUFDSElORV9FREsyX1JFUVVJUkVfc2dpNTc1 ID0gImVkazItZmlybXdhcmUtc2dpNTc1LmluYyIKK01BQ0hJTkVfRURLMl9SRVFVSVJFX2NvcnN0 b25lNzAwLWZ2cCA9ICJlZGsyLWZpcm13YXJlLWNvcnN0b25lNzAwLmluYyIKIAogcmVxdWlyZSAk e01BQ0hJTkVfRURLMl9SRVFVSVJFfQo=