Bug 14070 - Devtool modify failed if recipe doesn't do git properly
Summary: Devtool modify failed if recipe doesn't do git properly
Status: RESOLVED OBSOLETE
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: devtools / tool chain (show other bugs)
Version: 3.1.2
Hardware: x86 arm64
: Medium normal
Target Milestone: 4.3
Assignee: Ross Burton
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2020-09-30 04:19 UTC by Prabin CA
Modified: 2023-10-13 07:49 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
patch file to apply (3.70 KB, application/octet-stream)
2020-09-30 04:19 UTC, Prabin CA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Prabin CA 2020-09-30 04:19:07 UTC
Created attachment 4733 [details]
patch file to apply

Hi All,

I was trying "devtool modify" on one of my recipes (edk2-firmware.bb/bbappend for aarch64 platform). This recipe (both bb and bbappend) is not including any patches in SRC_URI. Even though its not using any patches, but devtool modify is failed at do_patch. How it can be failed at do_patch, since my recipes was not using any patches. As a hack, we can add do_patch[noexec]= "1" in recipes and bye pass the error. 

steps to replicate 

1. repo init -u ssh://git@git.linaro.org/landing-teams/working/arm/arm-reference-platforms-manifest.git -b master -m corstone700.xml
2. repo sync -j8
3. cd layers/meta-arm
4. git apply edk2-firmware-crostone.diff , I have attached patch file
5. cd -
   export MACHINE=corstone700-fvp
   export DISTRO=poky-tiny
   . ./setup-environment
6. devtool modify edk2-firmware

Error will be looks like below 

ERROR: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:patch_task_postfunc(d)
     0003:
File: '/data_sdb/prabin/Abdel/build-poky-tiny/conf/../../layers/openembedded-core/meta/classes/patch.bbclass', lineno: 82, function: patch_task_postfunc
     0078:        stdout, _ = bb.process.run('git status --porcelain .', cwd=srcsubdir)
     0079:        if stdout:
     0080:            useroptions = []
     0081:            oe.patch.GitApplyTree.gitCommandUserOptions(useroptions, d=d)
 *** 0082:            bb.process.run('git add .; git %s commit -a -m "Committing changes from %s\n\n%s"' % (' '.join(useroptions), func, oe.patch.GitApplyTree.ignore_commit_prefix + ' - from %s' % func), cwd=srcsubdir)
     0083:}
     0084:
     0085:def src_patches(d, all=False, expand=True):
     0086:    import oe.patch
File: '/data_sdb/prabin/Abdel/bitbake/lib/bb/process.py', lineno: 182, function: run
     0178:        if not stderr is None:
     0179:            stderr = stderr.decode("utf-8")
     0180:
     0181:    if pipe.returncode != 0:
 *** 0182:        raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
     0183:    return stdout, stderr
Exception: bb.process.ExecutionError: Execution of 'git add .; git -c user.name="OpenEmbedded" -c user.email="oe.patch@oe" commit -a -m "Committing changes from do_patch

%% ignore - from do_patch"' failed with exit code 1:
On branch devtool
Changes not staged for commit:
        modified:   edk2 (untracked content)

no changes added to commit


ERROR: Logfile of failure stored in: /data_sdb/prabin/Abdel/build-poky-tiny/tmp-poky_tiny/work/corstone700_fvp-poky-linux-musleabi/edk2-firmware/gitAUTOINC+11247ae126_1a785d6f0c-r0/devtooltmp-_d4cpxiq/temp/log.do_patch.27583
NOTE: Tasks Summary: Attempted 93 tasks of which 90 didn't need to be rerun and 1 failed.
Comment 1 Ross Burton 2020-10-12 09:17:28 UTC
Replicated with meta-arm master and MACHINE=n1sdp.  The problem is how edk2-firmware clones in a manner that devtool doesn't expect:

        modified:   edk2 (untracked content)

${S} isn't in fact a git repository, but a directory that contains a git repository.

We can fix this in meta-arm so that devtool works as expected, but it would be nice if devtool handled this case.

Basically the situation is:

S=${WORKDIR}/git
SRC_URI=git://foo.com;destsuffix=${S}/foo

WORKDIR/git is just a normal directory containing foo, a git clone.
Comment 2 Ross Burton 2020-10-20 04:00:47 UTC
Summary: devtool expects a recipe using the git fetcher to unpack a git repository in S.

This is a fair assumption, and I think in this case devtool should just detect and abort cleanly.
Comment 3 Ross Burton 2020-11-27 10:10:34 UTC
Our original reproducer is now fixed with meta-arm ed66883f144d5c30ee081a08bb57391b0a749622.  devtool not handling this situation nicely is still a valid issue.
Comment 4 Ross Burton 2022-04-28 15:10:49 UTC
Need to verify this is still a problem.
Comment 5 Randy MacLeod 2023-04-21 18:45:46 UTC
Moved en masse from 4.2 to 4.3 by Randy.
Comment 6 Julien Stephan 2023-08-10 13:06:38 UTC
Hi,
just tried to reproduce using:

S=${WORKDIR}/git
SRC_URI=git://foo.com;destsuffix=${S}/foo

but it seems that it is not allowed anymore to use ${S} in SRC_URI. I now get the following error: 

bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher failure: There are recursive references in fetcher variables, likely through SRC_URI                                                                                                                                                                                                       
The variable dependency chain for the failure is: SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> BP -> FILESPATH
Comment 7 Richard Purdie 2023-08-31 14:57:59 UTC
(In reply to Julien Stephan from comment #6)
> Hi,
> just tried to reproduce using:
> 
> S=${WORKDIR}/git
> SRC_URI=git://foo.com;destsuffix=${S}/foo
> 
> but it seems that it is not allowed anymore to use ${S} in SRC_URI. I now
> get the following error: 
> 
> bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression
> was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError:
> Fetcher failure: There are recursive references in fetcher variables, likely
> through SRC_URI                                                             
> 
> The variable dependency chain for the failure is: SRCPV -> PV -> WORKDIR ->
> S -> SRC_URI -> SRCPV -> PV -> BP -> FILESPATH

Correct, using ${S} in SRC_URI is a bad idea as we can need to query the source to work out PV and you get a circular dependency.

You probably want a relative path rather than an absolute one.
Comment 8 Julien Stephan 2023-09-21 14:11:04 UTC
Hi Richard, 
I tried to reproduce using

S=${WORKDIR}/git/foo
SRC_URI=git://foo.com;destsuffix=git/foo

but still not reproducing the error, so maybe this is not an issue anymore?
Comment 9 Julien Stephan 2023-10-13 07:49:48 UTC
Hi all,
closing this bug as I cannot reproduce it.