Bug 14944 - go recipes are not devtool upgradeable
Summary: go recipes are not devtool upgradeable
Status: NEW
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: oe-core other (show other bugs)
Version: 0.0.0
Hardware: x86 Multiple
: Medium+ enhancement
Target Milestone: 6.2 M1
Assignee: Unassigned
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2022-10-21 13:37 UTC by Thomas Roos
Modified: 2026-10-29 15:25 UTC (History)
8 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Roos 2022-10-21 13:37:31 UTC
When doing a "devtool upgrade" for a recipe which inherits "go" and using an SRC_URI to a git repo, this will fail.

E.g. set the go-helloworld bb srcrev to an older rev:
SRCREV = "ad95e7f791d8e1e7e7ec7f7cedfc2f6b29f6c9e2"

"devtool upgrade go-helloworld"

INFO: Extracting upgraded version source...
Traceback (most recent call last):
  File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 567, in upgrade
    rev2, md5, sha256, srcbranch, srcsubdir2 = _extract_new_source(args.version, srctree, args.no_patch,
  File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 183, in _extract_new_source
    __run('git checkout %s' % rev)
  File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 170, in __run
    return _run(cmd, srctree)
  File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 31, in _run
    return bb.process.run('%s' % cmd, cwd=cwd)
  File "/home/ubuntu/yocto/poky/bitbake/lib/bb/process.py", line 189, in run
    raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
bb.process.ExecutionError: Execution of 'git checkout 2e68773dfca072cb81f219fc3b97ad34fe9d9f94' failed with exit code 128:
fatal: reference is not a tree: 2e68773dfca072cb81f219fc3b97ad34fe9d9f94

this is because that go.bbclass is using a subdir called "src" and that will break the devtool check for upgrades mechanism because it's not looking into a src subdir. So git will fail checkout anything in the parent dir "reference is not a tree"
I think go.bbclass is not setting S correct.

This simple change in the go.bbclass fix it in my case, but this will break others:


# src folder will break devtool upgrade
python go_do_unpack() {
    src_uri = (d.getVar('SRC_URI') or "").split()
    if len(src_uri) == 0:
        return

    fetcher = bb.fetch2.Fetch(src_uri, d)
    for url in fetcher.urls:
        if fetcher.ud[url].type == 'git':
            if fetcher.ud[url].parm.get('destsuffix') is None:
                s_dirname = os.path.basename(d.getVar('S'))
#cause problems                fetcher.ud[url].parm['destsuffix'] = os.path.join(s_dirname, 'src', d.getVar('GO_IMPORT')) + '/'
#FIX
                fetcher.ud[url].parm['destsuffix'] = os.path.join(s_dirname, '', d.getVar('GO_IMPORT')) + '/'
#FIX
    fetcher.unpack(d.getVar('WORKDIR'))
}

# src folder will break devtool upgrade
go_do_configure() {
#cause problems      ln -snf ${S}/src ${B}/
#FIX
    ln -snf ${S} ${B}/
#FIX
}

came up with something simpler, but this will break other stuff:
python go_do_unpack() {
    src_uri = (d.getVar('SRC_URI') or "").split()
    if len(src_uri) == 0:
        return

    fetcher = bb.fetch2.Fetch(src_uri, d)
    for url in fetcher.urls:
        if fetcher.ud[url].type == 'git':
            if fetcher.ud[url].parm.get('destsuffix') is None:
                s_dirname = os.path.basename(d.getVar('S'))
#                fetcher.ud[url].parm['destsuffix'] = os.path.join(s_dirname, 'src', d.getVar('GO_IMPORT')) + '/'
                fetcher.ud[url].parm['destsuffix'] = os.path.join(s_dirname) + '/'
    fetcher.unpack(d.getVar('WORKDIR'))
}

# src folder will break devtool upgrade
go_do_configure() {
#cause problems      ln -snf ${S}/src ${B}/
#FIX
#    install -d ${B}/src/${GO_IMPORT}
#    cp -r ${S}/. ${B}/src/${GO_IMPORT}
     install -d ${B}/src/${GO_IMPORT}/
     rm -r ${B}/src/${GO_IMPORT}/
     ln -snf ${S} ${B}/src/${GO_IMPORT}
#FIX
}

go_do_install() {
	install -d ${D}${libdir}/go/src/${GO_IMPORT}
	tar -C ${B}/src/${GO_IMPORT} -cf - --exclude-vcs --exclude '*.test' --exclude 'testdata' . | \
		tar -C ${D}${libdir}/go/src/${GO_IMPORT} --no-same-owner -xf -
	tar -C ${B} -cf - --exclude-vcs --exclude '*.test' --exclude 'testdata' pkg | \
		tar -C ${D}${libdir}/go --no-same-owner -xf -

	if [ -n "`ls ${B}/${GO_BUILD_BINDIR}/`" ]; then
		install -d ${D}${bindir}
		install -m 0755 ${B}/${GO_BUILD_BINDIR}/* ${D}${bindir}/
	fi
}

Might somebody can say anything about that src dir - as said I don't have any knowledge about GO.

originated here: https://lists.yoctoproject.org/g/yocto/topic/94449982#58372
Comment 1 Randy MacLeod 2022-10-27 14:37:46 UTC
We want to fix this but the devtool maintainer is busy with other work for a whle.
Comment 2 Randy MacLeod 2023-07-26 21:22:06 UTC
Bulk move of 64 bugs to 4.3 M3 after a quick review.
If a bug is actually fixed, please add a commit link and resolve it.
-- Randy for YP bug team.
Comment 3 Trevor Gamblin 2023-09-14 15:47:06 UTC
Moved to 5.0 M1.
Comment 4 Randy MacLeod 2023-12-21 23:28:29 UTC
Bulk move of 52 5.0-M1 bugs to M2.
Comment 5 Randy MacLeod 2024-01-25 14:28:48 UTC
Bulk move of bugs owned by "unassigned" from 5.0-M2 to 5.0-M3.
Comment 6 Fawzi KHABER 2024-02-15 10:24:59 UTC
the issue is still here, the bug cannot be solved using Thomas solution, it does not work. I think it requires a devtool patch.
Comment 7 Randy MacLeod 2024-02-16 00:38:30 UTC
Fawzi, Thanks for checking. We're short of volunteers so it is unlikely to be worked on any time soon. If you or anyone else has a personal/business interest in this feature, please consider implementing the devtool enhancement or asking a Yocto-based OS vendor to do so.
Comment 8 Randy MacLeod 2025-01-30 16:06:37 UTC
Bulk move to 5.2-M3.
Comment 9 Randy MacLeod 2025-01-30 16:17:04 UTC
Bulk move to 5.2-M3 and this time I'll actually specify M3!
Comment 10 Randy MacLeod 2025-02-14 16:46:33 UTC
bulk change to add Saul's non-WR address.
Comment 11 Randy MacLeod 2025-02-14 16:48:17 UTC
Bulk change: Remove Saul's old WR address.