Bug 11484

Summary:	devtool: upgrade: Being able to check for specified source revision is on the specified branch
Product:	[Build System, Metadata & Runtime] OE-Core	Reporter:	Rebecca Chang <rebecca.swee.fun.chang>
Component:	Scripts and Tools	Assignee:	Rebecca Chang <rebecca.swee.fun.chang>
Status:	VERIFIED FIXED	QA Contact:	yeoh ee peng <ee.peng.yeoh>
Severity:	normal
Priority:	Medium+	CC:	bluelightning, ee.peng.yeoh, rebecca.swee.fun.chang, ross.burton
Version:	unspecified
Target Milestone:	2.4 M3
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	Don't know

Description Rebecca Chang 2017-05-09 04:06:24 UTC

Current issue with devtool is it isn't error out when there is no specified branch during devtool upgrade. devtool is checking out the specified source rev directly regardless of the source branch. This has caused bitbake build issue once we do "devtool finish" the package. Bitbake will not be able to fetch the src revision if the branch is not specified in the SRCURI.

devtool should be checking if the specified revision is on the specified branch (either the one specified in -B/--srcbranch, or the one in the SRC_URI entry, or master if none specified) and if it isn't, we should probably error out of the upgrade and tell the user they need to specify the branch.

To reproduce:
Use devtool upgrade --srcrev 5c1c0da352fe67377d88bf62be8cf55aaa84b8e1 ifupdown to upgrade ifupdown package to v0.8.19.

The source rev was not found on master branch but it was tagged on stretch branch.
devtool was able to build the package while bitbake build will failed.

WARNING: ifupdown-0.8.19-r0 do_fetch: Failed to fetch URL git://anonscm.debian.org/git/collab-maint/ifupdown.git, attempting MIRRORS if available
ERROR: ifupdown-0.8.19-r0 do_fetch: Fetcher failure: Unable to find revision 5c1c0da352fe67377d88bf62be8cf55aaa84b8e1 in branch master even from upstream
ERROR: ifupdown-0.8.19-r0 do_fetch: Fetcher failure for URL: 'git://anonscm.debian.org/git/collab-maint/ifupdown.git'. Unable to fetch URL from any source.
ERROR: ifupdown-0.8.19-r0 do_fetch: Function failed: base_do_fetch
ERROR: Logfile of failure stored in: /media/rebeccas/yocto/devtool-ifupdown/tmp/work/corei7-64-poky-linux/ifupdown/0.8.19-r0/temp/log.do_fetch.163732
ERROR: Task (/media/rebeccas/yocto/poky/meta/recipes-core/ifupdown/ifupdown_0.8.19.bb:do_fetch) failed with exit code '1'
NOTE: Tasks Summary: Attempted 820 tasks of which 819 didn't need to be rerun and 1 failed.

Comment 1 Rebecca Chang 2017-08-17 05:36:00 UTC

Submitted patch to mailing list
http://lists.openembedded.org/pipermail/openembedded-core/2017-August/140931.html

Comment 2 Rebecca Chang 2017-08-23 08:00:59 UTC

Patch accepted in upstream.
OE-Core rev: 29ced7387a92aed17b7fe93b1654790a981734c1

Comment 3 yeoh ee peng 2017-10-02 04:26:29 UTC

Verify the fixes (reproduce the issue before verified the fixes)

1) Reproduce the issue successfully using origin/pyro
Prepare the environment: 
First revert to origin/pyro branch in order to get the code base before Rebecca fixes. Then, prepared a github repo for the testing, where the dev1 branch contain commit f26a5d9e53524edc72b9f476a628c35eb82e497b which does not existing in master. 

Steps:
a) Perform devtool add bbexample https://github.com/epyeoh/bbexample.git
b) Perform devtool finish bbexample <local_layer_as_destination>
c) Perform devtool upgrade --srcrev f26a5d9e53524edc72b9f476a628c35eb82e497b bbexample 
d) Perform bitbake bbexample and receive the error
WARNING: bbexample-0.1+gitAUTOINC+f26a5d9e53-r0 do_fetch: Failed to fetch URL git://github.com/epyeoh/bbexample.git;protocol=https, attempting MIRRORS if available
ERROR: bbexample-0.1+gitAUTOINC+f26a5d9e53-r0 do_fetch: Fetcher failure: Unable to find revision f26a5d9e53524edc72b9f476a628c35eb82e497b in branch master even from upstream
ERROR: bbexample-0.1+gitAUTOINC+f26a5d9e53-r0 do_fetch: Fetcher failure for URL: 'git://github.com/epyeoh/bbexample.git;protocol=https'. Unable to fetch URL from any source.
ERROR: bbexample-0.1+gitAUTOINC+f26a5d9e53-r0 do_fetch: Function failed: base_do_fetch
ERROR: Logfile of failure stored in: /workdir/devtool_poky/poky/build/tmp/work/i586-poky-linux/bbexample/0.1+gitAUTOINC+f26a5d9e53-r0/temp/log.do_fetch.94803
ERROR: Task (/workdir/devtool_poky/poky/meta-yocto-bsp/recipes-bbexample/bbexample/bbexample_git.bb:do_fetch) failed with exit code '1'
NOTE: Tasks Summary: Attempted 468 tasks of which 467 didn't need to be rerun and 1 failed.

Comment 4 yeoh ee peng 2017-10-02 04:33:58 UTC

Verify the fixes

2) Verify the fixes from origin/master can upgrade recipe with srcrev exist only in dev1 branch.

a) Checkout to origin/master branch to get the fixes from Rebecca
b) Perform devtool upgrade --srcrev f26a5d9e53524edc72b9f476a628c35eb82e497b bbexample 
c) Verify that the SRC_URI generated contain the branch information
SRC_URI = "git://github.com/epyeoh/bbexample.git;protocol=https;branch=dev"
d) Verify that bitbake bbexample can build successfully
Initialising tasks: 100% |###################################################################################################| Time: 0:00:00
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 1 tasks of which 0 didn't need to be rerun and all succeeded.
pokyuser@6b6751c55719:/workdir/devtool_poky/poky/build$ bitbake bbexample
Loading cache: 100% |########################################################################################################| Time: 0:00:00
Loaded 1301 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION        = "1.34.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.3.2"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta
meta-poky
meta-yocto-bsp    = "pyro:717303e6fbcbbe181ad9645d762eb5a85d934523"

Initialising tasks: 100% |###################################################################################################| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 481 tasks of which 480 didn't need to be rerun and all succeeded.

Comment 5 yeoh ee peng 2017-10-02 04:38:33 UTC

Verify the fixes

2) Verify the fixes from origin/master can prompt user when try to upgrade recipe with srcrev exists on multiple branches (eg. dev2 & dev3) and srcrev deoes not exist in master branch.

a) Verify that dev2 & dev3 branch from https://github.com/epyeoh/bbexample.git
contain commit a1e995ea27eb2e800cee1fc7b37672116aee2305bbexample that does not exist in master.
b) Checkout to origin/master branch to get the fixes from Rebecca
c) Perform devtool upgrade --srcrev a1e995ea27eb2e800cee1fc7b37672116aee2305 bbexample 
d) Verify that devtool will prompt error message
Initialising tasks: 100% |#################################################################################################| Time: 0:00:00
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 3 tasks of which 0 didn't need to be rerun and all succeeded.
ERROR: Revision a1e995ea27eb2e800cee1fc7b37672116aee2305 was found on multiple branches:
  dev2
  dev3
Please provide the correct branch in the devtool command with "--srcbranch" or "-B" option.

Comment 6 yeoh ee peng 2017-10-02 04:57:07 UTC

Verify the fixes

2) Verify the fixes from origin/master can upgrade recipe when provide the srcrev and srcbranch argument.

a) Checkout to origin/master branch to get the fixes from Rebecca
b) Perform devtool upgrade --srcrev a1e995ea27eb2e800cee1fc7b37672116aee2305 --srcbranch dev2 bbexample
c) Verify that the SRC_URI generated contain the branch information & srcrev is correct

SRC_URI = "git://github.com/epyeoh/bbexample.git;protocol=https;branch=dev2"

# Modify these as desired
PV = "0.1+git${SRCPV}"
SRCREV = "a1e995ea27eb2e800cee1fc7b37672116aee2305"

d) Verify bitbake bbexample can build successfully
pokyuser@6b6751c55719:/workdir/devtool_poky/poky/build$ bitbake bbexample
Loading cache: 100% |########################################################################################################| Time: 0:00:00
Loaded 1301 entries from dependency cache.
Parsing recipes: 100% |######################################################################################################| Time: 0:00:00
Parsing of 832 .bb files complete (831 cached, 1 parsed). 1301 targets, 48 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION        = "1.34.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.3.2"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta
meta-poky
meta-yocto-bsp    = "pyro:717303e6fbcbbe181ad9645d762eb5a85d934523"

Initialising tasks: 100% |###################################################################################################| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 481 tasks of which 480 didn't need to be rerun and all succeeded.

Comment 7 yeoh ee peng 2017-11-08 03:00:37 UTC

Verified that the fixes had resolved the issue. Please refer to above verification log.