Bug 9043

Summary: Extensible SDK: sdk-update does not work from local path
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Yi Zhao <yi.zhao>
Component: devtools / tool chainAssignee: Juro Bystricky <juro.bystricky>
Status: VERIFIED FIXED QA Contact: Yi Zhao <yi.zhao>
Severity: normal    
Priority: Medium+ CC: alexandru.c.georgescu, anibal.limon, bluelightning, bogdanx.a.voiculescu, costin.c.constantin, ke.zou, meta.mr.watcher, meta.watcher, yi.zhao
Version: 2.1   
Target Milestone: 2.1 M3   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: Regression (Used to work)
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
Console output of bitbake -c testsdkext none

Description Yi Zhao 2016-02-04 02:33:03 UTC
Git rev: master/3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c

I got an error when running sdk-update from a local path on 2.1_M2

#######################
$ devtool sdk-update /buildarea/poky_sdk
Traceback (most recent call last):
  File "/buildarea/poky_sdk_old/sysroots/x86_64-pokysdk-linux//usr/bin/devtool", line 305, in <module>
    ret = main()
  File "/buildarea/poky_sdk_old/sysroots/x86_64-pokysdk-linux//usr/bin/devtool", line 292, in main
    ret = args.func(args, config, basepath, workspace)
  File "/buildarea/poky_sdk_old/layers/build/scripts/lib/devtool/sdk.py", line 138, in sdk_update
    sstate_dir = os.path.join(newsdk_path, 'sstate-cache')
NameError: global name 'newsdk_path' is not defined
#######################


Steps:
1. Build an ext_sdk toolchain against the old git rev
2. Build another ext_sdk toolchain against the new git rev
3. Install the old ext_sdk into poky_sdk_old directory, install the new ext_sdk into poky_sdk_new directory
4. cd poky_sdk_old, setup the environment
5. Run devtool sdk-update  poky_sdk_new
Comment 1 Bogdan Alexandru Voiculescu 2016-02-05 10:27:21 UTC
Argument sdk-update seems to be removed:
devtool sdk-update http://localhost:8000
ERROR: argument <subcommand>: invalid choice: 'sdk-update' (choose from 'create-workspace', 'deploy-target', 'undeploy-target', 'search', 'build', 'add', 'modify', 'extract', 'sync', 'update-recipe', 'status', 'reset', 'upgrade', 'edit-recipe', 'configure-help', 'build-image')
usage: devtool [--basepath BASEPATH] [--bbpath BBPATH] [-d] [-q]
               [--color COLOR] [-h]
               <subcommand> ...
Comment 2 Paul Eggleton 2016-02-25 00:19:09 UTC
sdk-update hasn't been removed. You won't find it in the list of commands supported by devtool outside of the extensible SDK however, it's only available within the extensible SDK environment.
Comment 3 Juro Bystricky 2016-02-26 21:06:21 UTC
Could reproduce with master e0fd96442a20e3fe7953bf38fc02520cd0dd4d6a on Feb 26/2016
Comment 4 Juro Bystricky 2016-02-29 17:32:47 UTC
This should fix it, (about to test the hypothesis):

diff --git a/scripts/lib/devtool/sdk.py b/scripts/lib/devtool/sdk.py
index fbf2e79..22c5279 100644
--- a/scripts/lib/devtool/sdk.py
+++ b/scripts/lib/devtool/sdk.py
@@ -133,6 +133,7 @@ def sdk_update(args, config, basepath, workspace):
             logger.debug("Found conf/locked-sigs.inc in %s" % updateserver)
         update_dict = generate_update_dict(new_locked_sig_file_path, old_locked_sig_file_path)
         logger.debug("update_dict = %s" % update_dict)
+        newsdk_path = updateserver
         sstate_dir = os.path.join(newsdk_path, 'sstate-cache')
         if not os.path.exists(sstate_dir):
             logger.error("sstate-cache directory not found under %s" % newsdk_path)
Comment 5 Aníbal Limón 2016-02-29 18:20:04 UTC
(In reply to comment #4)
> This should fix it, (about to test the hypothesis):
> 
> diff --git a/scripts/lib/devtool/sdk.py b/scripts/lib/devtool/sdk.py
> index fbf2e79..22c5279 100644
> --- a/scripts/lib/devtool/sdk.py
> +++ b/scripts/lib/devtool/sdk.py
> @@ -133,6 +133,7 @@ def sdk_update(args, config, basepath, workspace):
>              logger.debug("Found conf/locked-sigs.inc in %s" % updateserver)
>          update_dict = generate_update_dict(new_locked_sig_file_path,
> old_locked_sig_file_path)
>          logger.debug("update_dict = %s" % update_dict)
> +        newsdk_path = updateserver
>          sstate_dir = os.path.join(newsdk_path, 'sstate-cache')
>          if not os.path.exists(sstate_dir):
>              logger.error("sstate-cache directory not found under %s" %
> newsdk_path)

Hi Juro,

You could test using the testsdkext task, you need to inherit testsdk class in local.conf and then remove comments [1] of the local test code.

$ bitbake core-image-minimal -c populate_sdk_ext
$ bitbake core-image-minimal -c testsdkext

[1] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/sdkext/sdk_update.py#n33
Comment 6 Juro Bystricky 2016-02-29 20:41:20 UTC
At this point SDK installation does not work.

local.conf:
USER_CLASSES += " testsdk " 

$ bitbake core-image-minimal -c testsdkext


ERROR: core-image-minimal-1.0-r0 do_testsdkext: Couldn't install the SDK EXT:
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.0+snapshot
===================================================================================
You are about to install the SDK to "/data/master/poky/build-9043-fixed1/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/testsdkext/tc". Proceed[Y/n]? Y
Extracting SDK..........................................done
Setting it up...
Extracting buildtools...
Preparing build system...
ERROR: SDK preparation failed: see /data/master/poky/build-9043-fixed1/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/testsdkext/tc/preparing_build_system.log

ERROR: core-image-minimal-1.0-r0 do_testsdkext: Function failed: do_testsdkext
ERROR: Logfile of failure stored in: /data/master/poky/build-9043-fixed1/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_testsdkext.28359
ERROR: Task 0 (/data/master/poky/meta/recipes-core/images/core-image-minimal.bb, do_testsdkext) failed with exit code '1'
NOTE: Tasks Summary: Attempted 1 tasks of which 0 didn't need to be rerun and 1 failed.

It is probably related to renaming layers (meta-poky).
Manual installation of SDK generates the same error.
Comment 7 Aníbal Limón 2016-02-29 20:43:44 UTC
(In reply to comment #6)
> At this point SDK installation does not work.
> 
> local.conf:
> USER_CLASSES += " testsdk " 
> 
> $ bitbake core-image-minimal -c testsdkext
> 
> 
> ERROR: core-image-minimal-1.0-r0 do_testsdkext: Couldn't install the SDK EXT:
> Poky (Yocto Project Reference Distro) Extensible SDK installer version
> 2.0+snapshot
> =============================================================================
> ======
> You are about to install the SDK to
> "/data/master/poky/build-9043-fixed1/tmp/work/qemux86-poky-linux/core-image-
> minimal/1.0-r0/testsdkext/tc". Proceed[Y/n]? Y
> Extracting SDK..........................................done
> Setting it up...
> Extracting buildtools...
> Preparing build system...
> ERROR: SDK preparation failed: see
> /data/master/poky/build-9043-fixed1/tmp/work/qemux86-poky-linux/core-image-
> minimal/1.0-r0/testsdkext/tc/preparing_build_system.log
> 
> ERROR: core-image-minimal-1.0-r0 do_testsdkext: Function failed:
> do_testsdkext
> ERROR: Logfile of failure stored in:
> /data/master/poky/build-9043-fixed1/tmp/work/qemux86-poky-linux/core-image-
> minimal/1.0-r0/temp/log.do_testsdkext.28359
> ERROR: Task 0
> (/data/master/poky/meta/recipes-core/images/core-image-minimal.bb,
> do_testsdkext) failed with exit code '1'
> NOTE: Tasks Summary: Attempted 1 tasks of which 0 didn't need to be rerun
> and 1 failed.
> 
> It is probably related to renaming layers (meta-poky).
> Manual installation of SDK generates the same error.

Yes there was a break in the weekend due to unative class change, i expect to be fixed today.

Cheers,
    alimon
Comment 8 Juro Bystricky 2016-03-01 22:22:02 UTC
Created attachment 3014 [details]
Console output of bitbake -c testsdkext
Comment 9 Juro Bystricky 2016-03-01 22:23:39 UTC
(In reply to comment #7)
> (In reply to comment #6)
> > At this point SDK installation does not work.
> > 
> > local.conf:
> > USER_CLASSES += " testsdk " 
> > 
> > $ bitbake core-image-minimal -c testsdkext
> > 
> > 
> > ERROR: core-image-minimal-1.0-r0 do_testsdkext: Couldn't install the SDK EXT:
> > Poky (Yocto Project Reference Distro) Extensible SDK installer version
> > 2.0+snapshot
> > =============================================================================
> > ======
> > You are about to install the SDK to
> > "/data/master/poky/build-9043-fixed1/tmp/work/qemux86-poky-linux/core-image-
> > minimal/1.0-r0/testsdkext/tc". Proceed[Y/n]? Y
> > Extracting SDK..........................................done
> > Setting it up...
> > Extracting buildtools...
> > Preparing build system...
> > ERROR: SDK preparation failed: see
> > /data/master/poky/build-9043-fixed1/tmp/work/qemux86-poky-linux/core-image-
> > minimal/1.0-r0/testsdkext/tc/preparing_build_system.log
> > 
> > ERROR: core-image-minimal-1.0-r0 do_testsdkext: Function failed:
> > do_testsdkext
> > ERROR: Logfile of failure stored in:
> > /data/master/poky/build-9043-fixed1/tmp/work/qemux86-poky-linux/core-image-
> > minimal/1.0-r0/temp/log.do_testsdkext.28359
> > ERROR: Task 0
> > (/data/master/poky/meta/recipes-core/images/core-image-minimal.bb,
> > do_testsdkext) failed with exit code '1'
> > NOTE: Tasks Summary: Attempted 1 tasks of which 0 didn't need to be rerun
> > and 1 failed.
> > 
> > It is probably related to renaming layers (meta-poky).
> > Manual installation of SDK generates the same error.
> 
> Yes there was a break in the weekend due to unative class change, i expect
> to be fixed today.
> 
> Cheers,
>     alimon

OK, it worked today, tests passed. (See the attached log).
I'll post the patch to the mailing list.
Comment 10 Juro Bystricky 2016-03-02 21:01:32 UTC
I posted two patches to the mailing list.
Comment 12 Yi Zhao 2016-03-18 07:02:34 UTC
Verified it with 2.1 M3 RC1.
Git rev: master/8debfea81e69d038bd2d56314b272cb74f5582ed

I found another issue and filed a new bug 9301 to report it.