Bug 11431 - Crops-ESDK: devtool is unable to add recipe for nodejs-native recipe
Summary: Crops-ESDK: devtool is unable to add recipe for nodejs-native recipe
Status: RESOLVED NOTABUG
Alias: None
Product: CROPS
Classification: Yocto Project Subprojects
Component: crops-default (show other bugs)
Version: 2.3
Hardware: x86 Multiple
: Medium normal
Target Milestone: 2.7
Assignee: Anuj Mittal
QA Contact: nagarajagopal
URL:
Whiteboard: Backport to 2.6.1?
Depends on:
Blocks: 10653
  Show dependency tree
 
Reported: 2017-04-27 16:03 UTC by Libertad
Modified: 2019-05-09 00:54 UTC (History)
9 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Libertad 2017-04-27 16:03:52 UTC
DESCRIPTION:

Within the CROPS container the eSDK, devtool cannot add recipe for nodejs.


STATUS: FAILED
BUILD: 2.3 M4: rc2-7a0e795373653886452a7a2992ced10080711c26
ENVIRONMENT: user-preference (any supported environment)
STEPS TO REPRODUCE: 

lets say variable npm = npm://registry.npmjs.org;name=winston;version=2.2.0

Step 1. Initiate your Crops-esdk environment as it says in wiki https://github.com/crops/docker-win-mac-docs/wiki
Step 2: source environment-setup-i586-poky-linux

Step 3: run command which devtool

Step 4: devtool add npm



EXPECTED OUTCOME: 

devtool command to be successful 

ERROR:

sdkuser@72aec81c01b4:/workdir$ devtool add npm://registry.npmjs.org;name=winston;version=2.2.0
Loading cache: 100% |##########################################################################################################################################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
Parsing recipes: 100% |########################################################################################################################################################| Time: 0:00:00
Parsing of 831 .bb files complete (830 cached, 1 parsed). 1300 targets, 48 skipped, 0 masked, 0 errors.
Traceback (most recent call last):
  File "/workdir/sysroots/x86_64-pokysdk-linux/usr/bin/devtool", line 351, in <module>
    ret = main()
  File "/workdir/sysroots/x86_64-pokysdk-linux/usr/bin/devtool", line 338, in main
    ret = args.func(args, config, basepath, workspace)
  File "/workdir/layers/build/scripts/lib/devtool/standard.py", line 132, in add
    ensure_npm(config, basepath, args.fixed_setup)
  File "/workdir/layers/build/scripts/lib/devtool/__init__.py", line 272, in ensure_npm
    rd = tinfoil.parse_recipe('nodejs-native')
  File "/workdir/layers/build/bitbake/lib/bb/tinfoil.py", line 384, in parse_recipe
    fn = self.get_recipe_file(pn)
  File "/workdir/layers/build/bitbake/lib/bb/tinfoil.py", line 373, in get_recipe_file
    raise bb.providers.NoProvider('Unable to find any recipe file matching "%s"' % pn)
bb.providers.NoProvider: Unable to find any recipe file matching "nodejs-native"
Comment 1 Joshua Lock 2017-05-08 21:54:24 UTC
nodejs-native isn't part of oe-core, you need to add a layer which includes nodejs (such as meta-nodejs or meta-oe) before devtool can correctly handle nodejs recipes.
Comment 2 brian avery 2017-06-29 16:19:44 UTC
Since nodejs-native is not part of oe-core yet, we can revisit this later if it moves into oe-core at some in the future.
Comment 3 nagarajagopal 2018-06-04 09:23:07 UTC
still issue not solved, so reopened
Comment 4 Tim Orling 2018-06-07 15:08:13 UTC
The Test Case instructions need to be updated to add the meta-oe layer, as nodejs recipe [1] will likely not be moved into oe-core.

[1] http://layers.openembedded.org/layerindex/recipe/4427/
Comment 5 Anuj Mittal 2018-06-11 01:50:37 UTC
Test case will be updated.
Comment 6 nagarajagopal 2018-06-13 03:57:04 UTC
 We will wait for test case to be updated.
 we have marked the test case 1610 as failed for release 2.3.4.rc1.
Comment 7 nagarajagopal 2018-06-26 09:13:15 UTC
 We will wait for test case to be updated.
 we have marked the test case 1610 as failed for release 2.6 M1 rc1
Comment 8 pankaj yadav 2018-07-26 06:06:34 UTC
We will wait for test case to be updated.
we have marked the test case 1610 as failed for release 2.6 M2 rc1
Comment 9 pankaj yadav 2018-08-14 01:47:41 UTC
Test case has been updated and verified to be working on the releases 2.6 M2 rc1 and 2.5.1.rc1.
Comment 10 pankaj yadav 2018-11-22 03:48:08 UTC
Issue is reproducible with 2.4.4RC1 release with suggested work around as mentioned below, Issue is not seen 2.6 M2 rc1 release with same work around:

Steps to Reproduce:
1. Initiate your Crops-esdk environment as it says in wiki https://github.com/crops/docker-win-mac-docs/wiki 
2. source environment-setup-i586-poky-linux 
3. run command which devtool 
4. 4a) git clone git://git.openembedded.org/meta-openembedded in layers/build directory 
4b) Add meta-openembedded/meta-oe in bblayer.conf as mentioned below: 
${SDKBASEMETAPATH}/layers/build/meta-openembedded/meta-oe \ 
4c) devtool add "npm://registry.npmjs.org;name=npm;version=2.2.0" 

Logs are attached for the same :
sdkuser@c18c715ca295:/workdir$ ls
bitbake-cookerdaemon.log conf layers sstate-cache version-core2-64-poky-linux
buildtools downloads preparing_build_system.log sysroots workspace
cache environment-setup-core2-64-poky-linux site-config-core2-64-poky-linux tmp
sdkuser@c18c715ca295:/workdir$ source environment-setup-core2-64-poky-linux
SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.
sdkuser@c18c715ca295:/workdir$ devtool add "npm://registry.npmjs.org;name=npm;version=2.2.0"
NOTE: Starting bitbake server...
ERROR: Unable to start bitbake server
ERROR: Last 10 lines of server log for this session (/workdir/bitbake-cookerdaemon.log):
self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset)
File "/workdir/layers/build/bitbake/lib/bb/cooker.py", line 197, in _init_
self.initConfigurationData()
File "/workdir/layers/build/bitbake/lib/bb/cooker.py", line 356, in initConfigurationData
self.databuilder.parseBaseConfiguration()
File "/workdir/layers/build/bitbake/lib/bb/cookerdata.py", line 316, in parseBaseConfiguration
raise bb.BBHandledException
bb.BBHandledException
ERROR: Layer openembedded-layer is not compatible with the core layer which only supports these series: rocko (layer is compatible with thud)
ERROR: Layer openembedded-layer is not compatible with the core layer which only supports these series: rocko (layer is compatible with thud)

ERROR: Unable to start bitbake server
ERROR: Last 10 lines of server log for this session (/workdir/bitbake-cookerdaemon.log):
self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset)
File "/workdir/layers/build/bitbake/lib/bb/cooker.py", line 197, in _init_
self.initConfigurationData()
File "/workdir/layers/build/bitbake/lib/bb/cooker.py", line 356, in initConfigurationData
self.databuilder.parseBaseConfiguration()
File "/workdir/layers/build/bitbake/lib/bb/cookerdata.py", line 316, in parseBaseConfiguration
raise bb.BBHandledException
bb.BBHandledException
ERROR: Layer openembedded-layer is not compatible with the core layer which only supports these series: rocko (layer is compatible with thud)
ERROR: Layer openembedded-layer is not compatible with the core layer which only supports these series: rocko (layer is compatible with thud)

sdkuser@c18c715ca295:/workdir$
Comment 11 Anuj Mittal 2018-11-22 04:29:44 UTC
(In reply to comment #10)
> Issue is reproducible with 2.4.4RC1 release with suggested work around as
> mentioned below, Issue is not seen 2.6 M2 rc1 release with same work around:
> 
> Steps to Reproduce:
> 1. Initiate your Crops-esdk environment as it says in wiki
> https://github.com/crops/docker-win-mac-docs/wiki 
> 2. source environment-setup-i586-poky-linux 
> 3. run command which devtool 
> 4. 4a) git clone git://git.openembedded.org/meta-openembedded in
> layers/build directory 
> 4b) Add meta-openembedded/meta-oe in bblayer.conf as mentioned below: 
> ${SDKBASEMETAPATH}/layers/build/meta-openembedded/meta-oe \ 

If you're trying out rocko, you should checkout rocko layer ... Please retry.
Comment 12 pankaj yadav 2018-11-22 09:21:22 UTC
After checkout rocko layer, below mentioned step is getting passed 
step 4 c)devtool add "npm://registry.npmjs.org;name=npm;version=2.2.0"  

but  "devtool build npm" is getting failed as mentioned in attached logs:

Please find the logs for the same :
 
sdkuser@ff591c534f30:/workdir$ devtool build npm
NOTE: Starting bitbake server...
Loading cache: 100% |###################################################################################################################################| Time: 0:00:01
Loaded 2029 entries from dependency cache.
Parsing recipes: 100% |#################################################################################################################################| Time: 0:00:00
Parsing of 1431 .bb files complete (1430 cached, 1 parsed). 2030 targets, 53 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |##############################################################################################################################| Time: 0:00:01
Checking sstate mirror object availability: 100% |######################################################################################################| Time: 0:00:01
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: npm: compiling from external source tree /workdir/workspace/sources/npm
ERROR: npm-2.2.0-r0 do_compile: Function failed: do_compile (log file is located at /workdir/tmp/work/core2-64-poky-linux/npm/2.2.0-r0/temp/log.do_compile.55881)
ERROR: Logfile of failure stored in: /workdir/tmp/work/core2-64-poky-linux/npm/2.2.0-r0/temp/log.do_compile.55881
Log data follows:
| DEBUG: Executing python function externalsrc_compile_prefunc
| NOTE: npm: compiling from external source tree /workdir/workspace/sources/npm
| DEBUG: Python function externalsrc_compile_prefunc finished
| DEBUG: Executing shell function do_compile
| npm ERR! As of npm@5, the npm cache self-heals from corruption issues and data extracted from the cache is guaranteed to be valid. If you want to make sure everything is consistent, use 'npm cache verify' instead.
| npm ERR!
| npm ERR! If you're sure you want to delete the entire cache, rerun this command with --force.
|
| npm ERR! A complete log of this run can be found in:
| npm ERR!     /workdir/tmp/work/core2-64-poky-linux/npm/2.2.0-r0/npm_cache/_logs/2018-11-22T08_33_22_260Z-debug.log
| WARNING: exit code 1 from a shell command.
| ERROR: Function failed: do_compile (log file is located at /workdir/tmp/work/core2-64-poky-linux/npm/2.2.0-r0/temp/log.do_compile.55881)
ERROR: Task (/workdir/workspace/recipes/npm/npm_2.2.0.bb:do_compile) failed with exit code '1'
NOTE: Tasks Summary: Attempted 446 tasks of which 270 didn't need to be rerun and 1 failed.
NOTE: The errors for this build are stored in /workdir/tmp/log/error-report/error_report_20181122082037.txt
You can send the errors to a reports server by running:
  send-error-report /workdir/tmp/log/error-report/error_report_20181122082037.txt [-s server]
NOTE: The contents of these logs will be posted in public if you use the above command with the default server. Please ensure you remove any identifying or proprietary information when prompted before sending.

Summary: 1 task failed:
  /workdir/workspace/recipes/npm/npm_2.2.0.bb:do_compile
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
sdkuser@ff591c534f30:/workdir$
Comment 13 Anuj Mittal 2018-11-26 07:09:19 UTC
(In reply to comment #12)
> After checkout rocko layer, below mentioned step is getting passed 
> step 4 c)devtool add "npm://registry.npmjs.org;name=npm;version=2.2.0"  
> 
> but  "devtool build npm" is getting failed as mentioned in attached logs:
> 
> Please find the logs for the same :
>  
> sdkuser@ff591c534f30:/workdir$ devtool build npm
> NOTE: Starting bitbake server...
> Loading cache: 100%
> |############################################################################
> #######################################################| Time: 0:00:01
> Loaded 2029 entries from dependency cache.
> Parsing recipes: 100%
> |############################################################################
> #####################################################| Time: 0:00:00
> Parsing of 1431 .bb files complete (1430 cached, 1 parsed). 2030 targets, 53
> skipped, 0 masked, 0 errors.
> NOTE: Resolving any missing task queue dependencies
> Initialising tasks: 100%
> |############################################################################
> ##################################################| Time: 0:00:01
> Checking sstate mirror object availability: 100%
> |############################################################################
> ##########################| Time: 0:00:01
> NOTE: Executing SetScene Tasks
> NOTE: Executing RunQueue Tasks
> NOTE: npm: compiling from external source tree /workdir/workspace/sources/npm
> ERROR: npm-2.2.0-r0 do_compile: Function failed: do_compile (log file is
> located at
> /workdir/tmp/work/core2-64-poky-linux/npm/2.2.0-r0/temp/log.do_compile.55881)
> ERROR: Logfile of failure stored in:
> /workdir/tmp/work/core2-64-poky-linux/npm/2.2.0-r0/temp/log.do_compile.55881
> Log data follows:
> | DEBUG: Executing python function externalsrc_compile_prefunc
> | NOTE: npm: compiling from external source tree
> /workdir/workspace/sources/npm
> | DEBUG: Python function externalsrc_compile_prefunc finished
> | DEBUG: Executing shell function do_compile
> | npm ERR! As of npm@5, the npm cache self-heals from corruption issues and
> data extracted from the cache is guaranteed to be valid. If you want to make
> sure everything is consistent, use 'npm cache verify' instead.
> | npm ERR!
> | npm ERR! If you're sure you want to delete the entire cache, rerun this
> command with --force.
> |
> | npm ERR! A complete log of this run can be found in:
> | npm ERR!    
> /workdir/tmp/work/core2-64-poky-linux/npm/2.2.0-r0/npm_cache/_logs/2018-11-
> 22T08_33_22_260Z-debug.log
> | WARNING: exit code 1 from a shell command.
> | ERROR: Function failed: do_compile (log file is located at
> /workdir/tmp/work/core2-64-poky-linux/npm/2.2.0-r0/temp/log.do_compile.55881)

This is a different issue ... We need this commit to be backported:

https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit?id=d38e1e2c2ea4646b34ea6282d3d7620df5b0374b

Armin, can you please include this in rocko?
Comment 14 Armin Kuster 2018-11-26 14:23:21 UTC
backported to stable/rocko-nmut for build test before requesting a merge
Comment 15 Armin Kuster 2018-11-26 23:13:32 UTC
its in stable/rocko-next. waiting for merging to proper
Comment 16 Anuj Mittal 2019-02-25 02:50:39 UTC
(In reply to comment #15)
> its in stable/rocko-next. waiting for merging to proper

Looks like this was never merged. Also not sure why this was marked for 2.7/2.6 ... 

Can this be merged in rocko? If not, since this was not reproducible in 2.6, can this be closed?
Comment 17 Anuj Mittal 2019-05-09 00:54:21 UTC
I don't see the problem as indicated above and rocko isn't maintained anymore.