11425 2017-04-27 02:35:36 +0000 johnny-five npm example fails, npm not found 2017-05-02 10:22:59 +0000 1 1 7 Build System, Metadata & Runtime BitBake bitbake unspecified x86 x86_64 RESOLVED WORKSFORME Medium+ normal 2.0.4 1 minggang.wang patrick.ohly bluelightning henry.bruce mikko.ylinen poky.bs.watcher poky.watcher --- --- No (bug/feature does not impact docs) oldest_to_newest 72884 0 minggang.wang 2017-04-27 02:35:36 +0000 Environment: Ubuntu 16.04, build without docker, follow the instruction https://github.com/intel/intel-iot-refkit Triggering Action: 1. Add the following lines to local.conf SUPPORTED_RECIPES_CHECK_append = " warn" SUPPORTED_RECIPES_CHECK_remove = "fatal" REFKIT_IMAGE_EXTRA_INSTALL_append = " johnny-five" 2. run command $bitbake refkit-image-common Expectation: Build successfully Actual Result: npm view johnny-five@0.9.59 dist.tarball --registry http://registry.npmjs.org failed with exit code 127, output: /bin/sh: 1: npm: not found ERROR: johnny-five-0.9.59-r0 do_fetch: Fetcher failure for URL: 'npm://registry.npmjs.org;name=johnny-five;version=0.9.59'. Unable to fetch URL from any source. ERROR: johnny-five-0.9.59-r0 do_fetch: Function failed: base_do_fetch ERROR: Logfile of failure stored in: /home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/temp/log.do_fetch.26822 ERROR: Task (/home/minggang/work/intel-iot-refkit/openembedded-core/../meta-iot-web/recipes-devtools/johnny-five/johnny-five_0.9.59.bb:do_fetch) failed with exit code '1' Reproducibility: Yes 72904 1 patrick.ohly 2017-04-27 15:13:37 +0000 Which revision of intel-iot-refkit is this with? I tried with dbc675f0f97052ca2a9563d4a4d4b1467a88681e (almost current master) and for me at least do_fetch worked. "bitbake johnny-five" still fails, though: $ bitbake johnny-five ... meta-selftest = "HEAD:ba2884f6ad3a4e746fc80cbd707f83fa8abd4210" meta-intel = "HEAD:60c270394755c36d3cf2d1df61b3795521e1caa0" meta-refkit = "HEAD:dbc675f0f97052ca2a9563d4a4d4b1467a88681e" meta-oic = "HEAD:b9ba736871b8f8b741cc9c1d88dda56e63a35091" meta-iot-web = "HEAD:e4310f8a34a985fa62eb66c7742088dc7943bedb" meta-iotqa = "HEAD:dbc675f0f97052ca2a9563d4a4d4b1467a88681e" meta-security = "HEAD:6674749274e47937af640ffaa2480e26a362e5f6" meta-security-isafw = "HEAD:834272c2471a86aca826170036598889b5dd552a" meta-intel-realsense = "HEAD:1d50cb5b82926f8e69a78999e9e8d2ffe2169897" meta-clang = "HEAD:0cb6aaa39d22c461c076b145cecf4d7e8dae2c7f" meta-filesystems meta-gnome meta-networking meta-oe meta-perl meta-python = "HEAD:cdb428e7c49899675ee7b7a43f6cecdb5b4c2546" meta-integrity meta-security-framework meta-security-smack = "root-ca-cert:baac71ebfe20498edfeb057721161c102508fcf2" ... ERROR: johnny-five-0.9.59-r0 do_install: Function failed: do_install (log file is located at /fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/temp/log.do_install.8675) ERROR: Logfile of failure stored in: /fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/temp/log.do_install.8675 Log data follows: ... | DEBUG: Executing shell function do_install | npm ERR! Linux 4.9.0-0.bpo.2-amd64 | npm ERR! argv "/fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/node" "/fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/npm" "install" "--prefix" "/fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/image/usr" "-g" "--arch=x64" "--target_arch=x64" "--production" "--no-registry" | npm ERR! node v4.5.0 | npm ERR! npm v2.15.9 | | npm ERR! registry URL is required | npm ERR! | npm ERR! If you need help, you may report this error at: | npm ERR! <https://github.com/npm/npm/issues> | npm ERR! Linux 4.9.0-0.bpo.2-amd64 | npm ERR! argv "/fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/node" "/fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/npm" "install" "--prefix" "/fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/image/usr" "-g" "--arch=x64" "--target_arch=x64" "--production" "--no-registry" ... Paul, any idea what that is about? Mikko, do we currently test npm examples in refkit? One can check with "bitbake -c devshell johnny-five" whether there is a npm: $ bitbake -c devshell johnny-five ... # which npm /fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/npm # npm --version 2.15.9 Minggang, is there really no npm in your corresponding johnny-five/0.9.59-r0/recipe-sysroot-native? Paul, a 72923 2 minggang.wang 2017-04-28 02:04:00 +0000 Hi Patrick, I checked the path, there is a npm under tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin. Actually, I tried to workaround this problem by adding in local.conf: HOSTTOOLS_append = " npm" Then, it reports the same error exactly as you described above. 72925 3 patrick.ohly 2017-04-28 06:40:00 +0000 (In reply to comment #2) > Hi Patrick, I checked the path, there is a npm under > tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot- > native/usr/bin. Than that's the one it should have used. Which revision of intel-iot-refkit are you using? > Actually, I tried to workaround this problem by adding in local.conf: > HOSTTOOLS_append = " npm" > > Then, it reports the same error exactly as you described above. As a workaround that's okay, but it's not the right solution for the "npm not found" problem, because we don't want to depend on some random version of npm on the build host. 72926 4 minggang.wang 2017-04-28 06:49:53 +0000 (In reply to comment #3) > (In reply to comment #2) > > Hi Patrick, I checked the path, there is a npm under > > tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot- > > native/usr/bin. > > Than that's the one it should have used. > > Which revision of intel-iot-refkit are you using? > BitBake Build Tool Core version 1.33.2 > > Actually, I tried to workaround this problem by adding in local.conf: > > HOSTTOOLS_append = " npm" > > > > Then, it reports the same error exactly as you described above. > > As a workaround that's okay, but it's not the right solution for the "npm > not found" problem, because we don't want to depend on some random version > of npm on the build host. 72928 5 patrick.ohly 2017-04-28 07:48:27 +0000 (In reply to comment #4) > (In reply to comment #3) > > (In reply to comment #2) > > > Hi Patrick, I checked the path, there is a npm under > > > tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot- > > > native/usr/bin. > > > > Than that's the one it should have used. > > > > Which revision of intel-iot-refkit are you using? > > > BitBake Build Tool Core version 1.33.2 No, I meant "which git revision of intel-iot-refkit" are you using? One more test (without your HOSTTOOLS workaround): please run "bitbake -c devshell johnny-five", then in the console prompt of the devshell, type "which npm" and "echo $PATH". Does that find the npm that you have and if not, why? 72930 6 mikko.ylinen 2017-04-28 07:54:20 +0000 (In reply to comment #0) > Environment: > Ubuntu 16.04, build without docker, follow the instruction > https://github.com/intel/intel-iot-refkit > > Triggering Action: > 1. Add the following lines to local.conf > SUPPORTED_RECIPES_CHECK_append = " warn" > SUPPORTED_RECIPES_CHECK_remove = "fatal" > REFKIT_IMAGE_EXTRA_INSTALL_append = " johnny-five" > > 2. run command $bitbake refkit-image-common > > Expectation: > Build successfully > > Actual Result: > npm view johnny-five@0.9.59 dist.tarball --registry > http://registry.npmjs.org failed with exit code 127, output: > /bin/sh: 1: npm: not found > We don't install npm in refkit-image-common. refkit-image-gateway should have it. > ERROR: johnny-five-0.9.59-r0 do_fetch: Fetcher failure for URL: > 'npm://registry.npmjs.org;name=johnny-five;version=0.9.59'. Unable to fetch > URL from any source. > ERROR: johnny-five-0.9.59-r0 do_fetch: Function failed: base_do_fetch > ERROR: Logfile of failure stored in: > /home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit- > linux/johnny-five/0.9.59-r0/temp/log.do_fetch.26822 > ERROR: Task > (/home/minggang/work/intel-iot-refkit/openembedded-core/../meta-iot-web/ > recipes-devtools/johnny-five/johnny-five_0.9.59.bb:do_fetch) failed with > exit code '1' Is your proxy correctly set? Without the proxy setting wget fails. I'm seeing the install failure too. 72932 7 patrick.ohly 2017-04-28 09:51:53 +0000 On #yocto IRC, Paul said about the do_install error: it's basically because some dependency in the .json file isn't being correctly satisfied, so npm thinks it needs a registry, but we are telling it not to use one and so it blows up So that's something else, perhaps related to this specific example. I'm no NodeJS expert. 72963 8 bluelightning 2017-05-01 23:25:46 +0000 The npm not found issue is caused by a missing dependency from do_fetch on nodejs-native i.e. do_fetch[depends] += "nodejs-native:do_populate_sysroot" I will send a patch to add that to npm.bbclass shortly. 72964 9 bluelightning 2017-05-01 23:52:06 +0000 Actually, scratch that last comment - that can't be true, base.bbclass should already be adding that dependency. Will investigate. 72967 10 mikko.ylinen 2017-05-02 04:45:10 +0000 (In reply to comment #9) > Actually, scratch that last comment - that can't be true, base.bbclass > should already be adding that dependency. Will investigate. My understanding is npm is not found in the final image. See comment #6. That is expected. 72968 11 minggang.wang 2017-05-02 05:23:39 +0000 (In reply to comment #5) > (In reply to comment #4) > > (In reply to comment #3) > > > (In reply to comment #2) > > > > Hi Patrick, I checked the path, there is a npm under > > > > tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot- > > > > native/usr/bin. > > > > > > Than that's the one it should have used. > > > > > > Which revision of intel-iot-refkit are you using? > > > > > BitBake Build Tool Core version 1.33.2 > The SHA is 4d087e62975ff9ba > No, I meant "which git revision of intel-iot-refkit" are you using? > > One more test (without your HOSTTOOLS workaround): please run "bitbake -c > devshell johnny-five", then in the console prompt of the devshell, type > "which npm" and "echo $PATH". Does that find the npm that you have and if > not, why? When I was running (without adding the HOSTTOOLS) "bitbake -c devshell johnny-five", reports: /bin/sh: 1: npm: not found ERROR: johnny-five-0.9.59-r0 do_fetch: Fetcher failure for URL: 'npm://registry.npmjs.org;name=johnny-five;version=0.9.59'. Unable to fetch URL from any source. ERROR: johnny-five-0.9.59-r0 do_fetch: Function failed: base_do_fetch ERROR: Logfile of failure stored in: /home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/temp/log.do_fetch.27829 ERROR: Task (/home/minggang/work/intel-iot-refkit/openembedded-core/../meta-iot-web/recipes-devtools/johnny-five/johnny-five_0.9.59.bb:do_fetch) failed with exit code '1' 72969 12 minggang.wang 2017-05-02 05:27:09 +0000 (In reply to comment #6) > (In reply to comment #0) > > Environment: > > Ubuntu 16.04, build without docker, follow the instruction > > https://github.com/intel/intel-iot-refkit > > > > Triggering Action: > > 1. Add the following lines to local.conf > > SUPPORTED_RECIPES_CHECK_append = " warn" > > SUPPORTED_RECIPES_CHECK_remove = "fatal" > > REFKIT_IMAGE_EXTRA_INSTALL_append = " johnny-five" > > > > 2. run command $bitbake refkit-image-common > > > > Expectation: > > Build successfully > > > > Actual Result: > > npm view johnny-five@0.9.59 dist.tarball --registry > > http://registry.npmjs.org failed with exit code 127, output: > > /bin/sh: 1: npm: not found > > > > We don't install npm in refkit-image-common. refkit-image-gateway should > have it. > > > ERROR: johnny-five-0.9.59-r0 do_fetch: Fetcher failure for URL: > > 'npm://registry.npmjs.org;name=johnny-five;version=0.9.59'. Unable to fetch > > URL from any source. > > ERROR: johnny-five-0.9.59-r0 do_fetch: Function failed: base_do_fetch > > ERROR: Logfile of failure stored in: > > /home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit- > > linux/johnny-five/0.9.59-r0/temp/log.do_fetch.26822 > > ERROR: Task > > (/home/minggang/work/intel-iot-refkit/openembedded-core/../meta-iot-web/ > > recipes-devtools/johnny-five/johnny-five_0.9.59.bb:do_fetch) failed with > > exit code '1' > > Is your proxy correctly set? Without the proxy setting wget fails. > > I'm seeing the install failure too. I have set the proxy in ~/.wgetrc in the whole process. 72973 13 patrick.ohly 2017-05-02 08:21:24 +0000 (In reply to comment #11) > The SHA is 4d087e62975ff9ba I was able to reproduce the problem with that revision after deleting my copy of the johnny-five source with "bitbake -c cleanall johnny-five". It does not occur with current master (= de6a4227af102470aa3482750b81e18839152a1e). Please update to that and try again. Remember to do "git submodule update". 72976 14 minggang.wang 2017-05-02 09:41:17 +0000 (In reply to comment #13) > (In reply to comment #11) > > The SHA is 4d087e62975ff9ba > > I was able to reproduce the problem with that revision after deleting my > copy of the johnny-five source with "bitbake -c cleanall johnny-five". > > It does not occur with current master (= > de6a4227af102470aa3482750b81e18839152a1e). Please update to that and try > again. Remember to do "git submodule update". I pulled the latest code, and there is no "npm not found!" error now. Running "bitbake -c devshell johnny-five" and "npm" points to "tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/npm" in the shell. But still have the following error during the building process: | npm ERR! registry URL is required | npm ERR! | npm ERR! If you need help, you may report this error at: | npm ERR! <https://github.com/npm/npm/issues> | npm ERR! Linux 4.4.0-72-generic | npm ERR! argv "/home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/node" "/home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/npm" "install" "--prefix" "/home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/image/usr" "-g" "--arch=x64" "--target_arch=x64" "--production" "--no-registry" | npm ERR! node v4.5.0 | npm ERR! npm v2.15.9 72977 15 patrick.ohly 2017-05-02 10:22:40 +0000 (In reply to comment #14) > But still have the following error during > the building process: > > | npm ERR! registry URL is required > | npm ERR! > | npm ERR! If you need help, you may report this error at: > | npm ERR! <https://github.com/npm/npm/issues> > | npm ERR! Linux 4.4.0-72-generic > | npm ERR! argv > "/home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit- > linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/node" > "/home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit- > linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/npm" "install" > "--prefix" > "/home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit- > linux/johnny-five/0.9.59-r0/image/usr" "-g" "--arch=x64" "--target_arch=x64" > "--production" "--no-registry" > | npm ERR! node v4.5.0 > | npm ERR! npm v2.15.9 That's a separate problem. I've filed https://github.com/ostroproject/meta-iot-web/issues/38 for it.