Which revision of intel-iot-refkit is this with?

I tried with dbc675f0f97052ca2a9563d4a4d4b1467a88681e (almost current master) and for me at least do_fetch worked.

"bitbake johnny-five" still fails, though:

$ bitbake johnny-five
...
meta-selftest     = "HEAD:ba2884f6ad3a4e746fc80cbd707f83fa8abd4210"
meta-intel        = "HEAD:60c270394755c36d3cf2d1df61b3795521e1caa0"
meta-refkit       = "HEAD:dbc675f0f97052ca2a9563d4a4d4b1467a88681e"
meta-oic          = "HEAD:b9ba736871b8f8b741cc9c1d88dda56e63a35091"
meta-iot-web      = "HEAD:e4310f8a34a985fa62eb66c7742088dc7943bedb"
meta-iotqa        = "HEAD:dbc675f0f97052ca2a9563d4a4d4b1467a88681e"
meta-security     = "HEAD:6674749274e47937af640ffaa2480e26a362e5f6"
meta-security-isafw = "HEAD:834272c2471a86aca826170036598889b5dd552a"
meta-intel-realsense = "HEAD:1d50cb5b82926f8e69a78999e9e8d2ffe2169897"
meta-clang        = "HEAD:0cb6aaa39d22c461c076b145cecf4d7e8dae2c7f"
meta-filesystems  
meta-gnome        
meta-networking   
meta-oe           
meta-perl         
meta-python       = "HEAD:cdb428e7c49899675ee7b7a43f6cecdb5b4c2546"
meta-integrity    
meta-security-framework 
meta-security-smack = "root-ca-cert:baac71ebfe20498edfeb057721161c102508fcf2"
...
ERROR: johnny-five-0.9.59-r0 do_install: Function failed: do_install (log file is located at /fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/temp/log.do_install.8675)
ERROR: Logfile of failure stored in: /fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/temp/log.do_install.8675
Log data follows:
...
| DEBUG: Executing shell function do_install
| npm ERR! Linux 4.9.0-0.bpo.2-amd64
| npm ERR! argv "/fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/node" "/fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/npm" "install" "--prefix" "/fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/image/usr" "-g" "--arch=x64" "--target_arch=x64" "--production" "--no-registry"
| npm ERR! node v4.5.0
| npm ERR! npm  v2.15.9
| 
| npm ERR! registry URL is required
| npm ERR!
| npm ERR! If you need help, you may report this error at:
| npm ERR!     <https://github.com/npm/npm/issues>
| npm ERR! Linux 4.9.0-0.bpo.2-amd64
| npm ERR! argv "/fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/node" "/fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/npm" "install" "--prefix" "/fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/image/usr" "-g" "--arch=x64" "--target_arch=x64" "--production" "--no-registry"
...

Paul, any idea what that is about?

Mikko, do we currently test npm examples in refkit?

One can check with "bitbake -c devshell johnny-five" whether there is a npm:

$ bitbake -c devshell johnny-five
...
# which npm
/fast/build/refkit/intel-corei7-64/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/npm
# npm --version
2.15.9

Minggang, is there really no npm in your corresponding johnny-five/0.9.59-r0/recipe-sysroot-native?

Paul, a

Hi Patrick, I checked the path, there is a npm under tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin.

Actually, I tried to workaround this problem by adding in local.conf:
  HOSTTOOLS_append = " npm"

Then, it reports the same error exactly as you described above.

(In reply to comment #2)
> Hi Patrick, I checked the path, there is a npm under
> tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-
> native/usr/bin.

Than that's the one it should have used.

Which revision of intel-iot-refkit are you using?

> Actually, I tried to workaround this problem by adding in local.conf:
>   HOSTTOOLS_append = " npm"
> 
> Then, it reports the same error exactly as you described above.

As a workaround that's okay, but it's not the right solution for the "npm not found" problem, because we don't want to depend on some random version of npm on the build host.

(In reply to comment #3)
> (In reply to comment #2)
> > Hi Patrick, I checked the path, there is a npm under
> > tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-
> > native/usr/bin.
> 
> Than that's the one it should have used.
> 
> Which revision of intel-iot-refkit are you using?
> 
BitBake Build Tool Core version 1.33.2

> > Actually, I tried to workaround this problem by adding in local.conf:
> >   HOSTTOOLS_append = " npm"
> > 
> > Then, it reports the same error exactly as you described above.
> 
> As a workaround that's okay, but it's not the right solution for the "npm
> not found" problem, because we don't want to depend on some random version
> of npm on the build host.

(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> > > Hi Patrick, I checked the path, there is a npm under
> > > tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-
> > > native/usr/bin.
> > 
> > Than that's the one it should have used.
> > 
> > Which revision of intel-iot-refkit are you using?
> > 
> BitBake Build Tool Core version 1.33.2

No, I meant "which git revision of intel-iot-refkit" are you using?

One more test (without your HOSTTOOLS workaround): please run "bitbake -c devshell johnny-five", then in the console prompt of the devshell, type "which npm" and "echo $PATH". Does that find the npm that you have and if not, why?

(In reply to comment #0)
> Environment:
> Ubuntu 16.04, build without docker, follow the instruction
> https://github.com/intel/intel-iot-refkit
> 
> Triggering Action:
> 1. Add the following lines to local.conf
> SUPPORTED_RECIPES_CHECK_append = " warn"
> SUPPORTED_RECIPES_CHECK_remove = "fatal"
> REFKIT_IMAGE_EXTRA_INSTALL_append = " johnny-five"
> 
> 2. run command $bitbake refkit-image-common
> 
> Expectation:
> Build successfully
> 
> Actual Result:
> npm view johnny-five@0.9.59 dist.tarball --registry
> http://registry.npmjs.org failed with exit code 127, output:
> /bin/sh: 1: npm: not found
> 

We don't install npm in refkit-image-common. refkit-image-gateway should have it.

> ERROR: johnny-five-0.9.59-r0 do_fetch: Fetcher failure for URL:
> 'npm://registry.npmjs.org;name=johnny-five;version=0.9.59'. Unable to fetch
> URL from any source.
> ERROR: johnny-five-0.9.59-r0 do_fetch: Function failed: base_do_fetch
> ERROR: Logfile of failure stored in:
> /home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit-
> linux/johnny-five/0.9.59-r0/temp/log.do_fetch.26822
> ERROR: Task
> (/home/minggang/work/intel-iot-refkit/openembedded-core/../meta-iot-web/
> recipes-devtools/johnny-five/johnny-five_0.9.59.bb:do_fetch) failed with
> exit code '1'

Is your proxy correctly set? Without the proxy setting wget fails.

I'm seeing the install failure too.

On #yocto IRC, Paul said about the do_install error:

it's basically because some dependency in the .json file isn't being correctly satisfied, so npm thinks it needs a registry, but we are telling it not to use one and so it blows up

So that's something else, perhaps related to this specific example. I'm no NodeJS expert.

The npm not found issue is caused by a missing dependency from do_fetch on nodejs-native i.e.

do_fetch[depends] += "nodejs-native:do_populate_sysroot"

I will send a patch to add that to npm.bbclass shortly.

Actually, scratch that last comment - that can't be true, base.bbclass should already be adding that dependency. Will investigate.

(In reply to comment #9)
> Actually, scratch that last comment - that can't be true, base.bbclass
> should already be adding that dependency. Will investigate.

My understanding is npm is not found in the final image. See comment #6. That is expected.

(In reply to comment #5)
> (In reply to comment #4)
> > (In reply to comment #3)
> > > (In reply to comment #2)
> > > > Hi Patrick, I checked the path, there is a npm under
> > > > tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-
> > > > native/usr/bin.
> > > 
> > > Than that's the one it should have used.
> > > 
> > > Which revision of intel-iot-refkit are you using?
> > > 
> > BitBake Build Tool Core version 1.33.2
> 
The SHA is 4d087e62975ff9ba

> No, I meant "which git revision of intel-iot-refkit" are you using?
> 
> One more test (without your HOSTTOOLS workaround): please run "bitbake -c
> devshell johnny-five", then in the console prompt of the devshell, type
> "which npm" and "echo $PATH". Does that find the npm that you have and if
> not, why?
When I was running (without adding the HOSTTOOLS) "bitbake -c devshell johnny-five", reports:

/bin/sh: 1: npm: not found

ERROR: johnny-five-0.9.59-r0 do_fetch: Fetcher failure for URL: 'npm://registry.npmjs.org;name=johnny-five;version=0.9.59'. Unable to fetch URL from any source.
ERROR: johnny-five-0.9.59-r0 do_fetch: Function failed: base_do_fetch
ERROR: Logfile of failure stored in: /home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/temp/log.do_fetch.27829
ERROR: Task (/home/minggang/work/intel-iot-refkit/openembedded-core/../meta-iot-web/recipes-devtools/johnny-five/johnny-five_0.9.59.bb:do_fetch) failed with exit code '1'

(In reply to comment #6)
> (In reply to comment #0)
> > Environment:
> > Ubuntu 16.04, build without docker, follow the instruction
> > https://github.com/intel/intel-iot-refkit
> > 
> > Triggering Action:
> > 1. Add the following lines to local.conf
> > SUPPORTED_RECIPES_CHECK_append = " warn"
> > SUPPORTED_RECIPES_CHECK_remove = "fatal"
> > REFKIT_IMAGE_EXTRA_INSTALL_append = " johnny-five"
> > 
> > 2. run command $bitbake refkit-image-common
> > 
> > Expectation:
> > Build successfully
> > 
> > Actual Result:
> > npm view johnny-five@0.9.59 dist.tarball --registry
> > http://registry.npmjs.org failed with exit code 127, output:
> > /bin/sh: 1: npm: not found
> > 
> 
> We don't install npm in refkit-image-common. refkit-image-gateway should
> have it.
> 
> > ERROR: johnny-five-0.9.59-r0 do_fetch: Fetcher failure for URL:
> > 'npm://registry.npmjs.org;name=johnny-five;version=0.9.59'. Unable to fetch
> > URL from any source.
> > ERROR: johnny-five-0.9.59-r0 do_fetch: Function failed: base_do_fetch
> > ERROR: Logfile of failure stored in:
> > /home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit-
> > linux/johnny-five/0.9.59-r0/temp/log.do_fetch.26822
> > ERROR: Task
> > (/home/minggang/work/intel-iot-refkit/openembedded-core/../meta-iot-web/
> > recipes-devtools/johnny-five/johnny-five_0.9.59.bb:do_fetch) failed with
> > exit code '1'
> 
> Is your proxy correctly set? Without the proxy setting wget fails.
> 
> I'm seeing the install failure too.

I have set the proxy in ~/.wgetrc in the whole process.

(In reply to comment #11)
> The SHA is 4d087e62975ff9ba

I was able to reproduce the problem with that revision after deleting my copy of the johnny-five source with "bitbake -c cleanall johnny-five".

It does not occur with current master (= de6a4227af102470aa3482750b81e18839152a1e). Please update to that and try again. Remember to do "git submodule update".

(In reply to comment #13)
> (In reply to comment #11)
> > The SHA is 4d087e62975ff9ba
> 
> I was able to reproduce the problem with that revision after deleting my
> copy of the johnny-five source with "bitbake -c cleanall johnny-five".
> 
> It does not occur with current master (=
> de6a4227af102470aa3482750b81e18839152a1e). Please update to that and try
> again. Remember to do "git submodule update".

I pulled the latest code, and there is no "npm not found!" error now. Running "bitbake -c devshell johnny-five" and "npm" points to "tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/npm" in the shell. But still have the following error during the building process:

| npm ERR! registry URL is required
| npm ERR!
| npm ERR! If you need help, you may report this error at:
| npm ERR!     <https://github.com/npm/npm/issues>
| npm ERR! Linux 4.4.0-72-generic
| npm ERR! argv "/home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/node" "/home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/npm" "install" "--prefix" "/home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit-linux/johnny-five/0.9.59-r0/image/usr" "-g" "--arch=x64" "--target_arch=x64" "--production" "--no-registry"
| npm ERR! node v4.5.0
| npm ERR! npm  v2.15.9

(In reply to comment #14)
> But still have the following error during
> the building process:
> 
> | npm ERR! registry URL is required
> | npm ERR!
> | npm ERR! If you need help, you may report this error at:
> | npm ERR!     <https://github.com/npm/npm/issues>
> | npm ERR! Linux 4.4.0-72-generic
> | npm ERR! argv
> "/home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit-
> linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/node"
> "/home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit-
> linux/johnny-five/0.9.59-r0/recipe-sysroot-native/usr/bin/npm" "install"
> "--prefix"
> "/home/minggang/work/intel-iot-refkit/build/tmp-glibc/work/corei7-64-refkit-
> linux/johnny-five/0.9.59-r0/image/usr" "-g" "--arch=x64" "--target_arch=x64"
> "--production" "--no-registry"
> | npm ERR! node v4.5.0
> | npm ERR! npm  v2.15.9

That's a separate problem. I've filed https://github.com/ostroproject/meta-iot-web/issues/38 for it.