Hello I have several npm packages that are built correctly, like _express 4.18.2_ following the [manual](https://docs.yoctoproject.org/4.0.7/dev-manual/common-tasks.html?highlight=npm#using-the-registry-modules-method) But with package _express-ws 5.0.2_, I get the following issue: ```bash npm verb Linux 5.19.0-32-generic npm verb node v16.19.0 npm verb npm v8.19.3 npm ERR! code ENOTCACHED npm ERR! request to https://registry.npmjs.org/express failed: cache mode is 'only-if-cached' but no cached response is available. ``` Steps to reproduce : 1. Set up a Yocto environment using Kirkstone sha 82c75b466e55d7dca7a2364986ecb704cf63d141 2. Following the [manual](https://docs.yoctoproject.org/4.0.7/dev-manual/common-tasks.html?highlight=npm#using-the-registry-modules-method), create your recipe ```bash devtool add "npm://registry.npmjs.org;package=express-ws;version=5.0.2" ```` 3. Try to build,: ```bash devtool build express-ws ``` 4. Finaly, you probably get the same error as me : ```bash ERROR: express-ws-5.0.2-r0 do_compile: Error executing a python function in exec_func_python() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_func_python() autogenerated', lineno: 2, function: <module> 0001: *** 0002:npm_do_compile(d) 0003: File: '/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/../../layers/poky/meta/classes/npm.bbclass', lineno: 283, function: npm_do_compile 0279: 0280: # Pack and install the main package 0281: (tarball, _) = npm_pack(env, d.getVar("NPM_PACKAGE"), tmpdir) 0282: cmd = "npm install %s %s" % (shlex.quote(tarball), d.getVar("EXTRA_OENPM")) *** 0283: env.run(cmd, args=args) 0284:} 0285: 0286:npm_do_install() { 0287: # Step three: final install File: '/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/layers/poky/bitbake/lib/bb/fetch2/npm.py', lineno: 126, function: run 0122: if args: 0123: for key, value in args: 0124: cmd += " --%s=%s" % (key, shlex.quote(value)) 0125: *** 0126: return _run(cmd) 0127: 0128:class Npm(FetchMethod): 0129: """Class to fetch a package from a npm registry""" 0130: File: '/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/layers/poky/bitbake/lib/bb/fetch2/npm.py', lineno: 114, function: _run 0110: 0111: def _run(cmd): 0112: cmd = "NPM_CONFIG_USERCONFIG=%s " % (self.user_config.name) + cmd 0113: cmd = "NPM_CONFIG_GLOBALCONFIG=%s " % (self.global_config_name) + cmd *** 0114: return runfetchcmd(cmd, d, workdir=workdir) 0115: 0116: if configs: 0117: bb.warn("Use of configs argument of NpmEnvironment.run() function" 0118: " is deprecated. Please use args argument instead.") File: '/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/layers/poky/bitbake/lib/bb/fetch2/__init__.py', lineno: 929, function: runfetchcmd 0925: bb.utils.remove(f, True) 0926: except OSError: 0927: pass 0928: *** 0929: raise FetchError(error_message) 0930: 0931: return output 0932: 0933:def check_network_access(d, info, url): Exception: bb.fetch2.FetchError: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1000/bus"; export SSH_AUTH_SOCK="/run/user/1000/keyring/ssh"; export PATH="/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/usr/bin/python3-native:/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/layers/poky/scripts:/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux:/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot/usr/bin/crossscripts:/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/usr/sbin:/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/usr/bin:/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/sbin:/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/bin:/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/layers/poky/bitbake/bin:/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/hosttools"; export HOME="/tmp/tmppy9hegx1"; NPM_CONFIG_GLOBALCONFIG=/dev/null NPM_CONFIG_USERCONFIG=/tmp/tmpf2bqgbhd npm install /tmp/tmpxdxw65ko/express-ws-5.0.2.tgz --target_arch=aarch64 --build-from-source=true failed with exit code 1, output: npm verb cli /home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/usr/bin/node /home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/usr/bin/npm npm info using npm@8.19.3 npm info using node@v16.19.0 npm timing npm:load:whichnode Completed in 0ms npm timing config:load:defaults Completed in 1ms npm timing config:load:file:/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/usr/lib/node_modules/npm/npmrc Completed in 1ms npm timing config:load:builtin Completed in 1ms npm timing config:load:cli Completed in 1ms npm timing config:load:env Completed in 0ms npm timing config:load:file:/tmp/tmppy9hegx1/.npmrc Completed in 0ms npm timing config:load:project Completed in 0ms npm WARN config only Use `--omit=dev` to omit dev dependencies from the install. npm timing config:load:file:/tmp/tmpf2bqgbhd Completed in 1ms npm timing config:load:user Completed in 1ms npm timing config:load:file:/dev/null Completed in 1ms npm timing config:load:global Completed in 1ms npm timing config:load:validate Completed in 0ms npm timing config:load:credentials Completed in 1ms npm timing config:load:setEnvs Completed in 0ms npm timing config:load Completed in 7ms npm timing npm:load:configload Completed in 7ms npm timing npm:load:mkdirpcache Completed in 1ms npm timing npm:load:mkdirplogs Completed in 0ms npm verb title npm install /tmp/tmpxdxw65ko/express-ws-5.0.2.tgz npm verb argv "install" "/tmp/tmpxdxw65ko/express-ws-5.0.2.tgz" "--target_arch" "aarch64" "--build-from-source" "true" npm timing npm:load:setTitle Completed in 0ms npm timing config:load:flatten Completed in 1ms npm timing npm:load:display Completed in 3ms npm verb logfile logs-max:10 dir:/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/npm-cache/_logs npm verb logfile /home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/npm-cache/_logs/2023-02-21T19_16_43_089Z-debug-0.log npm timing npm:load:logFile Completed in 2ms npm timing npm:load:timers Completed in 0ms npm timing npm:load:configScope Completed in 0ms npm timing npm:load Completed in 14ms npm timing arborist:ctor Completed in 1ms npm sill logfile done cleaning log files npm timing idealTree:init Completed in 3ms npm timing idealTree:userRequests Completed in 26ms npm sill idealTree buildDeps npm sill fetch manifest express-ws@file:../../../../../../../../../../../../../tmp/tmpxdxw65ko/express-ws-5.0.2.tgz npm sill placeDep ROOT express-ws@5.0.2 OK for: want: file:../../../../../../../../../../../../../tmp/tmpxdxw65ko/express-ws-5.0.2.tgz npm sill fetch manifest express@^4.0.0 || ^5.0.0-alpha.1 npm sill fetch manifest ws@7.5.9 npm http fetch GET 200 https://registry.npmjs.org/ws 5ms (cache stale) npm timing idealTree:#root Completed in 18ms npm sill placeDep node_modules/express-ws express@ OK for: express-ws@5.0.2 want: ^4.0.0 || ^5.0.0-alpha.1 npm WARN ERESOLVE overriding peer dependency npm WARN While resolving: express-ws@5.0.2 npm WARN Found: express@undefined npm WARN node_modules/express-ws/node_modules/express npm WARN npm WARN Could not resolve dependency: npm WARN peer express@"^4.0.0 || ^5.0.0-alpha.1" from express-ws@5.0.2 npm WARN node_modules/express-ws npm WARN express-ws@"file:../../../../../../../../../../../../../tmp/tmpxdxw65ko/express-ws-5.0.2.tgz" from the root project npm sill placeDep node_modules/express-ws ws@7.5.9 OK for: express-ws@5.0.2 want: 7.5.9 npm timing idealTree:node_modules/express-ws Completed in 6ms npm timing idealTree:node_modules/express-ws/node_modules/express Completed in 0ms npm timing idealTree:node_modules/express-ws/node_modules/ws Completed in 0ms npm timing idealTree:buildDeps Completed in 24ms npm timing idealTree:fixDepFlags Completed in 1ms npm timing idealTree Completed in 55ms npm timing command:install Completed in 63ms npm verb stack Error: request to https://registry.npmjs.org/express failed: cache mode is 'only-if-cached' but no cached response is available. npm verb stack at cacheFetch (/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/usr/lib/node_modules/npm/node_modules/make-fetch-happen/lib/cache/index.js:12:13) npm verb stack at async fetch (/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/usr/lib/node_modules/npm/node_modules/make-fetch-happen/lib/fetch.js:98:7) npm verb stack at async RegistryFetcher.packument (/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/usr/lib/node_modules/npm/node_modules/pacote/lib/registry.js:92:19) npm verb stack at async RegistryFetcher.manifest (/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/usr/lib/node_modules/npm/node_modules/pacote/lib/registry.js:124:23) npm verb stack at async Promise.all (index 0) npm verb stack at async Arborist.[buildDepStep] (/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/usr/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:1046:5) npm verb stack at async Arborist.buildIdealTree (/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/usr/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:211:7) npm verb stack at async Arborist.reify (/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/usr/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/reify.js:154:5) npm verb stack at async Install.exec (/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/usr/lib/node_modules/npm/lib/commands/install.js:145:5) npm verb stack at async module.exports (/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/recipe-sysroot-native/usr/lib/node_modules/npm/lib/cli.js:78:5) npm verb cwd /tmp/tmppy9hegx1 npm verb Linux 5.19.0-32-generic npm verb node v16.19.0 npm verb npm v8.19.3 npm ERR! code ENOTCACHED npm ERR! request to https://registry.npmjs.org/express failed: cache mode is 'only-if-cached' but no cached response is available. npm verb exit 1 npm timing npm Completed in 124ms npm verb unfinished npm timer reify 1677007003150 npm verb unfinished npm timer reify:loadTrees 1677007003157 npm verb code 1 npm ERR! A complete log of this run can be found in: npm ERR! /home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/npm-cache/_logs/2023-02-21T19_16_43_089Z-debug-0.log ERROR: Logfile of failure stored in: /home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/tmp/work/cortexa72-poky-linux/express-ws/5.0.2-r0/temp/log.do_compile.482404 ERROR: Task (/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-rc-debug/workspace/recipes/express-ws/express-ws_5.0.2.bb:do_compile) failed with exit code '1' ``` Is there any fix or workaround to solve this issue ? Best regards
Tim will review.
This looks to be a problem with upstream metadata being out of date (unable to resolve the 'express' dependency). (Unless it is a different problem with the caching and/or network access restrictions during a task). The project has been questioned as to whether it is 'dead': https://github.com/HenningM/express-ws/issues/135 A comment above mentioned a more recently maintained fork: https://github.com/HenningM/express-ws/issues/135#issuecomment-756978950 https://github.com/aral/express-ws Some additional alternatives are also mentioned. Pierre, can you: (1) Try to build 'express-ws' outside of Bitbake environment (make sure npm/yarn is actually able to resolve the dependencies) and report success/failure? (2) Try one of the alternatives and see if it works for you?
As a workaround, it appears you can run 'npm install express-ws' (after installing the appropriate 'nodejs' version e.g. with 'nvm' on your host) and then use the resulting 'package-lock.json' (replace the 'npm-shrinkwrap.json' in the recipe). This might be uncovering an issue with 'nodejs-oe-cache-native', but I am no expert on the caching.
Thank you or the workaround, (1) outside the bitbake environement it works perectly. And as workaround I simply do on the target a 'npm install' and it is ok . Now I ve simplified my nodejs application which use now 'ws' instead of 'express-ws' and I have no problem installing it using Bitbake unlike 'express-ws'. Anyway, the issue remains in bitbake but it doesn't affect me now
Archana, you could work on this if you like.
Hey everyone, I solved this issue normally in master branch, can you please check and tell me if it works ? Thanks
Hi Thanks for efforts, unfortunately Im on 'kirkstone' branch and I do not have the opportunity to move to 'master' for now
Belouarga, Do you have a link to the commit(s) that fix this issue? If it works on master, Archana may backport it to mickledore/kirkstone branches. Thanks, ../Randy
Hey Randy, These are the commits, I added some commits on bitbake and some others on oe-core. You can just look at commits with my name, I merely corrected this problem on yocto project. Thanks
Thanks Mohamed. For the record for those who like to have such things in bugzilla, here are your commits in: oe-core: ❯ git log --oneline --author=BELOUARGA 6459959bee linux-firmware : Add firmware of RTL8822 serie f8365c0216 classe-recipes: npm: Add support for dependencies and devDependencies 89e02fa47e classes: npm: Add support for the new format of the shrinkwrap file f36021a749 recipetool: create: npm: Add support to handle peer dependencies 488d17c2af recipetool: create: npm: Add support for the new format of the shrinkwrap file a573414864 classes: npm: Handle peer dependencies for npm packages 18e5438de5 recipetool: create: npm: Remove duplicate function to not have future conflicts 6110d9e24e meta: lib: oe: npm_registry: Add more safe caracters bitbake: ❯ git log --oneline --author=BELOUARGA 47ab9d21 fetch2: Check if path is 'None' before calculating checksums 48a102e4 bitbake: fetch2/npmsw: Check if there are dependencies before trying to fetch them b2e1be67 fetch2/npm: Remove special caracters that causes recipe tool to fail 224c2bb1 fetch2/npmsw: Don't fetch dev dependencies when they are not demanded 19b9f7f0 fetch2/npmsw: Add support for the new format of the shrinkwrap file Thanks!
Mohammad, Pierre, The defect is resolved on master and maybe scarthgap but not on kirkstone. It sounds like Pierre would like the work backported. Is someone willing to try that, test it and to send patches ? poky.git on kirkstone ❯ git log --oneline -2 186f1ee88d6981e9f4a4a48ff1ee0bd0b719b18b 186f1ee88d classe-recipes: npm: Add support for dependencies and devDependencies 17732f9cb4 classes: npm: Add support for the new format of the shrinkwrap file ❯ git tag --contains 186f1ee88d6981e9f4a4a48ff1ee0bd0b719b18b 4.3_M2 4.3_M3 5.0_M1 5.0_M2 5.0_M3 nanbield-4.3 nanbield-4.3.1 nanbield-4.3.2 nanbield-4.3.3 nanbield-4.3.4 scarthgap-5.0 scarthgap-5.0.1 yocto-4.3 yocto-4.3.1 yocto-4.3.2 yocto-4.3.3 yocto-4.3.4 yocto-5.0 yocto-5.0.1
Hi I plan to update to scarthgap in 3 months, so if you can wait until it I ll give you my test result at this moment. Regards
Thanks for the reply Pierre. Mohammad, back to you. You can back-port if you think there are interested npm users on kirkstone.
Bulk move of 5.1 bugs to 5.2. -- YP bug review (Randy)
Fixed in master so closing.