9692 2016-06-01 13:10:32 +0000 QA check for #! interpreter should handle '/usr/bin/env something' 2025-02-06 16:20:14 +0000 1 1 7 Build System, Metadata & Runtime OE-Core core 2.1 x86 Multiple NEW Medium normal Future 1 alex.kanavin unassigned jeanbouch418 joshuagloe louis.rannou meta.mr.watcher meta.watcher randy.macleod rasmus.villemoes richard.purdie sgw stephano tim.orling --- --- No (bug/feature does not impact docs) oldest_to_newest 62556 0 alex.kanavin 2016-06-01 13:10:32 +0000 Currently there is a QA check for shebang interpreters that warns about missing dependencies, but it doesn't handle the case when the interpreter is called via /usr/bin/env. So #!/usr/bin/python without python in dependencies would be caught, but #!/usr/bin/env python would not. QA could handle /usr/bin/env as a special case and do additional search for the interpreter binary in common binary paths - and print a warning if nothing was found, or the interpreter is not listed in dependencies. 62673 1 stephano 2016-06-06 18:24:04 +0000 This will require a change to rpmdeps as well as some higher level searching to correctly resolve the env entry since we need to search PATH. 63160 2 stephano 2016-06-20 21:48:30 +0000 I spoke with the RPM folks and they are interested in this feature as well. I'll be working with them. See here: http://rpm.org/ticket/136 and http://rpm.org/gitweb?p=rpm.git;a=commitdiff;h=e7f73133cbb35ed0f21938996018654391b0a7d7 88720 3 tim.orling 2020-12-03 16:18:24 +0000 Both links in Comment 2 appear to be dead? 95157 4 tim.orling 2023-03-16 15:28:24 +0000 http://rpm.org/ticket/136 is live again. The comment at the bottom is "fixed" but it seems more like a "won't fix". "04/24/15 08:03:57 changed by FlorianFesti status changed from assigned to closed. resolution set to fixed. RPM uses a simplified version of this that does not try to evaluate to the full patch for quite a while now. Guess we stay with this. Closing." Patch link is now: https://github.com/rpm-software-management/rpm/commit/e7f73133cbb35ed0f21938996018654391b0a7d7 Related: https://github.com/rpm-software-management/rpm/issues/136 95179 5 richard.purdie 2023-03-22 17:35:17 +0000 It does look like RPM has basic support and I wonder if we could just use that. Even if RPM don't, that doesn't stop us doing this... 95294 6 louis.rannou 2023-04-05 15:39:55 +0000 Hello, I am having a look. Can you say where the shebang is parsed. I have checked in the insane.bbclass but at that stage it seems we already lost what follows /usr/bin/env As an example, here is what we have in opkg: ``` tmp/work/cortexa15t2hf-neon-poky-linux-gnueabi/xz/5.4.1-r0/pkgdata-sysroot/runtime/opkg-utils:FILERDEPENDS:/usr/bin/opkg.py:opkg-utils: /usr/bin/env ``` My guess is that we have to change what remains in the FILERDEPENDS. I have not found where this is done 97173 7 rasmus.villemoes 2023-11-01 15:04:55 +0000 I also recently stumbled on this (because valgrind has failed to grow a python dependency even if some helper scripts have been rewritten from perl to python) and was also led down the rabbit hole of figuring out how the rpmdeps actually works. It seems unlikely we'll get something into rpmdeps upstream since they've had that # TODO: handle "#!/usr/bin/env foo" somehow for so many years. However, I think Yocto could still do something sensible. So just thinking out loud: Suppose we add a oe-core-only patch to rpmdeps that just adds a sed line that makes #!/usr/bin/env foo output "R interpreter/foo" (or "R interpreter:foo" or "R interpreter(foo)", whatever is easiest to handle later on and doesn't clash with, say, version dependency syntax). At first, the qa check is updated to ignore any dependency starting with interpreter/, just as it currently ignores "perl(". Then, for a subset of well-known possible values of foo (say, perl, python, python3, awk, gawk), the qa code is taught to translate interpreter/foo to /usr/bin/foo. All other foo values continue to be ignored. Maybe that interpreter/foo -> /usr/bin/foo should only be done with usrmerge DISTRO_FEATURE in effect - that should still help catch missing dependencies, since anybody building with that in effect would do the qa check. Alternatively, maybe the gawk recipe could be taught to RPROVIDE interpreter/awk, sort-of like the virtual/ namespace, then the qa code wouldn't need to do any translations (but may still need, for a while, to ignore all but a known set of foo values where the recipes have been updated with the RPROVIDE). 97199 8 4984 rasmus.villemoes 2023-11-03 21:57:40 +0000 Created attachment 4984 combined hacky patch Something like the attached seems to work. The QA check is hack-demoted to a warning just to avoid breaking builds left and right but still see what we'd get. As expected this gives quite a few warnings. Fx WARNING: slang-2.3.3-r0 do_package_qa: /usr/share/slsh/scripts/svnsh contained in package slang requires interpreter/slsh, but no providers found in RDEPENDS:slang? Probably the package should just grow a RPROVIDE interpreter/slsh , but I just did python3 and perl in the POC. WARNING: gobject-introspection-1.78.1-r0 do_package_qa: /usr/bin/g-ir-scanner contained in package gobject-introspection requires interpreter/python3, but no providers found in RDEPENDS:gobject-introspection? It has RDEPENDS on two python modules, so will probably pull in python3-core, but should probably have an explicit RDEPENDS (?) WARNING: glib-2.0-1_2.78.0-r0 do_package_qa: /usr/bin/gdbus-codegen contained in package glib-2.0-codegen requires interpreter/python3, but no providers found in RDEPENDS:glib-2.0-codegen? Had RDEPEND on python3, but that's an empty package that doesn't seem to pull in python3-core. Maybe that RDEPEND should just be changed to python3-core? WARNING: glib-2.0-1_2.78.0-r0 do_package_qa: /usr/bin/gtester-report contained in package glib-2.0-utils requires interpreter/python3, but no providers found in RDEPENDS:glib-2.0-utils? No RDEPENDs AFAICT, so needs fixing. WARNING: gdb-13.2-r0 do_package_qa: /usr/bin/gcore contained in package gdb requires interpreter/bash, but no providers found in RDEPENDS:gdb? WARNING: systemd-1_254.4-r0 do_package_qa: /usr/lib/systemd/systemd-update-helper contained in package systemd requires interpreter/bash, but no providers found in RDEPENDS:systemd? These seem legitimate, and should either be fixed by adding the bash dependency, or if undesired, split from the main package to somewhere else that could then have that dependency. But, maybe this isn't what we want at all. 97200 9 rasmus.villemoes 2023-11-03 22:11:19 +0000 I forgot to add: There are some cases where "#!/usr/bin/env python" is used, from the python3 recipe itself, e.g. WARNING: python3-3.11.5-r0 do_package_qa: /usr/lib/python3.11/cgi.py contained in package python3-netserver requires interpreter/python, but no providers found in RDEPENDS:python3-netserver? But AFAICT we're not shipping something like Debian's python-is-python3 providing a python->python3 symlink, so I don't know how that case should be handled. We can't very well pretend that python3-core provides interpreter/python when the package doesn't have a /usr/bin/python . 98808 10 randy.macleod 2024-04-25 01:09:32 +0000 There's a patch attached, someone could send it to the oe-core list for review. 98954 11 jeanbouch418 2024-05-02 01:41:46 +0000 The path attached is only touching the RPM package. Do we need to also consider DEB package ? 98965 12 jeanbouch418 2024-05-03 01:45:44 +0000 (In reply to jeanbouch418 from comment #11) > The path attached is only touching the RPM package. Do we need to also > consider DEB package ? Never mind, I found the answer. The package.py recipe call rpm utility regardless. 100868 13 randy.macleod 2025-02-06 16:20:14 +0000 The ideas in the patch are sound but the implementation needs some work and guidance. Dropping NEWCOMER. 4984 2023-11-03 21:57:40 +0000 2023-11-03 21:57:40 +0000 combined hacky patch interp.patch text/plain 3710 rasmus.villemoes ZGlmZiAtLWdpdCBhL21ldGEvY2xhc3Nlcy1nbG9iYWwvaW5zYW5lLmJiY2xhc3MgYi9tZXRhL2Ns YXNzZXMtZ2xvYmFsL2luc2FuZS5iYmNsYXNzCmluZGV4IDZmM2NkMzAuLjA4MDI1ZmQgMTAwNjQ0 Ci0tLSBhL21ldGEvY2xhc3Nlcy1nbG9iYWwvaW5zYW5lLmJiY2xhc3MKKysrIGIvbWV0YS9jbGFz c2VzLWdsb2JhbC9pbnNhbmUuYmJjbGFzcwpAQCAtOTcyLDYgKzk3Miw5IEBAIGRlZiBwYWNrYWdl X3FhX2NoZWNrX3JkZXBlbmRzKHBrZywgcGtnZGVzdCwgc2tpcCwgdGFza2RlcHMsIHBhY2thZ2Vz LCBkKToKICAgICAgICAgICAgICAgICBmb3Iga2V5IGluIGZpbGVyZGVwZW5kczoKICAgICAgICAg ICAgICAgICAgICAgZXJyb3JfbXNnID0gIiVzIGNvbnRhaW5lZCBpbiBwYWNrYWdlICVzIHJlcXVp cmVzICVzLCBidXQgbm8gcHJvdmlkZXJzIGZvdW5kIGluIFJERVBFTkRTOiVzPyIgJSBcCiAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgKGZpbGVyZGVwZW5kc1trZXldLnJlcGxhY2UoIjolcyIg JSBwa2csICIiKS5yZXBsYWNlKCJAdW5kZXJzY29yZUAiLCAiXyIpLCBwa2csIGtleSwgcGtnKQor ICAgICAgICAgICAgICAgICAgICBpZiBrZXkuc3RhcnRzd2l0aCgnaW50ZXJwcmV0ZXIvJyk6Cisg ICAgICAgICAgICAgICAgICAgICAgICBiYi53YXJuKGVycm9yX21zZykKKyAgICAgICAgICAgICAg ICAgICAgICAgIGNvbnRpbnVlCiAgICAgICAgICAgICAgICAgICAgIG9lLnFhLmhhbmRsZV9lcnJv cigiZmlsZS1yZGVwcyIsIGVycm9yX21zZywgZCkKIHBhY2thZ2VfcWFfY2hlY2tfcmRlcGVuZHNb dmFyZGVwc2V4Y2x1ZGVdID0gIk9WRVJSSURFUyIKIApkaWZmIC0tZ2l0IGEvbWV0YS9yZWNpcGVz LWRldnRvb2xzL3BlcmwvcGVybF81LjM4LjAuYmIgYi9tZXRhL3JlY2lwZXMtZGV2dG9vbHMvcGVy bC9wZXJsXzUuMzguMC5iYgppbmRleCA5NTZlNGQ2Li43N2U5MjA4IDEwMDY0NAotLS0gYS9tZXRh L3JlY2lwZXMtZGV2dG9vbHMvcGVybC9wZXJsXzUuMzguMC5iYgorKysgYi9tZXRhL3JlY2lwZXMt ZGV2dG9vbHMvcGVybC9wZXJsXzUuMzguMC5iYgpAQCAtMjcyLDcgKzI3Miw3IEBAIEZJTEVTOiR7 UE59ID0gIiR7YmluZGlyfS9wZXJsICR7YmluZGlyfS9wZXJsLnJlYWwgJHtiaW5kaXJ9L3Blcmwk e1BWfSAke2xpYmRpcn0vCiAgICAgICAgICAgICAgICAke2xpYmRpcn0vcGVybDUvJHtQVn0vRXh0 VXRpbHMvdHlwZW1hcCBcCiAgICAgICAgICAgICAgICAiCiBSUFJPVklERVM6JHtQTn0gKz0gInBl cmwtbW9kdWxlLXN0cmljdCBwZXJsLW1vZHVsZS12YXJzIHBlcmwtbW9kdWxlLWNvbmZpZyBwZXJs LW1vZHVsZS13YXJuaW5ncyBcCi0gICAgICAgICAgICAgICAgICAgIHBlcmwtbW9kdWxlLXdhcm5p bmdzLXJlZ2lzdGVyIgorICAgICAgICAgICAgICAgICAgICBwZXJsLW1vZHVsZS13YXJuaW5ncy1y ZWdpc3RlciBpbnRlcnByZXRlci9wZXJsIgogCiBGSUxFUzoke1BOfS1zdGF0aWNkZXY6YXBwZW5k ID0gIiAke2xpYmRpcn0vcGVybDUvJHtQVn0vKi9DT1JFL2xpYnBlcmwuYSIKIApkaWZmIC0tZ2l0 IGEvbWV0YS9yZWNpcGVzLWRldnRvb2xzL3B5dGhvbi9weXRob24zXzMuMTEuNS5iYiBiL21ldGEv cmVjaXBlcy1kZXZ0b29scy9weXRob24vcHl0aG9uM18zLjExLjUuYmIKaW5kZXggOGUwMjNjNy4u YzA0NDhkNiAxMDA2NDQKLS0tIGEvbWV0YS9yZWNpcGVzLWRldnRvb2xzL3B5dGhvbi9weXRob24z XzMuMTEuNS5iYgorKysgYi9tZXRhL3JlY2lwZXMtZGV2dG9vbHMvcHl0aG9uL3B5dGhvbjNfMy4x MS41LmJiCkBAIC0zOTYsNiArMzk2LDggQEAgUlJFQ09NTUVORFM6JHtQTn0tY29yZTphcHBlbmQ6 Y2xhc3MtbmF0aXZlc2RrID0gIiBuYXRpdmVzZGstcHl0aG9uMy1tb2R1bGVzIgogUlJFQ09NTUVO RFM6JHtQTn0tY3J5cHQ6YXBwZW5kOmNsYXNzLXRhcmdldCA9ICIgJHtNTFBSRUZJWH1vcGVuc3Ns ICR7TUxQUkVGSVh9Y2EtY2VydGlmaWNhdGVzIgogUlJFQ09NTUVORFM6JHtQTn0tY3J5cHQ6YXBw ZW5kOmNsYXNzLW5hdGl2ZXNkayA9ICIgJHtNTFBSRUZJWH1vcGVuc3NsICR7TUxQUkVGSVh9Y2Et Y2VydGlmaWNhdGVzIgogCitSUFJPVklERVM6JHtQTn0tY29yZSArPSAiaW50ZXJwcmV0ZXIvcHl0 aG9uMyIKKwogIyBGb3IgaGlzdG9yaWNhbCByZWFzb25zIFBOIGlzIGVtcHR5IGFuZCBwcm92aWRl ZCBieSBweXRob24zLW1vZHVsZXMKIEZJTEVTOiR7UE59ID0gIiIKIFJQUk9WSURFUzoke1BOfS1t b2R1bGVzID0gIiR7UE59IgpkaWZmIC0tZ2l0IGEvbWV0YS9yZWNpcGVzLWRldnRvb2xzL3JwbS9m aWxlcy9oYW5kbGUtdXNyLWJpbi1lbnYtaW50ZXJwcmV0ZXIucGF0Y2ggYi9tZXRhL3JlY2lwZXMt ZGV2dG9vbHMvcnBtL2ZpbGVzL2hhbmRsZS11c3ItYmluLWVudi1pbnRlcnByZXRlci5wYXRjaApu ZXcgZmlsZSBtb2RlIDEwMDY0NAppbmRleCAwMDAwMDAwLi4wNzMyNjFmCi0tLSAvZGV2L251bGwK KysrIGIvbWV0YS9yZWNpcGVzLWRldnRvb2xzL3JwbS9maWxlcy9oYW5kbGUtdXNyLWJpbi1lbnYt aW50ZXJwcmV0ZXIucGF0Y2gKQEAgLTAsMCArMSwxMyBAQAorVXBzdHJlYW0tU3RhdHVzOiBJbmFw cHJvcHJpYXRlIFtvZS1jb3JlIHNwZWNpZmljXQorCitkaWZmIC0tZ2l0IGEvc2NyaXB0cy9zY3Jp cHQucmVxIGIvc2NyaXB0cy9zY3JpcHQucmVxCitpbmRleCA5NTBkYzRhNWMuLjUyM2ZjMTdlMyAx MDA3NTUKKy0tLSBhL3NjcmlwdHMvc2NyaXB0LnJlcQorKysrIGIvc2NyaXB0cy9zY3JpcHQucmVx CitAQCAtNiw0ICs2LDYgQEAgd2hpbGUgcmVhZCBmaWxlbmFtZTsgZG8KKyAgICAgc2VkIC1uIC1l ICcxczpeIyFbWzpzcGFjZTpdXSpcKC9bXls6c3BhY2U6XV1cezEsXH1cKS4qOlwxOnAnICIkZmls ZW5hbWUiCisgICAgICMhL3Vzci9iaW4vZW52IC9mb28vYmFyCisgICAgIHNlZCAtbiAtZSAnMXM6 XiMhW1s6c3BhY2U6XV0qW15bOnNwYWNlOl1dKi9iaW4vZW52W1s6c3BhY2U6XV1cezEsXH1cKC9b Xls6c3BhY2U6XV1cezEsXH1cKTpcMTpwJyAiJGZpbGVuYW1lIgorKyAgICAjIS91c3IvYmluL2Vu diBweXRob24zCisrICAgIHNlZCAtbiAtZSAnMXM6XiMhW1s6c3BhY2U6XV0qW15bOnNwYWNlOl1d Ki9iaW4vZW52W1s6c3BhY2U6XV1cezEsXH1cKFteL1s6c3BhY2U6XV1cezEsXH1cKTppbnRlcnBy ZXRlci9cMTpwJyAiJGZpbGVuYW1lIgorIGRvbmUKZGlmZiAtLWdpdCBhL21ldGEvcmVjaXBlcy1k ZXZ0b29scy9ycG0vcnBtXzQuMTguMS5iYiBiL21ldGEvcmVjaXBlcy1kZXZ0b29scy9ycG0vcnBt XzQuMTguMS5iYgppbmRleCA5NWE5ZTkyLi5kMjk1NTU5IDEwMDY0NAotLS0gYS9tZXRhL3JlY2lw ZXMtZGV2dG9vbHMvcnBtL3JwbV80LjE4LjEuYmIKKysrIGIvbWV0YS9yZWNpcGVzLWRldnRvb2xz L3JwbS9ycG1fNC4xOC4xLmJiCkBAIC00Miw2ICs0Miw4IEBAIFNSQ19VUkkgPSAiZ2l0Oi8vZ2l0 aHViLmNvbS9ycG0tc29mdHdhcmUtbWFuYWdlbWVudC9ycG07YnJhbmNoPXJwbS00LjE4Lng7cHJv dG9jCiAgICAgICAgICAgIGZpbGU6Ly9lYTMxODdjZmNmOWNhYzg3ZTViYzVlN2RiNzliMDMzOGRh OWUzNTVlLnBhdGNoIFwKICAgICAgICAgICAgIgogCitTUkNfVVJJOmFwcGVuZDpjbGFzcy1uYXRp dmUgPSAiIGZpbGU6Ly9oYW5kbGUtdXNyLWJpbi1lbnYtaW50ZXJwcmV0ZXIucGF0Y2giCisKIFBF ID0gIjEiCiBTUkNSRVYgPSAiNDU4OGJjM2Y5OTQzMzg1MDJkMjc3MGFkMjRjYmZjZGFhNmMzMzVl YyIKIAo=