| Summary: | AB-INT: rust.RustCompileTest.test_cargo_build failure | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Alexandre Belloni <alexandre.belloni> | ||||
| Component: | devtools / tool chain | Assignee: | Sundeep KOKKONDA <sundeep.kokkonda> | ||||
| Status: | RESOLVED FIXED | QA Contact: | |||||
| Severity: | normal | ||||||
| Priority: | High | CC: | alexandre.belloni, luca.ceresoli, meta.mr.watcher, meta.watcher, peter, randy.macleod, richard.purdie | ||||
| Version: | unspecified | ||||||
| Target Milestone: | 4.1 M4 | ||||||
| Hardware: | x86 | ||||||
| OS: | Multiple | ||||||
| Whiteboard: | AB-INT | ||||||
| OS type for building Yocto: | --- | Type of Regression: | --- | ||||
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |||||
| Attachments: |
|
||||||
|
Description
Alexandre Belloni
2022-08-16 20:53:56 UTC
The interesting thing here is that it is complaining about /bin/sh not working but in my changes here: https://git.yoctoproject.org/poky/commit/?id=0650f9c770cf675535fdfd04baa53dbc2c7a727b it should be using the shell from the SDK. I checked a breaking build and it does have a shell path within the SDK. The issue is the length of the shebang being over 128 characters (~150 in this case). I think rust then falls back to trying to execute it with /bin/sh if the original interpreter fails. So, to recap: a) We can't use /bin/sh as rust is setting LD_LIBRARY_PATH which confuses the sdk libs with the host binaries with poor results b) We can't use the shell from the SDK as shebang overflows c) I couldn't find where rust executes target-cc-ld to stop it setting LD_LIBRARY_PATH. Even if I could, that might break something? So our options appear to be to either try c) and remove LD_LIBRARY_PATH, or alternatively, write a C wrapper which does what this shell script wrapper is trying to do, exec() $CC with the args and with LD_LIBRARY_PATH unset. I'm leaning to the C exec() wrapper. It would use the SDK's linker so would avoid most linkage problems. Sundeep, now that you've figured out where n32 is, please work on this as the next top priority. https://autobuilder.yoctoproject.org/typhoon/#builders/47/builds/5715/steps/18/logs/stdio qemuarm-oecore ubuntu1804-ty-3 Hello, I could not reproduce this issue on my local machine. All the tests are passed when I executed the 'bitbake core-image-sato -c do_testsdk' command. ... ... RESULTS - python.Python3Test.test_python3: PASSED (0.04s) RESULTS - rust.RustCompileTest.test_cargo_build: PASSED (0.39s) SUMMARY: core-image-sato sdk (poky-glibc-x86_64-core-image-sato-cortexa57-qemuarm64-toolchain-4.1+snapshot.sh:environment-setup-cortexa57-poky-linux) - Ran 12 tests in 196.103s core-image-sato sdk - OK - All required tests passed (successes=12, skipped=0, failures=0, errors=0) ... I used the same build config as in Yocto auto-builder error log. Also, I tried this on both ppc & arm64 machines in 2 different local machines. In the 'testimage-sdk/sysroots/x86_64-pokysdk-linux/usr/bin/target-rust-ccld' file I verified the shebang which is more than 128 chars (171 characters here). #!/ala-lpggp31/skokkonda/yocto-worker/qemuarm64/poky/build-aarch64/tmp/work/qemuarm64-poky-linux/core-image-sato/1.0-r0/testimage-sdk/sysroots/x86_64-pokysdk-linux/bin/sh Is this issue appearing sporadically? Let me know if this happens only in any special cases. The issue will only appear when the SDK is run on hosts with older glibc versions. From #oe IRC:
[03:26] <pbergin> looking in to adding rust in SDK after the refactoring and clean up RP has done in oe-core around packagegroup-rust-cross-canadian.bb. But building my rust app ands up in the error message "error: linker `x86_64-pokysdk-linux-gcc` not found". Anyone that can help with some tip how to add 'x86_64-pokysdk-linux-gcc' to SDK? I can't find it. My MACHINE is qemuarm64 with a standard Poky setup.
[04:02] <RP> pbergin: it should be using target-rust-ccld
[04:02] <RP> i.e. that is the wrong linker
[04:04] <pbergin> RP: the file .../sysroots/x86_64-pokysdk-linux/usr/lib/aarch64-poky-linux/rustlib/x86_64-pokysdk-linux-gnu.json have a row "linker": "x86_64-pokysdk-linux-gcc"
[04:05] <pbergin> RP: is ot wrong?
[04:35] <RP> pbergin: that is the json for the host, not the target
[04:37] <pbergin> RP: yes. got it. so it seems that my rust project is somehow using the host linker for some package. then looking for 'x86_64-pokysdk-linux-gcc' which is not present. can that be a correct analyze?
[04:38] <RP> pbergin: yes
[04:38] <pbergin> RP: Shouldn't the the "linker" statement for the host point to something that exists in the SDK?
[05:19] <RP> pbergin: arguably that json shouldn't really be there at all
[05:20] <RP> pbergin: it is basically saying "this is the config that built the rust compiler in the SDK"
[05:34] <pbergin> ok. but it is causing problem that needs to be solved in some way. I have been able to understand one step further. some crate have a build.rs script file that is first compiled with host tools then used for building for target with target tools. so there must be a sane config for host tools also.
[05:36] <pbergin> RP: here is a way to reproduce: "cargo new hello;cd hello;echo "fn main(){}" > build.rs;cargo build -v"
[05:52] --> rber|res (~rber|res@62-47-47-177.adsl.highway.telekom.at) has joined this channel.
[05:59] <RP> pbergin: we currently don't include a host rust compiler in the SDK so that would be the issue
[06:00] <RP> pbergin: that could be nice as an SDK test in the test suite
[06:01] <RP> pbergin: https://git.yoctoproject.org/poky/tree/meta/lib/oeqa/sdk/cases/rust.py
[06:40] <pbergin> RP: do you think adding rust host compiler to SDK is a big job? I can try to help out but probably need some pointers.
[06:40] --> frieder (~frieder@i4DF677E2.static.tripleplugandplay.com) has joined this channel.
[06:49] <RP> pbergin: actually, now I think about it we have the appropriate rust compiler, we just don't have a compiler itself. You could add nativesdk-gcc to the SDK and that might work
[06:51] <RP> pbergin: or we patch/symlink it to the host gcc
[06:53] <pbergin> RP: how do I run the rust.py SDK testcase in my own setup?
[07:07] <RP> pbergin: bitbake <xxx> -c populate_sdk; bitbake -c testsdk
[07:08] <RP> with an <xxx> in the second too
[07:18] <pbergin> RP: adding nativesdk-gcc to my sdk takes me one step further. the x86_64-pokysdk-linux-gcc is present in sdk but my reproducer ends up in a segmentation fault in 'ld'
[07:22] <RP> pbergin: ah. That is a known problem :(
[07:22] <RP> rust sets LD_LIBRARY_PATH and this breaks sdk binaries
[07:25] <RP> well, it breaks host libraries due to interference from the sdk ones
[07:36] <pbergin> RP: known problem hard to fix or known problem soon to be solved?
[07:37] <RP> pbergin: so far it was a known issue running the linker for the target and we have a plan for that
[07:37] <RP> pbergin: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14892
[07:40] <pbergin> RP: thanks!
[09:36] <pbergin> RP: by adding "nativesdk-binutils nativesdk-gcc nativesdk-glibc-dev nativesdk-libgcc-dev" to TOOLCHAIN_HOST_TASK my simple rust example with host compilation of build.rs works. is that a suitable way forward to ass those to packagegroup-rust-cross-canadian? or do you want to wait for another fix in the issue you linked?
[09:36] <pbergin> *add
[09:44] <RP> pbergin: I guess that is reasonable but those are heavy dependencies to add :/
[09:45] <RP> pbergin: the issue I linked to is still relevant and will also still need fixing
[09:47] <pbergin> RP: do you want me to send a patch or keep it as a local workaround? I will still send a patch with addition to test case.
[09:58] <RP> pbergin: please send so we can discuss, I'd like to see what others think. You won't be the only one to run into this
[09:58] <RP> vmeson: ^^^
Reference to a patch that adds more comoponents to the SDK. https://patchwork.yoctoproject.org/project/oe-core/patch/20220823085636.30868-1-peter@berginkonsult.se/ The thing when dealing with this in the SDK was that the host compiler took some rally low level files like crtbeginS.o and friend from my host computer and then things bailed out. Fir that reason I had to fill the SDK with all files needed to build my build.rs script. During debugging I used strace. If this error also is version specific to what you have on your host it could probably be similar in the sense that it is taking some files from the host during builds. https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/5696/steps/16/logs/stdio qemuppc ubuntu1804-ty-3 https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/5749/steps/16/logs/stdio qemuarm ubuntu1804-ty-3 I tried to reproduce the issue with glibc v2.17 in CentOS-7, still I could see the test is PASSED. Can someone let me know in which version of glibc this issue is reproducible? https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/5710/steps/15/logs/stdio qemuppc on ubuntu1804-ty-3 https://autobuilder.yoctoproject.org/typhoon/#/builders/59/builds/5720/steps/16/logs/stdio qemux86 ubuntu1804-ty-3 FYI: rmacleod@ubuntu1804-ty-3:~$ cat /etc/os-release NAME="Ubuntu" VERSION="18.04.6 LTS (Bionic Beaver)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 18.04.6 LTS" VERSION_ID="18.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=bionic UBUNTU_CODENAME=bionic rmacleod@ubuntu1804-ty-3:~$ dpkg -l | wc -l 1191 rmacleod@ubuntu1804-ty-3:~$ dpkg -l > ubuntu1804-ty-3.yocto.io--dpkg-l.log # I'll attach that package list. WR has exactly this minor version of ubuntu as well. The package list is a bit longer: $ dpkg -l | wc -l 1429 I'm not sure if that's relevant but FYI. Created attachment 4899 [details]
dpkg -l > ubuntu1804-ty-3.yocto.io--dpkg-l.log
https://autobuilder.yoctoproject.org/typhoon/#builders/60/builds/5728/steps/16/logs/stdio ubuntu1804-ty-3 1661444006 https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/5743/steps/16/logs/stdio qemumips ubuntu1804-ty-3 https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/5727/steps/16/logs/stdio qemumips64 ubuntu1804-ty-3 https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/5731/steps/16/logs/stdio qemuppc ubuntu1804-ty-3 https://autobuilder.yoctoproject.org/typhoon/#/builders/59/builds/5737/steps/16/logs/stdio qemux86 ubuntu1804-ty-3 https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/5794/steps/15/logs/stdio qemuarm ubuntu1804-ty-3 https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/5748/steps/16/logs/stdio qemuppc ubuntu1804-ty-3 https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/5749/steps/15/logs/stdio qemumips64 ubuntu1804-ty-3 https://autobuilder.yoctoproject.org/typhoon/#/builders/42/builds/5790/steps/16/logs/stdio qemuarm64 ubuntu1804-ty-3 |