| Summary: | many valgrind tests fail for arm64 | ||
|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Randy MacLeod <randy.macleod> |
| Component: | devtools / tool chain | Assignee: | Randy MacLeod <randy.macleod> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | Medium+ | CC: | Aryaman.Gupta, meta.mr.watcher, meta.watcher, Zheng.Qiu |
| Version: | 3.0 | ||
| Target Milestone: | 4.2 M2 | ||
| Hardware: | Other | ||
| OS: | arm64 | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
The current autobuilder results are better but the issue for arm64 still exists and many tests are not being run: https://autobuilder.yocto.io/pub/non-release/20200322-1/testresults/testresult-report.txt -------------------------------------------------------------------------------------------------------------- Recipe | Passed | Failed | Skipped | Time(s) -------------------------------------------------------------------------------------------------------------- qemuarm64 PTest Result Summary valgrind | 380 | 1 | 17 | 805 qemux86-64 PTest Result Summary valgrind | 682 | 0 | 37 | 1304 For fun, I built valgrind from it's git tree (commit: ba73f8d2e) on my Rpi4 running the 64-bit beta of Raspbian. Way more tests fail compared to the yocto ptest so we can look at the compile log to see what the major difference is. -------------------------------------------------------------------- Recipe | Passed | Failed | Skipped -------------------------------------------------------------------- valgrind - yocto | 380 | 1 | 17 valgrind - Raspbian/64 | 618 | 13 | 15 -------------------------------------------------------------------- Ignore the previous comment since I made an error (I used fail rather than pass !!). In this commemnt, I also explain the difference in the number of tests run. For fun, I built valgrind from it's git tree (commit: ba73f8d2e) on my Rpi4 running the 64-bit beta of Raspbian. Way more tests **pass** compared to the yocto ptest so we can look at the compile log to see what the major difference is. -------------------------------------------------------------------- Recipe | Passed | Failed | Skipped -------------------------------------------------------------------- valgrind - yocto | 380 | 1 | 17 valgrind - Raspbian/64 | 618 | 13 | 15 -------------------------------------------------------------------- For the Yocto ptests for arm64, 100s of tests are hidden at ptest execution time since they cause qemuarm64 to use all memory and iirc, for the tests to hang indefinitely. See the valgrind run-ptest script for the line: if [ "$arch" = "aarch64" ]; then If we can figure out why these tests are passing for Rasbian but not for the Yocto ptest runs, we can add them back. I guess the first step is to add the tests back in which you can do by editing the run-ptests script on target before executing the tests.. Stacy, I'm going to work with Kiran on this. Let me know if you would like to be involved. This is no longer happening in dunfell and later. Bug 14960 will track the remaining issues. A patch is already sent to the email list. |
On qemuarm64 systems, about half (250+) of the valgrind tests fail and often result in defunct processes, eg: root 819 818 25 00:12 pts/0 00:01:52 /usr/bin/valgrind --command-line-only=yes --memcheck:leak-check=no --tool=memcheck -q --track-origins= root 861 420 0 00:13 pts/0 00:00:00 [sh] <defunct> Eventually these processes use so much memory that the out of memory killer runs. I've hidden the tests that fail as a horrid work-around until the root cause can be resolved since this at least allows the ptest run to complete. This work-around is done in the run-ptest script using a sorted list of tests so that they can be easily restored one by one without a rebuild during testing. With core-image-minimal on qemuarm64: Recipe | Passed | Failed | Skipped | Time(s) valgrind | 333 | 49 | 17 | 7637 This problem happens for both qemuarm64 and for an arm64 hardware target. It also happens for the warrior branch so it's not a regression.