Bug 14228

Summary: valgrind ptest failing on qemuarm64
Product: [QA/Testing] Package Testing (ptest) Reporter: Minjae Kim <flowergom>
Component: ptestAssignee: Yi Fan Yu <yf3yu>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: alexandre.belloni, Jagadheesan_Duraisamy, randy.macleod
Version: 3.3   
Target Milestone: 3.3 M3   
Hardware: Other   
OS: arm64   
Whiteboard: AB-INT
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Minjae Kim 2021-02-09 00:44:49 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1503/steps/12/logs/stdio

Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f
    return func(*args, **kwargs)
  File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f
    return func(*args, **kwargs)
  File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f
    return func(*args, **kwargs)
  File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 25, in test_ptestrunner_expectfail
    self.do_ptestrunner()
  File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 112, in do_ptestrunner
    self.fail(failmsg)
AssertionError: Failed ptests:
{'valgrind': ['drd/tests/std_list']}
Cannot run ptests without @expectedFailure as ptests are expected to fail
RESULTS:
RESULTS - ping.PingTest.test_ping: PASSED (0.16s)
RESULTS - ssh.SSHTest.test_ssh: PASSED (0.30s)
RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectsuccess: SKIPPED (0.00s)
RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectfail: EXPECTEDFAIL (12180.10s)
SUMMARY:
Comment 1 Minjae Kim 2021-02-09 01:08:39 UTC
This issue happened again:
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1511/steps/12/logs/stdio

WARNING: core-image-sato-sdk-ptest-1.0-r0 do_testimage: There were failing ptests.
Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f
    return func(*args, **kwargs)
  File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f
    return func(*args, **kwargs)
  File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f
    return func(*args, **kwargs)
  File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 25, in test_ptestrunner_expectfail
    self.do_ptestrunner()
  File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 112, in do_ptestrunner
    self.fail(failmsg)
AssertionError: Failed ptests:
{'valgrind': ['drd/tests/std_list']}
Comment 4 Yi Fan Yu 2021-02-22 20:49:48 UTC
noticing that valgrind ptest runs pretty slow on qemuarm compared the x86-64 kvm . 

on a local test (also a pretty heavily loaded server building chromium), many tests fail due to timeout.


will increase the timeout to see if this specific test fails again
Comment 5 Yi Fan Yu 2021-02-23 15:14:48 UTC
this specific test `drd/tests/std_list` takes around 47s with glibc2.33

timeout on all tests is 30s.

could consider bumping up the timeout to 60s
but from experience this doesn't scale well 
since the autobuilder could have a much heavier load
Comment 6 Yi Fan Yu 2021-02-24 19:33:33 UTC
drd/tests/std_list takes around 45s on my system to run (its pretty consistent)

same thing with glibc2.32

Proposing just bumping timeout to 90s to remove the possible noise
Comment 7 Randy MacLeod 2021-02-24 19:38:01 UTC
Sure a 90 second timeout for valgrind seems reasonable.
Comment 8 Yi Fan Yu 2021-02-24 19:46:46 UTC
Side note:
Tested with increased timeout
and the following tests (not reported by AB) fail on qemuarm64 building a core-image-minimal with ptest only for valgrind

FAIL: memcheck/tests/linux/timerfd-syscall
FAIL: massif/tests/new-cpp
FAIL: massif/tests/overloaded-new
Comment 11 Yi Fan Yu 2021-03-02 21:59:09 UTC
sent a server-side patch
http://git.yoctoproject.org/cgit/cgit.cgi/error-report-web/commit/?id=178da5ccf6154ecf5968261a16bd245971b909cf

might also change the MACHINE=False to MACHINE=Unknown
Comment 12 Yi Fan Yu 2021-03-04 16:30:20 UTC
(In reply to comment #11)
> sent a server-side patch
> http://git.yoctoproject.org/cgit/cgit.cgi/error-report-web/commit/
> ?id=178da5ccf6154ecf5968261a16bd245971b909cf
> 
> might also change the MACHINE=False to MACHINE=Unknown

ignore this... wrong bug




Resolved : timeout is increased 
https://git.openembedded.org/openembedded-core/commit/?id=aac00b1c8042e41cd6bb1aea8e3033a1c6dd2b05