Bug 13522 - libevent: inconsistent ptest results compared to autobuilder
Summary: libevent: inconsistent ptest results compared to autobuilder
Status: RESOLVED FIXED
Alias: None
Product: Package Testing (ptest)
Classification: QA/Testing
Component: ptest (show other bugs)
Version: unspecified
Hardware: x86 x86_64
: Medium+ normal
Target Milestone: 3.1 M2
Assignee: Oleg Polyakov
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2019-09-12 17:29 UTC by Trevor Gamblin
Modified: 2020-01-21 20:41 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
libevent qemu log (9.49 KB, text/x-log)
2019-09-12 17:29 UTC, Trevor Gamblin
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Trevor Gamblin 2019-09-12 17:29:55 UTC
Created attachment 4560 [details]
libevent qemu log

While troubleshooting https://bugzilla.yoctoproject.org/show_bug.cgi?id=13465, the failures for libevent's ptests were unable to be replicated on a local machine running QEMU.

Testing on local build systems reported 300 tests passed, 16 skipped (see attached log). Compare attached with: https://autobuilder.yocto.io/pub/non-release/20190911-7/testresults/qemux86-64-ptest/libevent.log

May be something simple, but clarification on autobuilder environment would help nail down the issue.
Comment 1 Richard Purdie 2019-09-19 14:36:00 UTC
Its run as part of a core-image-sato-ptest which has various ptests installed in it which may be the difference.
Comment 2 Oleg Polyakov 2020-01-21 19:21:16 UTC
The failed tests were removed in a later commit:
    Date: Jan 14 2020
    libevent: disable ptests that require a DNS and an Iternet connection

However, these disabled tests run successfully in local QEMU.

Assumption: the tests fail due to the unstable network connection in autobuilder environment.

Conclusion: Since the tests were already removed, nothing should be done here, and the ticket is to be closed.
Comment 3 Oleg Polyakov 2020-01-21 19:23:57 UTC
See previous comment
Comment 4 Randy MacLeod 2020-01-21 20:41:49 UTC
The commit URL that disabled these tests, to save anyone the look-up time:

http://git.openembedded.org/openembedded-core/commit/?id=65ef2810e94bb8feb6be5febfe8a7b73076c3420