Bug 13296 - perl ptest results are inconsistent per image
Summary: perl ptest results are inconsistent per image
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 2.7
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 3.1 M3
Assignee: Matthew Zeng
QA Contact:
URL:
Whiteboard: NEWCOMER
Depends on:
Blocks:
 
Reported: 2019-04-18 18:30 UTC by Joshua Watt
Modified: 2020-03-02 22:06 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
ptest log from core-image-sato build (95.27 KB, text/x-log)
2019-04-18 18:30 UTC, Joshua Watt
no flags Details
ptest log from core-image-sato-sdk build (91.57 KB, text/x-log)
2019-04-18 18:30 UTC, Joshua Watt
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Joshua Watt 2019-04-18 18:30:07 UTC
perl is generating different ptest results between the core-image-sato and core-image-sato-sdk images. Specifically:

core-image-sato: https://autobuilder.yocto.io/pub/non-release/20190417-10/testresults/qemux86-64-ptest/testresults.json

Recipe                       | Passed      | Failed   | Skipped   | 
perl                         | 2425        | 20       | 235       | 

core-image-sato-sdk: https://autobuilder.yocto.io/pub/non-release/20190417-14/testresults/qemux86-64-ptest/testresults.json

Recipe                       | Passed      | Failed   | Skipped   | 
perl                         | 2430        | 19       | 231       |
Comment 1 Joshua Watt 2019-04-18 18:30:39 UTC
Created attachment 4495 [details]
ptest log from core-image-sato build
Comment 2 Joshua Watt 2019-04-18 18:30:53 UTC
Created attachment 4496 [details]
ptest log from core-image-sato-sdk build
Comment 3 Randy MacLeod 2019-04-25 14:50:11 UTC
Needs gcc dependency.
Comment 4 Richard Purdie 2019-05-30 14:26:15 UTC
A core-image-minimal image shows:

perl                         | 2423        | 19       | 238       |
Comment 5 Randy MacLeod 2020-02-20 16:48:17 UTC
Matthew is trying to limit the test run to just one. 
Have any of you done that before?
I looked yesterday but my perl-foo is weak and the 'make test' back-end is perly!
Comment 6 Matthew Zeng 2020-02-20 19:49:33 UTC
To run a perl test individually:

```
cd /usr/lib/perl/ptest/t/
./perl harness ../foo/boo.t
```

In my case, lib/warnings is the only inconsistent test, i have to run `./perl harness ../lib/warnings.t`

Hope it helps anyone who runs into this in the future.
Comment 7 Matthew Zeng 2020-02-20 20:38:30 UTC
As opposed from the large number of different ptest results previous reported, there is only one inconsistent ptest that's failing: lib/warnings

```
PROG: 
require '../loc_tools.pl';
unless (locales_enabled()) {
    print("SKIPPED\n# locales not available\n"),exit;
}
eval { require POSIX; POSIX->import("locale_h") };
if ($@) {
    print("SKIPPED\n# no POSIX\n"),exit;
}
my $utf8_locale = find_utf8_ctype_locale();
unless ($utf8_locale) {
    print("SKIPPED\n# No UTF-8 locale available\n"),exit;
}
use warnings 'locale';
use locale;
setlocale(&POSIX::LC_CTYPE, "C");
 "abc def" =~ /\b{wb}.*?/;
 "abc def" =~ /\B{wb}.*?/;
setlocale(&POSIX::LC_CTYPE, $utf8_locale);
 "abc def" =~ /\b{wb}.*?/;
 "abc def" =~ /\B{wb}.*?/;
EXPECTED:
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 16.
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 16.
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 17.
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 17.
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 17.
GOT:
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 16, <DATA> line 47.
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 16, <DATA> line 47.
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 17, <DATA> line 47.
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 17, <DATA> line 47.
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 17, <DATA> line 47.
# Failed test 704 - \b{} in UTF-8 locale at lib/warnings/regexec line 189
FAIL: lib/warnings
```

It fails only in core-image-sato, all tests passed in core-image-sato-sdk.

I'm currently working on this.
Comment 8 Matthew Zeng 2020-02-24 16:37:05 UTC
Patch submitted. https://patchwork.openembedded.org/patch/170483/

Testing locally after applying this patch, all perl ptests passed for core-image-minimal, core-image-sato and core-image-sato-sdk.
Comment 9 Matthew Zeng 2020-02-24 18:12:30 UTC
(In reply to comment #8)
> Patch submitted. https://patchwork.openembedded.org/patch/170483/
> 
> Testing locally after applying this patch, all perl ptests passed for
> core-image-minimal, core-image-sato and core-image-sato-sdk.

v2 to fix incorrect format https://patchwork.openembedded.org/patch/170485/
Comment 11 Richard Purdie 2020-03-02 22:06:07 UTC
Thanks for fixing this, much appreciated!