Bug 13296

Summary: perl ptest results are inconsistent per image
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Joshua Watt <JPEWhacker>
Component: coreAssignee: Matthew Zeng <matthew.zeng>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: JPEWhacker, meta.mr.watcher, meta.watcher, randy.macleod, richard.purdie, TicoTimo, tim.orling
Version: 2.7   
Target Milestone: 3.1 M3   
Hardware: x86   
OS: Multiple   
Whiteboard: NEWCOMER
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
ptest log from core-image-sato build
none
ptest log from core-image-sato-sdk build none

Description Joshua Watt 2019-04-18 18:30:07 UTC
perl is generating different ptest results between the core-image-sato and core-image-sato-sdk images. Specifically:

core-image-sato: https://autobuilder.yocto.io/pub/non-release/20190417-10/testresults/qemux86-64-ptest/testresults.json

Recipe                       | Passed      | Failed   | Skipped   | 
perl                         | 2425        | 20       | 235       | 

core-image-sato-sdk: https://autobuilder.yocto.io/pub/non-release/20190417-14/testresults/qemux86-64-ptest/testresults.json

Recipe                       | Passed      | Failed   | Skipped   | 
perl                         | 2430        | 19       | 231       |
Comment 1 Joshua Watt 2019-04-18 18:30:39 UTC
Created attachment 4495 [details]
ptest log from core-image-sato build
Comment 2 Joshua Watt 2019-04-18 18:30:53 UTC
Created attachment 4496 [details]
ptest log from core-image-sato-sdk build
Comment 3 Randy MacLeod 2019-04-25 14:50:11 UTC
Needs gcc dependency.
Comment 4 Richard Purdie 2019-05-30 14:26:15 UTC
A core-image-minimal image shows:

perl                         | 2423        | 19       | 238       |
Comment 5 Randy MacLeod 2020-02-20 16:48:17 UTC
Matthew is trying to limit the test run to just one. 
Have any of you done that before?
I looked yesterday but my perl-foo is weak and the 'make test' back-end is perly!
Comment 6 Matthew Zeng 2020-02-20 19:49:33 UTC
To run a perl test individually:

```
cd /usr/lib/perl/ptest/t/
./perl harness ../foo/boo.t
```

In my case, lib/warnings is the only inconsistent test, i have to run `./perl harness ../lib/warnings.t`

Hope it helps anyone who runs into this in the future.
Comment 7 Matthew Zeng 2020-02-20 20:38:30 UTC
As opposed from the large number of different ptest results previous reported, there is only one inconsistent ptest that's failing: lib/warnings

```
PROG: 
require '../loc_tools.pl';
unless (locales_enabled()) {
    print("SKIPPED\n# locales not available\n"),exit;
}
eval { require POSIX; POSIX->import("locale_h") };
if ($@) {
    print("SKIPPED\n# no POSIX\n"),exit;
}
my $utf8_locale = find_utf8_ctype_locale();
unless ($utf8_locale) {
    print("SKIPPED\n# No UTF-8 locale available\n"),exit;
}
use warnings 'locale';
use locale;
setlocale(&POSIX::LC_CTYPE, "C");
 "abc def" =~ /\b{wb}.*?/;
 "abc def" =~ /\B{wb}.*?/;
setlocale(&POSIX::LC_CTYPE, $utf8_locale);
 "abc def" =~ /\b{wb}.*?/;
 "abc def" =~ /\B{wb}.*?/;
EXPECTED:
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 16.
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 16.
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 17.
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 17.
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 17.
GOT:
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 16, <DATA> line 47.
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 16, <DATA> line 47.
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 17, <DATA> line 47.
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 17, <DATA> line 47.
Use of \b{} or \B{} for non-UTF-8 locale is wrong.  Assuming a UTF-8 locale at - line 17, <DATA> line 47.
# Failed test 704 - \b{} in UTF-8 locale at lib/warnings/regexec line 189
FAIL: lib/warnings
```

It fails only in core-image-sato, all tests passed in core-image-sato-sdk.

I'm currently working on this.
Comment 8 Matthew Zeng 2020-02-24 16:37:05 UTC
Patch submitted. https://patchwork.openembedded.org/patch/170483/

Testing locally after applying this patch, all perl ptests passed for core-image-minimal, core-image-sato and core-image-sato-sdk.
Comment 9 Matthew Zeng 2020-02-24 18:12:30 UTC
(In reply to comment #8)
> Patch submitted. https://patchwork.openembedded.org/patch/170483/
> 
> Testing locally after applying this patch, all perl ptests passed for
> core-image-minimal, core-image-sato and core-image-sato-sdk.

v2 to fix incorrect format https://patchwork.openembedded.org/patch/170485/
Comment 11 Richard Purdie 2020-03-02 22:06:07 UTC
Thanks for fixing this, much appreciated!