Bug 14051

This issues is observed in yocto-3.2_M3.rc2 also.

Three additional failures observed in yocto-3.2_M3.rc2 : 
-ptestresult.valgrind.drd/tests/annotate_order_2
-ptestresult.valgrind.drd/tests/annotate_spinlock
-ptestresult.valgrind.drd/tests/std_mutex

See https://bugzilla.yoctoproject.org/show_bug.cgi?id=14051

Following two tests passed in yocto-3.2.rc1:

-ptestresult.valgrind.drd/tests/bar_bad
-ptestresult.valgrind.drd/tests/bar_bad_xml

However three new failures are observed in valgrind.drd: 

-ptestresult.valgrind.drd/tests/annotate_ignore_read
-ptestresult.valgrind.drd/tests/annotate_order_3
-ptestresult.valgrind.drd/tests/annotate_rwlock

Following tests passed in yocto-3.2.rc2 : 

ptestresult.valgrind.drd/tests/annotate_hb_err
ptestresult.valgrind.drd/tests/annotate_ignore_read
ptestresult.valgrind.drd/tests/annotate_order_2
ptestresult.valgrind.drd/tests/annotate_order_3
ptestresult.valgrind.drd/tests/annotate_rwlock
ptestresult.valgrind.drd/tests/annotate_sem
ptestresult.valgrind.drd/tests/annotate_smart_pointer
ptestresult.valgrind.drd/tests/annotate_spinlock
ptestresult.valgrind.drd/tests/concurrent_close
ptestresult.valgrind.drd/tests/monitor_example
ptestresult.valgrind.drd/tests/pth_cond_race
ptestresult.valgrind.drd/tests/pth_inconsistent_cond_wait
ptestresult.valgrind.drd/tests/pth_spinlock
ptestresult.valgrind.drd/tests/tc23_bogus_condwait
ptestresult.valgrind.drd/tests/thread_name
ptestresult.valgrind.drd/tests/thread_name_xml
ptestresult.valgrind.drd/tests/tls_threads

(In reply to comment #3)
> Following two tests passed in yocto-3.2.rc1:
> 
> -ptestresult.valgrind.drd/tests/bar_bad
> -ptestresult.valgrind.drd/tests/bar_bad_xml

-ptestresult.valgrind.drd/tests/std_mutex also passed in yocto-3.2.rc1

> 
> However three new failures are observed in valgrind.drd: 
> 
> -ptestresult.valgrind.drd/tests/annotate_ignore_read
> -ptestresult.valgrind.drd/tests/annotate_order_3
> -ptestresult.valgrind.drd/tests/annotate_rwlock

The remaining ptest that is still failing in yocto-3.2.rc2 is : 

-ptestresult.valgrind.helgrind/tests/pth_destroy_cond

As a summary for this bug, the only ptest that is failing finally is : 

-ptestresult.valgrind.helgrind/tests/pth_destroy_cond

Patch for helgrind was accepted.

commit id:
https://git.openembedded.org/openembedded-core/commit/?id=5da46a552d54de34a5243e1d90dcc6f52b7af746

Thanks Stacy!

The following ptest is passing in yocto-3.2.1_rc2 : 

-ptestresult.valgrind.helgrind/tests/pth_destroy_cond

However, these following ptest(s) are observed failing in yocto-3.2.1_rc2 :  

-ptestresult.valgrind.drd/tests/bar_bad
-ptestresult.valgrind.drd/tests/bar_bad_xml

Anuj, Stacy has fix this on master but she's wrapping up her work term so can you find the commits and backport them? If that doesn't work then assign this bug to me and I'll have someone work on it in January.

All the commits are already in gatesgarth:

https://git.openembedded.org/openembedded-core/commit/meta/recipes-devtools/valgrind?h=gatesgarth&id=4c33ce1b1eca9aff0009bf71ce50f6398f7cd281

https://git.openembedded.org/openembedded-core/commit/meta/recipes-devtools/valgrind?h=gatesgarth&id=e8f265e957e187b4f84f566aaeec8f514e3044d6

https://git.openembedded.org/openembedded-core/commit/meta/recipes-devtools/valgrind?h=gatesgarth&id=9c82b9302c6ee84864e9a9cc870729395804e794

So the issue could be something else.

The following ptest is passing in yocto-3.3_M1_rc2 : 

-ptestresult.valgrind.helgrind/tests/pth_destroy_cond

But the following ptest(s) are failing in yocto-3.3_M1_rc2 :  

-ptestresult.valgrind.drd/tests/bar_bad
-ptestresult.valgrind.drd/tests/bar_bad_xml

Anuj, Stacy has one week left with WR and has some internal work to do but she'll try to take a look at these valgrind ptest regressions.

maybe this commit temporarily resolves this specific issue? 
oe-core: 7f7d2fa18267090891754d976cbc3e628324d3dd

tests removed:

-ptestresult.valgrind.drd/tests/bar_bad
-ptestresult.valgrind.drd/tests/bar_bad_xml


https://bugs.kde.org/show_bug.cgi?id=430321 is perhaps the more recent bug report concerning `bar_bad` and `bar_bad_xml`

Need info from Alexander Kanavin

Alexander Kanavin,

any updates regarding these 2 tests?

-ptestresult.valgrind.drd/tests/bar_bad
-ptestresult.valgrind.drd/tests/bar_bad_xml

They were disabled in oe-core master some time ago:

commit c4c6bb89cfd083a294a5f760f9b8542b1d5e6715
Author: Alexander Kanavin <alex.kanavin@gmail.com>
Date:   Mon Dec 14 10:10:39 2020 +0100

    valgrind: exclude bar_bad/bar_bad_xml from ptests
    
    The tests' output is non-deterministic and sometimes
    doesn't match the sample output. This has been reported at
    https://bugs.kde.org/show_bug.cgi?id=430321
    (see also an earlier related bug https://bugs.kde.org/show_bug.cgi?id=358213)
    
    Until upstream figures out how to fix this, let's not run the tests.
    
    (From OE-Core rev: 7f7d2fa18267090891754d976cbc3e628324d3dd)
    
    Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

not sure if related (i was investigating BUG 14223 of another ptest failing)
I saw this when running ptest

```
2021-02-11T20:36
BEGIN: /usr/lib/valgrind/ptest
Hide valgrind tests that are non-deterministic
Reported at https://bugs.kde.org/show_bug.cgi?id=430321
mv: cannot stat 'drd/tests/bar_bad.vgtest': No such file or directory
mv: cannot stat 'drd/tests/bar_bad_xml.vgtest': No such file or directory
```

(In reply to comment #18)
> not sure if related (i was investigating BUG 14223 of another ptest failing)
> I saw this when running ptest
> 
> ```
> 2021-02-11T20:36
> BEGIN: /usr/lib/valgrind/ptest
> Hide valgrind tests that are non-deterministic
> Reported at https://bugs.kde.org/show_bug.cgi?id=430321
> mv: cannot stat 'drd/tests/bar_bad.vgtest': No such file or directory
> mv: cannot stat 'drd/tests/bar_bad_xml.vgtest': No such file or directory
> ```

nvm its because i ran the test twice.

I think we can close this issue, the solution was to disable it until they fix it in valgrind

(In reply to comment #19)
> (In reply to comment #18)
> > not sure if related (i was investigating BUG 14223 of another ptest failing)
> > I saw this when running ptest
> > 
> > ```
> > 2021-02-11T20:36
> > BEGIN: /usr/lib/valgrind/ptest
> > Hide valgrind tests that are non-deterministic
> > Reported at https://bugs.kde.org/show_bug.cgi?id=430321
> > mv: cannot stat 'drd/tests/bar_bad.vgtest': No such file or directory
> > mv: cannot stat 'drd/tests/bar_bad_xml.vgtest': No such file or directory
> > ```
> 
> nvm its because i ran the test twice.
> 
> I think we can close this issue, the solution was to disable it until they
> fix it in valgrind

Should we close this bug now? If you are working on it, could you assign this to yourself please?

I'll see what I can find.

Was not able to reproduce on qemux86-64 kvm running 10 times those 2 specific ptest.

It might be fixed by 3.17 uprev...

If I can reproduce with 3.16 (where this workaround/disabling was introduced), then I will send a patch to revert this test.

Patch in the mailing list to reenable these 2 tests. Let's see if they still fail on AB

https://git.openembedded.org/openembedded-core/commit/?h=master-next&id=1e8f2a1dffd03781d74d770867ae505023c29857

its reenabled on master-next,
if this reoccurs, we can reopen a new ticket, and likely mark it AB

passed on last night AB on qemux86-64

https://autobuilder.yocto.io/pub/non-release/20210421-20/testresults/qemux86-64-ptest/


closing this ticket as fixed. since Stacy fixed the helgrind issue.

see comment above