Bug 12837 - [2.6 M1 RC1] valgrind ptest failed
Summary: [2.6 M1 RC1] valgrind ptest failed
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 2.6
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 2.6 M4
Assignee: Randy MacLeod
QA Contact:
URL:
Whiteboard:
: 8471 12847 (view as bug list)
Depends on:
Blocks:
 
Reported: 2018-07-02 06:05 UTC by Manish Das
Modified: 2018-10-25 16:25 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: Regression (Found by QA test)
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Manish Das 2018-07-02 06:05:05 UTC
Performed diff and found that valgrind tests that passed in 2.5 M4 rc1 are failing on 2.6 M1 RC1

-FAIL: drd/tests/annotate_hbefore
-FAIL: drd/tests/annotate_ignore_read
+PASS: drd/tests/annotate_hbefore
+PASS: drd/tests/annotate_ignore_read


-FAIL: drd/tests/annotate_order_1
-FAIL: drd/tests/annotate_order_2
-FAIL: drd/tests/annotate_order_3
+PASS: drd/tests/annotate_order_1
+PASS: drd/tests/annotate_order_2
+PASS: drd/tests/annotate_order_3


-FAIL: drd/tests/annotate_rwlock
+PASS: drd/tests/annotate_rwlock


-FAIL: drd/tests/annotate_sem
-FAIL: drd/tests/annotate_smart_pointer
+PASS: drd/tests/annotate_sem
+PASS: drd/tests/annotate_smart_pointer


-FAIL: drd/tests/annotate_spinlock
-FAIL: drd/tests/annotate_static
+PASS: drd/tests/annotate_spinlock
+PASS: drd/tests/annotate_static


-SKIP: drd/tests/boost_thread
-FAIL: drd/tests/bug-235681
+PASS: drd/tests/boost_thread
+PASS: drd/tests/bug-235681



-FAIL: drd/tests/concurrent_close
+PASS: drd/tests/concurrent_close


-FAIL: drd/tests/dlopen
+PASS: drd/tests/dlopen


-FAIL: drd/tests/free_is_write
-FAIL: drd/tests/free_is_write2
+PASS: drd/tests/free_is_write
+PASS: drd/tests/free_is_write2


-FAIL: drd/tests/linuxthreads_det
+PASS: drd/tests/linuxthreads_det



-FAIL: drd/tests/pth_cleanup_handler
+PASS: drd/tests/pth_cleanup_handler



-FAIL: drd/tests/pth_detached2
+PASS: drd/tests/pth_detached2


-FAIL: drd/tests/pth_process_shared_mutex
+PASS: drd/tests/pth_process_shared_mutex


-FAIL: drd/tests/pth_spinlock
+PASS: drd/tests/pth_spinlock



-FAIL: drd/tests/rwlock_test
+PASS: drd/tests/rwlock_test



-FAIL: drd/tests/sem_as_mutex2
+PASS: drd/tests/sem_as_mutex2

-FAIL: drd/tests/sem_open2
+PASS: drd/tests/sem_open2


-FAIL: drd/tests/sem_open_traced
-FAIL: drd/tests/sem_wait
+PASS: drd/tests/sem_open_traced
+PASS: drd/tests/sem_wait


-PASS: drd/tests/std_list
+FAIL: drd/tests/std_list

-FAIL: drd/tests/tc24_nonzero_sem
+PASS: drd/tests/tc24_nonzero_sem


-PASS: none/tests/amd64/avx-1
+FAIL: none/tests/amd64/avx-1
Comment 1 Ross Burton 2018-07-05 23:10:34 UTC
Maxin, can you spend an hour looking into this?
Comment 2 Maxin B. John 2018-07-06 09:28:26 UTC
(In reply to comment #1)
> Maxin, can you spend an hour looking into this?
Yes, will look into it.
Comment 3 Ross Burton 2018-07-12 14:50:31 UTC
*** Bug 12847 has been marked as a duplicate of this bug. ***
Comment 4 Ross Burton 2018-07-12 14:50:53 UTC
*** Bug 8471 has been marked as a duplicate of this bug. ***
Comment 5 Manish Das 2018-10-03 09:39:40 UTC
Performed diff and found that valgrind tests that passed in 2.6 M2 rc1 ( -PASS) are failing on 2.6 M3 RC1 (+ FAIL)


--------------------------------------------------------------------------------------
Package Name:  valgrind
--------------------------------------------------------------------------------------
-PASS: drd/tests/bar_trivial

+FAIL: drd/tests/bar_trivial

-PASS: drd/tests/circular_buffer

+FAIL: drd/tests/circular_buffer

-PASS: drd/tests/custom_alloc

+FAIL: drd/tests/custom_alloc

-PASS: drd/tests/fp_race2

+FAIL: drd/tests/fp_race2

-PASS: drd/tests/hg01_all_ok

+FAIL: drd/tests/hg01_all_ok

-PASS: drd/tests/hg02_deadlock

+FAIL: drd/tests/hg02_deadlock

-PASS: drd/tests/matinv

+FAIL: drd/tests/matinv

-PASS: drd/tests/monitor_example

+FAIL: drd/tests/monitor_example

-PASS: drd/tests/pth_barrier

+FAIL: drd/tests/pth_barrier

-PASS: drd/tests/pth_barrier2

+FAIL: drd/tests/pth_barrier2

-PASS: drd/tests/pth_barrier3

+FAIL: drd/tests/pth_barrier3

-PASS: drd/tests/pth_barrier_thr_cr

+FAIL: drd/tests/pth_barrier_thr_cr

-PASS: drd/tests/pth_broadcast

+FAIL: drd/tests/pth_broadcast

-PASS: drd/tests/pth_cond_race2

+FAIL: drd/tests/pth_cond_race2

-PASS: drd/tests/pth_cond_race3

+FAIL: drd/tests/pth_cond_race3

-PASS: drd/tests/pth_create_chain

+FAIL: drd/tests/pth_create_chain

-PASS: drd/tests/pth_detached

+FAIL: drd/tests/pth_detached

-PASS: drd/tests/pth_detached_sem

+FAIL: drd/tests/pth_detached_sem

-PASS: drd/tests/pth_once

+FAIL: drd/tests/pth_once

-PASS: drd/tests/std_atomic

+FAIL: drd/tests/std_atomic

-PASS: drd/tests/std_list

+FAIL: drd/tests/std_list

-PASS: drd/tests/std_string

+FAIL: drd/tests/std_string

-PASS: drd/tests/tc02_simple_tls

+FAIL: drd/tests/tc02_simple_tls

-PASS: drd/tests/tc03_re_excl

+FAIL: drd/tests/tc03_re_excl

-PASS: drd/tests/tc05_simple_race

+FAIL: drd/tests/tc05_simple_race

-PASS: drd/tests/tc06_two_races

+FAIL: drd/tests/tc06_two_races

-PASS: drd/tests/tc08_hbl2

+FAIL: drd/tests/tc08_hbl2

-PASS: drd/tests/tc17_sembar

+FAIL: drd/tests/tc17_sembar

-PASS: drd/tests/threaded-fork-vcs

+FAIL: drd/tests/threaded-fork-vcs

-PASS: drd/tests/threaded-fork

+FAIL: drd/tests/threaded-fork-vcs

-PASS: drd/tests/tls_threads

+FAIL: drd/tests/tls_threads

-PASS: drd/tests/unit_vc

+FAIL: drd/tests/unit_vc

-PASS: helgrind/tests/annotate_hbefore

+FAIL: helgrind/tests/annotate_hbefore

-PASS: helgrind/tests/annotate_smart_pointer

+FAIL: helgrind/tests/annotate_smart_pointer

-PASS: helgrind/tests/bar_trivial

+FAIL: helgrind/tests/bar_trivial

-PASS: helgrind/tests/hg01_all_ok

+FAIL: helgrind/tests/hg01_all_ok

-PASS: helgrind/tests/rwlock_test

+FAIL: helgrind/tests/rwlock_test

-PASS: helgrind/tests/shmem_abits

+FAIL: helgrind/tests/shmem_abits

-PASS: helgrind/tests/stackteardown

+FAIL: helgrind/tests/stackteardown

-PASS: helgrind/tests/tc02_simple_tls

+FAIL: helgrind/tests/tc02_simple_tls

-PASS: helgrind/tests/tc03_re_excl

+FAIL: helgrind/tests/tc03_re_excl

-PASS: helgrind/tests/tc04_free_lock

+FAIL: helgrind/tests/tc04_free_lock

-PASS: helgrind/tests/tc08_hbl2

+FAIL: helgrind/tests/tc08_hbl2

-PASS: helgrind/tests/tc17_sembar

+FAIL: helgrind/tests/tc17_sembar

-PASS: helgrind/tests/tc24_nonzero_sem

+FAIL: helgrind/tests/tc24_nonzero_sem

-PASS: helgrind/tests/tls_threads

+FAIL: helgrind/tests/tls_threads

-PASS: massif/tests/culling1

+FAIL: massif/tests/culling1

-PASS: massif/tests/culling2

+FAIL: massif/tests/culling2

-PASS: massif/tests/malloc_usable

+FAIL: massif/tests/malloc_usable

-PASS: massif/tests/one

+FAIL: massif/tests/one

-PASS: massif/tests/zero1

+FAIL: massif/tests/zero1

-PASS: massif/tests/zero2

+FAIL: massif/tests/zero2

-PASS: memcheck/tests/error_counts

+FAIL: memcheck/tests/error_counts

-PASS: memcheck/tests/leak-0

+FAIL: memcheck/tests/leak-0

-PASS: memcheck/tests/leak-cases-summary

+FAIL: memcheck/tests/leak-cases-summary

-PASS: memcheck/tests/linux/proc-auxv

+FAIL: memcheck/tests/linux/proc-auxv

-PASS: memcheck/tests/linux/sigqueue

+FAIL: memcheck/tests/linux/sigqueue

-PASS: memcheck/tests/mallinfo

+FAIL: memcheck/tests/mallinfo

-PASS: memcheck/tests/malloc1_ks_none

+FAIL: memcheck/tests/malloc1_ks_none

-PASS: memcheck/tests/malloc_free_fill

+FAIL: memcheck/tests/malloc_free_fill

-PASS: memcheck/tests/malloc_usable

+FAIL: memcheck/tests/malloc_usable

-PASS: memcheck/tests/memalign2

+FAIL: memcheck/tests/memalign2

-PASS: memcheck/tests/pdb-realloc2

+FAIL: memcheck/tests/pdb-realloc2

-PASS: memcheck/tests/sigkill

+FAIL: memcheck/tests/sigkill

-PASS: memcheck/tests/stpncpy

+FAIL: memcheck/tests/stpncpy

-PASS: memcheck/tests/str_tester

+FAIL: memcheck/tests/str_tester

-PASS: memcheck/tests/supp_unknown

+FAIL: memcheck/tests/supp_unknown

-PASS: memcheck/tests/wcs

+FAIL: memcheck/tests/wcs

-PASS: memcheck/tests/wrapmalloc

+FAIL: memcheck/tests/wrapmalloc

-PASS: memcheck/tests/wrapmallocstatic

+FAIL: memcheck/tests/wrapmallocstatic

---------------------------------------------------------------------------------------
Total number of test cases passing in Previous release but failing in Current release:  70
---------------------------------------------------------------------------------------
Comment 6 Randy MacLeod 2018-10-25 16:25:36 UTC
I've upgraded valgrind to 3.14.0 and re-ran ptest.

Test run:       qa-3.13   vg-3.14   vg-3.13

Tests ran       558       547       538
Passed          215       287       201
Failed          343       260       337
Passrate        38.53     52.5      37.4
Last Passrate   49.1

where:
  qa-3.13 is from the 2.6-M3 QA report
  vg-3.14 is this update and
  vg-3.13 is with this update removed using poky at:
     3b77e7b785 systemtap: Fix issues from 4.0 upgrade

So the ptest results really are better and they are even
better than the last pass rate that QA reported.
Nonetheless, a 52.5% pass rate isn't good enough so
I'll work on that for master-wr with backports to 2.6.1.