Bug 13242

Summary: [2.7 M3 RC1] strace ptest failure
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Sudhir Sharma <sharmasudhir>
Component: oe-core otherAssignee: Kai Kang <kai.kang>
Status: RESOLVED NOTABUG QA Contact:
Severity: normal    
Priority: Medium+ CC: randy.macleod, ross.burton
Version: 2.7   
Target Milestone: 2.7 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know
Attachments:
Description Flags
Full log for 2.7 m3 rc1 none

Description Sudhir Sharma 2019-03-26 09:46:17 UTC
Created attachment 4453 [details]
Full log for 2.7 m3 rc1

On running the ptest on 2.7 M3 RC1,  it is seen that strace test is getting failed:

              Line 1514:     ptestresult.strace.attach-f-p.test: FAILED -> PASSED
              Line 1515:     ptestresult.strace.attach-p-cmd.test: FAILED -> PASSED
              Line 1516:     ptestresult.strace.bexecve.test: FAILED -> PASSED
              Line 1519:     ptestresult.strace.bpf-success-v.test: FAILED -> PASSED
              Line 1520:     ptestresult.strace.bpf-success.test: FAILED -> PASSED
              Line 1521:     ptestresult.strace.brk.test: FAILED -> PASSED
              Line 1522:     ptestresult.strace.btrfs-X.test: FAILED -> PASSED
              Line 1523:     ptestresult.strace.btrfs-v.test: FAILED -> PASSED
              Line 1524:     ptestresult.strace.btrfs-vX.test: FAILED -> PASSED
              Line 1525:     ptestresult.strace.caps-abbrev.test: FAILED -> PASSED
              Line 1526:     ptestresult.strace.caps.test: FAILED -> PASSED
              Line 1527:     ptestresult.strace.clone_parent.test: FAILED -> PASSED
              Line 1528:     ptestresult.strace.clone_ptrace.test: FAILED -> PASSED
              Line 1529:     ptestresult.strace.count-f.test: FAILED -> PASSED
              Line 1530:     ptestresult.strace.count.test: FAILED -> PASSED
              Line 1531:     ptestresult.strace.delay.test: FAILED -> PASSED
              Line 1532:     ptestresult.strace.detach-running.test: FAILED -> PASSED
              Line 1533:     ptestresult.strace.detach-sleeping.test: FAILED -> PASSED
              Line 1534:     ptestresult.strace.detach-stopped.test: FAILED -> PASSED
              Line 1535:     ptestresult.strace.eventfd.test: FAILED -> PASSED
              Line 1536:     ptestresult.strace.execve-v.test: FAILED -> PASSED
              Line 1537:     ptestresult.strace.execve.test: FAILED -> PASSED
              Line 1538:     ptestresult.strace.fadvise64.test: FAILED -> PASSED
              Line 1539:     ptestresult.strace.fflush.test: FAILED -> PASSED
              Line 1540:     ptestresult.strace.filter-unavailable.test: FAILED -> PASSED
              Line 1541:     ptestresult.strace.first_exec_failure.test: FAILED -> PASSED
              Line 1542:     ptestresult.strace.futex.test: FAILED -> PASSED
              Line 1543:     ptestresult.strace.get_regs.test: FAILED -> PASSED
              Line 1544:     ptestresult.strace.getuid.test: FAILED -> PASSED
              Line 1545:     ptestresult.strace.inject-nf.test: FAILED -> PASSED
              Line 1546:     ptestresult.strace.interactive_block.test: FAILED -> PASSED
              Line 1547:     ptestresult.strace.ioctl.test: FAILED -> PASSED
              Line 1548:     ptestresult.strace.ioctl_evdev-success-v.test: FAILED -> PASSED
              Line 1549:     ptestresult.strace.ioctl_evdev-success.test: FAILED -> PASSED
              Line 1550:     ptestresult.strace.ioctl_perf-success.test: FAILED -> PASSED
              Line 1551:     ptestresult.strace.ipc_msgbuf.test: FAILED -> PASSED
              Line 1554:     ptestresult.strace.lseek.test: FAILED -> PASSED
              Line 1555:     ptestresult.strace.mmap.test: FAILED -> PASSED
              Line 1556:     ptestresult.strace.net-tpacket_stats-success.test: FAILED -> PASSED
              Line 1557:     ptestresult.strace.net-y-unix.test: FAILED -> PASSED
              Line 1558:     ptestresult.strace.net-yy-inet.test: FAILED -> PASSED
              Line 1561:     ptestresult.strace.net.test: FAILED -> PASSED
              Line 1563:     ptestresult.strace.nsyscalls-d.test: FAILED -> PASSED
              Line 1564:     ptestresult.strace.nsyscalls-nd.test: FAILED -> PASSED
              Line 1565:     ptestresult.strace.nsyscalls.test: FAILED -> PASSED
              Line 1566:     ptestresult.strace.opipe.test: FAILED -> PASSED
              Line 1567:     ptestresult.strace.pc.test: FAILED -> PASSED
              Line 1568:     ptestresult.strace.personality.test: FAILED -> PASSED
              Line 1569:     ptestresult.strace.pipe.test: FAILED -> PASSED
              Line 1570:     ptestresult.strace.poll-P.test: FAILED -> PASSED
              Line 1571:     ptestresult.strace.poll.test: FAILED -> PASSED
              Line 1572:     ptestresult.strace.prctl-arg2-intptr.test: FAILED -> PASSED
              Line 1573:     ptestresult.strace.prctl-dumpable.test: FAILED -> PASSED
              Line 1574:     ptestresult.strace.prctl-name.test: FAILED -> PASSED
              Line 1575:     ptestresult.strace.prctl-no-args.test: FAILED -> PASSED
              Line 1576:     ptestresult.strace.prctl-pdeathsig.test: FAILED -> PASSED
              Line 1577:     ptestresult.strace.prctl-seccomp-filter-v.test: FAILED -> PASSED
              Line 1578:     ptestresult.strace.prctl-seccomp-strict.test: FAILED -> PASSED
              Line 1579:     ptestresult.strace.prctl-securebits.test: FAILED -> PASSED
              Line 1580:     ptestresult.strace.prctl-spec-inject.test: FAILED -> PASSED
              Line 1581:     ptestresult.strace.prctl-tid_address.test: FAILED -> PASSED
              Line 1582:     ptestresult.strace.prctl-tsc.test: FAILED -> PASSED
              Line 1583:     ptestresult.strace.printpath-umovestr-legacy.test: FAILED -> PASSED
              Line 1584:     ptestresult.strace.printstrn-umoven-legacy.test: FAILED -> PASSED
              Line 1586:     ptestresult.strace.printstrn-umoven-undumpable.gen.test: FAILED -> PASSED
              Line 1587:     ptestresult.strace.printstrn-umoven.gen.test: FAILED -> PASSED
              Line 1588:     ptestresult.strace.prlimit64.gen.test: FAILED -> PASSED
              Line 1589:     ptestresult.strace.process_vm_readv.gen.test: FAILED -> PASSED
              Line 1590:     ptestresult.strace.process_vm_writev.gen.test: FAILED -> PASSED
              Line 1591:     ptestresult.strace.pselect6.gen.test: FAILED -> PASSED
              Line 1592:     ptestresult.strace.ptrace.gen.test: FAILED -> PASSED
              Line 1593:     ptestresult.strace.ptrace_syscall_info.gen.test: FAILED -> PASSED
              Line 1594:     ptestresult.strace.pwritev.gen.test: FAILED -> PASSED
              Line 1595:     ptestresult.strace.qual_fault-exit_group.test: FAILED -> PASSED
              Line 1596:     ptestresult.strace.qual_fault-syntax.test: FAILED -> PASSED
              Line 1597:     ptestresult.strace.qual_fault-syscall.test: FAILED -> PASSED
              Line 1598:     ptestresult.strace.qual_fault.test: FAILED -> PASSED
              Line 1599:     ptestresult.strace.qual_inject-error-signal-syscall.test: FAILED -> PASSED
              Line 1600:     ptestresult.strace.qual_inject-error-signal.test: FAILED -> PASSED
              Line 1601:     ptestresult.strace.qual_inject-retval-syscall.test: FAILED -> PASSED
              Line 1602:     ptestresult.strace.qual_inject-retval.test: FAILED -> PASSED
              Line 1603:     ptestresult.strace.qual_inject-signal-syscall.test: FAILED -> PASSED
              Line 1604:     ptestresult.strace.qual_inject-signal.test: FAILED -> PASSED
              Line 1605:     ptestresult.strace.qual_inject-syntax.test: FAILED -> PASSED
              Line 1606:     ptestresult.strace.qual_signal.test: FAILED -> PASSED
              Line 1607:     ptestresult.strace.qual_syscall.test: FAILED -> PASSED
              Line 1608:     ptestresult.strace.quotactl-Xabbrev.gen.test: FAILED -> PASSED
              Line 1609:     ptestresult.strace.quotactl-Xraw.gen.test: FAILED -> PASSED
              Line 1610:     ptestresult.strace.quotactl-Xverbose.gen.test: FAILED -> PASSED
              Line 1611:     ptestresult.strace.quotactl-success-v.test: FAILED -> PASSED
              Line 1612:     ptestresult.strace.quotactl-success.test: FAILED -> PASSED
              Line 1613:     ptestresult.strace.quotactl-v.gen.test: FAILED -> PASSED
              Line 1614:     ptestresult.strace.quotactl-xfs-success-v.test: FAILED -> PASSED
              Line 1615:     ptestresult.strace.quotactl-xfs-success.test: FAILED -> PASSED
              Line 1616:     ptestresult.strace.quotactl-xfs-v.gen.test: FAILED -> PASSED
              Line 1617:     ptestresult.strace.quotactl-xfs.gen.test: FAILED -> PASSED
              Line 1618:     ptestresult.strace.quotactl.gen.test: FAILED -> PASSED
              Line 1619:     ptestresult.strace.readahead.gen.test: FAILED -> PASSED
              Line 1621:     ptestresult.strace.readlink.gen.test: FAILED -> PASSED
              Line 1622:     ptestresult.strace.readlinkat.gen.test: FAILED -> PASSED
              Line 1623:     ptestresult.strace.readv.test: FAILED -> PASSED
              Line 1624:     ptestresult.strace.reboot.gen.test: FAILED -> PASSED
              Line 1625:     ptestresult.strace.recvfrom.gen.test: FAILED -> PASSED
              Line 1626:     ptestresult.strace.recvmmsg-timeout.gen.test: FAILED -> PASSED
              Line 1627:     ptestresult.strace.recvmsg.gen.test: FAILED -> PASSED
              Line 1629:     ptestresult.strace.remap_file_pages-Xabbrev.gen.test: FAILED -> PASSED
              Line 1630:     ptestresult.strace.remap_file_pages-Xraw.gen.test: FAILED -> PASSED
              Line 1631:     ptestresult.strace.remap_file_pages-Xverbose.gen.test: FAILED -> PASSED
              Line 1632:     ptestresult.strace.remap_file_pages.gen.test: FAILED -> PASSED
              Line 1633:     ptestresult.strace.rename.gen.test: FAILED -> PASSED
              Line 1634:     ptestresult.strace.renameat.gen.test: FAILED -> PASSED
              Line 1635:     ptestresult.strace.renameat2.gen.test: FAILED -> PASSED
              Line 1636:     ptestresult.strace.request_key.gen.test: FAILED -> PASSED
              Line 1638:     ptestresult.strace.rmdir.gen.test: FAILED -> PASSED
              Line 1639:     ptestresult.strace.rt_sigaction.test: FAILED -> PASSED
              Line 1640:     ptestresult.strace.rt_sigpending.gen.test: FAILED -> PASSED
              Line 1641:     ptestresult.strace.rt_sigprocmask.gen.test: FAILED -> PASSED
              Line 1642:     ptestresult.strace.rt_sigqueueinfo.gen.test: FAILED -> PASSED
              Line 1643:     ptestresult.strace.rt_sigreturn.gen.test: FAILED -> PASSED
              Line 1644:     ptestresult.strace.rt_sigsuspend.gen.test: FAILED -> PASSED
              Line 1645:     ptestresult.strace.rt_sigtimedwait.gen.test: FAILED -> PASSED
              Line 1646:     ptestresult.strace.rt_tgsigqueueinfo.gen.test: FAILED -> PASSED
              Line 1653:     ptestresult.strace.sched_get_priority_mxx.gen.test: FAILED -> PASSED
              Line 1654:     ptestresult.strace.sched_rr_get_interval.gen.test: FAILED -> PASSED
              Line 1655:     ptestresult.strace.sched_xetaffinity.gen.test: FAILED -> PASSED
              Line 1656:     ptestresult.strace.sched_xetattr.gen.test: FAILED -> PASSED
              Line 1657:     ptestresult.strace.sched_xetparam.gen.test: FAILED -> PASSED
              Line 1658:     ptestresult.strace.sched_xetscheduler.gen.test: FAILED -> PASSED
              Line 1659:     ptestresult.strace.sched_yield.gen.test: FAILED -> PASSED
              Line 1661:     ptestresult.strace.seccomp-filter-v.gen.test: FAILED -> PASSED
              Line 1662:     ptestresult.strace.seccomp-filter.gen.test: FAILED -> PASSED
              Line 1663:     ptestresult.strace.seccomp-strict.test: FAILED -> PASSED
              Line 1664:     ptestresult.strace.seccomp_get_action_avail.gen.test: FAILED -> PASSED
              Line 1665:     ptestresult.strace.select-P.gen.test: FAILED -> PASSED
              Line 1666:     ptestresult.strace.select.gen.test: FAILED -> PASSED
              Line 1667:     ptestresult.strace.semop.gen.test: FAILED -> PASSED
              Line 1668:     ptestresult.strace.sendfile.gen.test: FAILED -> PASSED
              Line 1671:     ptestresult.strace.setdomainname.gen.test: FAILED -> PASSED
              Line 1672:     ptestresult.strace.setfsgid.gen.test: FAILED -> PASSED
              Line 1674:     ptestresult.strace.setfsuid.gen.test: FAILED -> PASSED
              Line 1676:     ptestresult.strace.setgid.gen.test: FAILED -> PASSED
              Line 1678:     ptestresult.strace.setgroups.gen.test: FAILED -> PASSED
              Line 1680:     ptestresult.strace.sethostname.gen.test: FAILED -> PASSED
              Line 1681:     ptestresult.strace.setns.gen.test: FAILED -> PASSED
              Line 1682:     ptestresult.strace.setregid.gen.test: FAILED -> PASSED
              Line 1684:     ptestresult.strace.setresgid.gen.test: FAILED -> PASSED
              Line 1686:     ptestresult.strace.setresuid.gen.test: FAILED -> PASSED
              Line 1688:     ptestresult.strace.setreuid.gen.test: FAILED -> PASSED
              Line 1690:     ptestresult.strace.setrlimit.gen.test: FAILED -> PASSED
              Line 1691:     ptestresult.strace.setuid.gen.test: FAILED -> PASSED
              Line 1693:     ptestresult.strace.shmxt.gen.test: FAILED -> PASSED
              Line 1694:     ptestresult.strace.shutdown.gen.test: FAILED -> PASSED
              Line 1696:     ptestresult.strace.sigaltstack.test: FAILED -> PASSED
              Line 1697:     ptestresult.strace.siginfo.gen.test: FAILED -> PASSED
              Line 1699:     ptestresult.strace.signal_receive.gen.test: FAILED -> PASSED
              Line 1700:     ptestresult.strace.signalfd4.gen.test: FAILED -> PASSED
              Line 1705:     ptestresult.strace.so_error.gen.test: FAILED -> PASSED
              Line 1706:     ptestresult.strace.so_linger.gen.test: FAILED -> PASSED
              Line 1707:     ptestresult.strace.so_peercred-Xabbrev.gen.test: FAILED -> PASSED
              Line 1708:     ptestresult.strace.so_peercred-Xraw.gen.test: FAILED -> PASSED
              Line 1709:     ptestresult.strace.so_peercred-Xverbose.gen.test: FAILED -> PASSED
              Line 1710:     ptestresult.strace.so_peercred.gen.test: FAILED -> PASSED
              Line 1711:     ptestresult.strace.sock_filter-v-Xabbrev.gen.test: FAILED -> PASSED
              Line 1712:     ptestresult.strace.sock_filter-v-Xraw.gen.test: FAILED -> PASSED
              Line 1713:     ptestresult.strace.sock_filter-v-Xverbose.gen.test: FAILED -> PASSED
              Line 1714:     ptestresult.strace.sock_filter-v.gen.test: FAILED -> PASSED
              Line 1716:     ptestresult.strace.sockopt-sol_netlink.gen.test: FAILED -> PASSED
              Line 1717:     ptestresult.strace.splice.gen.test: FAILED -> PASSED
              Line 1718:     ptestresult.strace.stat.gen.test: FAILED -> PASSED
              Line 1720:     ptestresult.strace.statfs.gen.test: FAILED -> PASSED
              Line 1722:     ptestresult.strace.statx.gen.test: FAILED -> PASSED
              Line 1723:     ptestresult.strace.sun_path.test: FAILED -> PASSED
              Line 1724:     ptestresult.strace.swap.gen.test: FAILED -> PASSED
              Line 1726:     ptestresult.strace.symlink.gen.test: FAILED -> PASSED
              Line 1727:     ptestresult.strace.symlinkat.gen.test: FAILED -> PASSED
              Line 1728:     ptestresult.strace.sync.gen.test: FAILED -> PASSED
              Line 1729:     ptestresult.strace.sync_file_range.gen.test: FAILED -> PASSED
              Line 1731:     ptestresult.strace.sysinfo.gen.test: FAILED -> PASSED
              Line 1732:     ptestresult.strace.syslog.gen.test: FAILED -> PASSED
              Line 1733:     ptestresult.strace.tee.gen.test: FAILED -> PASSED
              Line 1734:     ptestresult.strace.time.gen.test: FAILED -> PASSED
              Line 1735:     ptestresult.strace.timer_create.gen.test: FAILED -> PASSED
              Line 1736:     ptestresult.strace.timer_xettime.gen.test: FAILED -> PASSED
              Line 1737:     ptestresult.strace.timerfd_xettime.gen.test: FAILED -> PASSED
              Line 1738:     ptestresult.strace.times-fail.gen.test: FAILED -> PASSED
              Line 1739:     ptestresult.strace.times.gen.test: FAILED -> PASSED
              Line 1740:     ptestresult.strace.truncate.gen.test: FAILED -> PASSED
              Line 1743:     ptestresult.strace.uio.test: FAILED -> PASSED
              Line 1744:     ptestresult.strace.umask.gen.test: FAILED -> PASSED
              Line 1746:     ptestresult.strace.umount2.test: FAILED -> PASSED
              Line 1749:     ptestresult.strace.umovestr.test: FAILED -> PASSED
              Line 1750:     ptestresult.strace.umovestr2.test: FAILED -> PASSED
              Line 1751:     ptestresult.strace.umovestr3.gen.test: FAILED -> PASSED
              Line 1752:     ptestresult.strace.uname.test: FAILED -> PASSED
              Line 1754:     ptestresult.strace.unix-pair-sendto-recvfrom.test: FAILED -> PASSED
              Line 1755:     ptestresult.strace.unlink.gen.test: FAILED -> PASSED
              Line 1756:     ptestresult.strace.unlinkat.gen.test: FAILED -> PASSED
              Line 1757:     ptestresult.strace.unshare.gen.test: FAILED -> PASSED
              Line 1758:     ptestresult.strace.userfaultfd.gen.test: FAILED -> PASSED
              Line 1759:     ptestresult.strace.ustat.gen.test: FAILED -> PASSED
              Line 1760:     ptestresult.strace.utime.gen.test: FAILED -> PASSED
              Line 1761:     ptestresult.strace.utimensat-Xabbrev.gen.test: FAILED -> PASSED
              Line 1762:     ptestresult.strace.utimensat-Xraw.gen.test: FAILED -> PASSED
              Line 1763:     ptestresult.strace.utimensat-Xverbose.gen.test: FAILED -> PASSED
              Line 1764:     ptestresult.strace.utimensat.gen.test: FAILED -> PASSED
              Line 1765:     ptestresult.strace.utimes.gen.test: FAILED -> PASSED
              Line 1766:     ptestresult.strace.vfork-f.gen.test: FAILED -> PASSED
              Line 1767:     ptestresult.strace.vhangup.gen.test: FAILED -> PASSED
              Line 1768:     ptestresult.strace.vmsplice.gen.test: FAILED -> PASSED
              Line 1769:     ptestresult.strace.wait4-v.gen.test: FAILED -> PASSED
              Line 1770:     ptestresult.strace.wait4.gen.test: FAILED -> PASSED
              Line 1771:     ptestresult.strace.waitid-v.gen.test: FAILED -> PASSED
              Line 1772:     ptestresult.strace.waitid.gen.test: FAILED -> PASSED
              Line 1774:     ptestresult.strace.xattr-strings.gen.test: FAILED -> PASSED
              Line 1775:     ptestresult.strace.xattr.gen.test: FAILED -> PASSED
              Line 1776:     ptestresult.strace.xet_robust_list.gen.test: FAILED -> PASSED
              Line 1777:     ptestresult.strace.xet_thread_area_x86.test: FAILED -> PASSED
              Line 1778:     ptestresult.strace.xetitimer.gen.test: FAILED -> PASSED
              Line 1779:     ptestresult.strace.xetpgid.gen.test: FAILED -> PASSED
              Line 1780:     ptestresult.strace.xetpriority.gen.test: FAILED -> PASSED
              Line 1781:     ptestresult.strace.xettimeofday.gen.test: FAILED -> PASSED
Comment 1 Ross Burton 2019-03-28 15:02:45 UTC
$ cat log |grep strace| cut -d: -f2|sort|uniq
 FAILED -> PASSED
 FAILED -> SKIPPED
 SKIPPED -> PASSED

None of those tests regressed.
Comment 2 Randy MacLeod 2019-03-28 15:17:28 UTC
Kai, 

There's some concern about the reproducibility and accuracy of the ptest results that may be related to the ptest-runner script for each packages. That's why Ross closed this a resolved, not a bug. I've re-opened it and I ask you to see what you think of the results you get for strace and try to improve the pass rate. If after an initial look, you think that there is no way to improve the situation, just explain why in this bug report and we'll discuss how to handle the defect.
Comment 3 Sudhir Sharma 2019-03-29 09:38:08 UTC
These Ptest results are generated using new method which is inconsistent.  So, Not a defect, Need to follow old method till new one stabilized. Hence, closing it.