Bug 14491

Summary: [QA 3.4_M2.rc1] - stap.StapTest.test_stap failure
Product: [QA/Testing] Runtime Testing Reporter: Teoh Jay Shen <jay.shen.teoh>
Component: oeqa/runtimeAssignee: Anuj Mittal <anuj.mittal>
Status: RESOLVED FIXED QA Contact: Teoh Jay Shen <jay.shen.teoh>
Severity: normal    
Priority: Medium+ CC: akuster808, anuj.mittal, randy.macleod, richard.purdie, sangeeta.jain, tim.orling
Version: 3.4   
Target Milestone: 3.3.4   
Hardware: All   
OS: x86_64   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
Error Log none

Description Teoh Jay Shen 2021-07-26 05:44:50 UTC
Created attachment 4816 [details]
Error Log

The following runtime test is failing in the current release(3.4_M2.rc1): 
         
   stap.StapTest.test_stap

This failure is reproducible and the error log for this failure is extracted and attached in this bug as well.
Comment 1 Richard Purdie 2021-07-27 16:35:22 UTC
Was this an image downloaded from the yocto project build or one you built locally? 

If downloaded, which image was it?
If built locally, what configuration was used to build it?

It is possible the 5.13 linux-libc-headers upgrade may have caused it but we're struggling to understand how to reproduce it and why qemux86/qemux8-64 are passing on the autobuilder.

We could do with more information to understand this, it would be helpful to be able to reproduce the issue.
Comment 2 Teoh Jay Shen 2021-07-28 05:30:33 UTC
(In reply to comment #1)
> Was this an image downloaded from the yocto project build or one you built
> locally? 
> 
> If downloaded, which image was it?
> If built locally, what configuration was used to build it?

Hi Richard, this failure was first observed on the genericx86/genericx86-64 images built by our internal autobuilder. However, to identify this issue also occurs on the genericx86-64 image from the yocto project build, I did run the test both manually and automatically on the genericx86-64 image downloaded from the yocto project build and the stap test case is still failing. Outputs are same as the info logged in the error log attached in this bug.

Image src:
https://autobuilder.yocto.io/pub/releases/yocto-3.4_M2.rc1/machines/genericx86-64/core-image-sato-sdk-genericx86-64.wic

> It is possible the 5.13 linux-libc-headers upgrade may have caused it but
> we're struggling to understand how to reproduce it and why qemux86/qemux8-64
> are passing on the autobuilder.

I will try to reproduce it on qemux86-64.

> We could do with more information to understand this, it would be helpful to
> be able to reproduce the issue.

Below is the hardware and configurations I used to reproduce this issue for genericx86-64 architecture: 

Target machine : Intel NUC7i7BNH

Steps taken to reproduce this issue : 
-Execute the stap test case manually and automatically.

Link for this test case :
https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/runtime/cases/stap.py


My local.conf's configurations :
**********************************************************
MACHINE = "genericx86-64"
DISTRO = "poky"
SDKMACHINE = "i686"
PACKAGE_CLASSES = "package_rpm package_deb package_ipk"
CORE_IMAGE_EXTRA_INSTALL += "mesa-demos"
HOSTTOOLS_NONFATAL += "git-lfs"
IMAGE_FSTYPES_append = " wic wic.bmap tar.gz"
INHERIT += "testimage"
TEST_SUITES_remove = "ptest ltp ltp_compliance"
TEST_TARGET = "simpleremote"
TEST_SERVER_IP = ""
TEST_TARGET_IP = ""
**********************************************************
Comment 3 Teoh Jay Shen 2021-07-28 05:36:52 UTC
Seems like there might be some changes in the field definitions due to the kernel updates, but I still need some more time to investigate it.
Comment 4 Teoh Jay Shen 2021-07-28 06:30:27 UTC
(In reply to comment #2)
> (In reply to comment #1)
> > Was this an image downloaded from the yocto project build or one you built
> > locally? 
> > 
> > If downloaded, which image was it?
> > If built locally, what configuration was used to build it?
> 
> Hi Richard, this failure was first observed on the genericx86/genericx86-64
> images built by our internal autobuilder. However, to identify this issue
> also occurs on the genericx86-64 image from the yocto project build, I did
> run the test both manually and automatically on the genericx86-64 image
> downloaded from the yocto project build and the stap test case is still
> failing. Outputs are same as the info logged in the error log attached in
> this bug.
> 
> Image src:
> https://autobuilder.yocto.io/pub/releases/yocto-3.4_M2.rc1/machines/
> genericx86-64/core-image-sato-sdk-genericx86-64.wic
> 
> > It is possible the 5.13 linux-libc-headers upgrade may have caused it but
> > we're struggling to understand how to reproduce it and why qemux86/qemux8-64
> > are passing on the autobuilder.
> 
> I will try to reproduce it on qemux86-64.
> 

This failure was not able to be reproduce on qemux86-64 image.

> > We could do with more information to understand this, it would be helpful to
> > be able to reproduce the issue.
> 
> Below is the hardware and configurations I used to reproduce this issue for
> genericx86-64 architecture: 
> 
> Target machine : Intel NUC7i7BNH
> 
> Steps taken to reproduce this issue : 
> -Execute the stap test case manually and automatically.
> 
> Link for this test case :
> https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/runtime/
> cases/stap.py
> 
> 
> My local.conf's configurations :
> **********************************************************
> MACHINE = "genericx86-64"
> DISTRO = "poky"
> SDKMACHINE = "i686"
> PACKAGE_CLASSES = "package_rpm package_deb package_ipk"
> CORE_IMAGE_EXTRA_INSTALL += "mesa-demos"
> HOSTTOOLS_NONFATAL += "git-lfs"
> IMAGE_FSTYPES_append = " wic wic.bmap tar.gz"
> INHERIT += "testimage"
> TEST_SUITES_remove = "ptest ltp ltp_compliance"
> TEST_TARGET = "simpleremote"
> TEST_SERVER_IP = ""
> TEST_TARGET_IP = ""
> **********************************************************
Comment 5 Randy MacLeod 2021-07-29 14:38:49 UTC
likely due to linux libc headers for 5.13 according to Richard.
Comment 6 Randy MacLeod 2021-07-29 14:50:31 UTC
Put in NEEDINFO accidentally.
Comment 7 Richard Purdie 2021-07-29 17:18:06 UTC
I was able to reproduce this on genericx86-64 under runqemu.

https://sourceware.org/git/?p=systemtap.git;a=commitdiff;h=ef5a8b9eda402e4e96c4e3ce01e7ff95d3e10470

appears to fix it.
Comment 8 Teoh Jay Shen 2021-07-30 01:15:46 UTC
Noted & Thanks for the info!
Comment 10 Teoh Jay Shen 2021-09-15 05:04:59 UTC
This issue still present in yocto-3.3.3.rc2
Comment 11 Anuj Mittal 2021-09-15 16:52:22 UTC
(In reply to comment #10)
> This issue still present in yocto-3.3.3.rc2

Thanks Jay. I have included Richard's patch for the next PR.