Bug 8655 - ptest - two openssl ptests fail on 2.0.rc3
Summary: ptest - two openssl ptests fail on 2.0.rc3
Status: VERIFIED FIXED
Alias: None
Product: Runtime Testing
Classification: QA/Testing
Component: general (show other bugs)
Version: 2.0
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 2.1 M3
Assignee: Maxin B. John
QA Contact: Francisco Pedraza
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2015-11-06 07:42 UTC by Alexandru Georgescu
Modified: 2018-04-24 04:10 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: Regression (Used to work)
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexandru Georgescu 2015-11-06 07:42:49 UTC
A regrssion has been found while running openssl ptest on YP 2.0.rc3:fc45deac89ef63ca1c44e763c38ced7dfd72cbe1


AB Image: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.0.rc3/ptest/machines/genericx86-64/core-image-sato-sdk-genericx86-64.hddimg

pTest openSSL report: https://wiki.yoctoproject.org/wiki/Ptest_fc45deac89ef63ca1c44e763c38ced7dfd72cbe1

HW Tested: NUC D54250WYK


Tests that failed now and worked last time: 2
test_bn
test_cms
Comment 1 Maxin B. John 2016-01-28 11:47:15 UTC
Executed the openssl ptest on a core-image-sato image in the following HW. 

a. NUC D54250WYK
b. NUC 3815TYKE

All openssl ptests passed in these HW.
Comment 2 Francisco Pedraza 2016-02-04 10:11:23 UTC
Retested on  NUC 5425WYK genericx86_64, 2016-01-28:
poky: 3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c
AB Link: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.1_M2.rc1/ptest/machines/genericx86-64/core-image-sato-sdk-genericx86-64.hddimg

Report: https://wiki.yoctoproject.org/wiki/Ptest_3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c

getting the following results:
Tests that failed now and worked last time: 1
test_clienthello
Tests that passed now and and failed last time: 2
test_bn
test_cms
Tests that were skipped now and worked last time: 0

Tests that were skipped last time and worked now: 1
test_clienthello
Tests that failed now: 1
test_clienthello
Comment 3 Maxin B. John 2016-02-04 11:33:38 UTC
(In reply to comment #2)
> Retested on  NUC 5425WYK genericx86_64, 2016-01-28:
> poky: 3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c
> AB Link:
> http://autobuilder.yoctoproject.org/pub/releases/yocto-2.1_M2.rc1/ptest/
> machines/genericx86-64/core-image-sato-sdk-genericx86-64.hddimg
> 
> Report:
> https://wiki.yoctoproject.org/wiki/
> Ptest_3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c
> 
> getting the following results:
> Tests that failed now and worked last time: 1
> test_clienthello
> Tests that passed now and and failed last time: 2
> test_bn
> test_cms

Apparently, these tests fail randomly.

> Tests that were skipped now and worked last time: 0
> 
> Tests that were skipped last time and worked now: 1
> test_clienthello
> Tests that failed now: 1
> test_clienthello
Comment 4 Maxin B. John 2016-02-18 16:26:55 UTC
Big Number library tests (test_bn) and Cryptographic Message Syntax tests (test_cms) passed in the last 3 automated ptest results:

1. https://wiki.yoctoproject.org/wiki/Ptest_3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c
2. https://wiki.yoctoproject.org/wiki/Ptest_34e95b0349043030b430a8fa22ffd15aae17017c
3. https://wiki.yoctoproject.org/wiki/Ptest_34e95b0349043030b430a8fa22ffd15aae17017c


All tests passed when I tried to run the openssl ptest on NUCs and qemu targets. So, closing with "Resolved/Worksforme" status.
Comment 5 Francisco Pedraza 2016-03-10 20:20:56 UTC
(In reply to comment #4)
> Big Number library tests (test_bn) and Cryptographic Message Syntax tests
> (test_cms) passed in the last 3 automated ptest results:
> 
> 1.
> https://wiki.yoctoproject.org/wiki/
> Ptest_3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c
> 2.
> https://wiki.yoctoproject.org/wiki/
> Ptest_34e95b0349043030b430a8fa22ffd15aae17017c
> 3.
> https://wiki.yoctoproject.org/wiki/
> Ptest_34e95b0349043030b430a8fa22ffd15aae17017c
> 
> 
> All tests passed when I tried to run the openssl ptest on NUCs and qemu
> targets. So, closing with "Resolved/Worksforme" status.


Hi, 
I am trying to verify this one the link: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.1_M2.rc1/ptest/
is broken, do you know who can help me on this ?
Regards,
Comment 6 Jose Perez C 2016-10-20 19:27:17 UTC
Tested on commit : 4b94b498e21aeba945fe7e72a6b7c4bb0314fb83 and is not reproducible anymore

https://wiki.yoctoproject.org/wiki/Ptest_4b94b498e21aeba945fe7e72a6b7c4bb0314fb83