Bug 10840

Summary:	pTest- openssl: test cases "test_bn" and "test_cms" are failing
Product:	[Build System, Metadata & Runtime] OE-Core	Reporter:	Jose Perez C <jose.perez.carranza>
Component:	connectivity	Assignee:	Alexander Kanavin <alex.kanavin>
Status:	VERIFIED FIXED	QA Contact:
Severity:	normal
Priority:	Medium+	CC:	jku, meta.mr.watcher, meta.watcher, ross.burton
Version:	2.3
Target Milestone:	2.4 M1
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	No (bug/feature does not impact docs)

Description Jose Perez C 2016-12-21 21:50:40 UTC

Commit: 573c646d4cc62dcd0c230381df4940bdf314d495 
Image: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.3_M1.rc1/machines/genericx86-64/core-image-sato-sdk-ptest-genericx86-64.hddimg
HW: NUC6 

----- Steps ----

1. Download pTest image from autobuilder( you can find core-image-sato-sdk image in pTest directory)
* we test it on NUC with genericx86-64
2. Install the image on DUT (using legacy boot)
3. Boot the image
4. In command line type
   $ ptest-runner > ptest.log 

EXPECTED RESULT

 All openssl test cases should be PASS 

ACTUAL RESULT 

test cases are failing as below:
FAIL:test_bn
FAIL:test_cms

--- Ouput ----
make[2]: Entering directory '/usr/lib/openssl/ptest/test'
starting big number library test, could take a while...
running bc
make[2]: Leaving directory '/usr/lib/openssl/ptest/test'
FAIL: test_bn
---

make[2]: Entering directory '/usr/lib/openssl/ptest/test'
CMS consistency test
/bin/perl cms-test.pl
make[2]: Leaving directory '/usr/lib/openssl/ptest/test'
FAIL: test_cms


NOTE: full lg can be downloaded from here https://wiki.yoctoproject.org/wiki/images/8/81/Logs_2.3.M1.rc1.txt

Comment 1 Alexander Kanavin 2017-01-19 11:03:46 UTC

Postponing to M4, my hands are tied with bug 9675 now. In any case, I'd like to further investigate moving to openssl 1.1 first, by looking at specific packages that currently fail to build against it:
http://lists.openembedded.org/pipermail/openembedded-core/2016-December/130132.html

Comment 2 Jussi Kukkonen 2017-03-21 13:01:51 UTC

Had a moment while waiting for a build:

Both tests end up trying to execute build host perl (with absolute path). This is because even though test/Makefile sets "PERL = perl", the top-level makefile runs test/Makefile with "$(MAKE) -e $(BUILDENV)" and BUILDENV includes PERL, overriding the local PERL definition.

Removing PERL from BUILDENV makes the tests pass, but I'm not sure if it would break something during build -- probably not in our case at least.

Comment 3 Alexander Kanavin 2017-04-10 15:39:36 UTC

I ended up patching the Makefile that is installed into ptest directory. This is not particularly important though, as openssl 1.1 will run ptests directly, bypassing the Makefile altogether.

Comment 4 Jose Perez C 2017-04-28 17:33:54 UTC

Verified on 2.3 rc2 are working correctly 

https://wiki.yoctoproject.org/wiki/WW17_-_2017-04-21_-_Full_Test_Cycle_2.3_rc2#results