Bug 8307

Impact both QA and CI testing, pls. prioritize.

Adding the SIGTERM handler broke the (the already broken) exported test feature. The problem right now with exported test is that has been broken for months and is not easy to test changes against it.

zjh, cathy,

If you have a working version of runexported tests, please consider to share it in the mailing list.

Send patch to the mailing list to fix the break introduced when handling the SIGTERM signal in testimage. You can find the patch here:

http://lists.openembedded.org/pipermail/openembedded-core/2015-September/110708.html

And the commit is also in poky contrib here:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=mariano/bug8307

I've already checked and the sigterm var is handled fine when exporting the test, although running the exported test is still broken

(In reply to comment #3)
> zjh, cathy,
> 
> If you have a working version of runexported tests, please consider to share
> it in the mailing list.

We didn't use the original export feature of yocto, the original feature has missed bb package(since runtest.py had imported bb).
Most of our export features are from yocto, you can try it with these steps:

ssh://zjh@10.239.52.71/home/zjh/meta-qa
passwd: 123456
1. add this layer to your poky: 
    cd [path to your poky]; git clone ssh://zjh@10.239.52.71/home/zjh/meta-qa; vi build/conf/bblayers.conf
2. add thease info to your build/conf/local.conf:
INHERIT += "test-iot"
TEST_EXPORT_DIR = "iottest"
TEST_TARGET = "simpleremote"
TEST_TARGET_IP = "192.168.7.2"
TEST_SERVER_IP = "192.168.7.1"
TEST_SUITES_MANIFEST = "[path to your poky]/meta-qa/conf/test/iottest.manifest"

3. bitbake core-image-sato, this step will build hello and hello-host
4. start your qemu
5. use another terminal to run test suite: bitbake core-image-sato -c test_iot
6. export testsuite: bitbake core-image-sato -c test_iot_export
7. run exported test suite: cd tmp/deploy/testsuite; tar xf  iot-testsuite.tar.gz; tar xf iot-testfiles.qemux86 .tar.gz -C iottest; cd iottest; python runtest.py -m qemux86 -t 192.168.7.2

(In reply to comment #4)
> Send patch to the mailing list to fix the break introduced when handling the
> SIGTERM signal in testimage. You can find the patch here:
> 
> http://lists.openembedded.org/pipermail/openembedded-core/2015-September/
> 110708.html
> 
> And the commit is also in poky contrib here:
> 
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=mariano/bug8307
> 
> I've already checked and the sigterm var is handled fine when exporting the
> test, although running the exported test is still broken

If you follow my steps from my previous comment, you will find this issue is still exist after apply your patch

(In reply to comment #5)
> 
> (In reply to comment #3)
> > zjh, cathy,
> > 
> > If you have a working version of runexported tests, please consider to share
> > it in the mailing list.
> 
> We didn't use the original export feature of yocto, the original feature has
> missed bb package(since runtest.py had imported bb).
> Most of our export features are from yocto, you can try it with these steps:
> 
> ssh://zjh@10.239.52.71/home/zjh/meta-qa
> passwd: 123456
> 1. add this layer to your poky: 
>     cd [path to your poky]; git clone
> ssh://zjh@10.239.52.71/home/zjh/meta-qa; vi build/conf/bblayers.conf
> 2. add thease info to your build/conf/local.conf:
> INHERIT += "test-iot"
> TEST_EXPORT_DIR = "iottest"
> TEST_TARGET = "simpleremote"
> TEST_TARGET_IP = "192.168.7.2"
> TEST_SERVER_IP = "192.168.7.1"
> TEST_SUITES_MANIFEST = "[path to your
> poky]/meta-qa/conf/test/iottest.manifest"
> 
> 3. bitbake core-image-sato, this step will build hello and hello-host
> 4. start your qemu
> 5. use another terminal to run test suite: bitbake core-image-sato -c
> test_iot
> 6. export testsuite: bitbake core-image-sato -c test_iot_export
> 7. run exported test suite: cd tmp/deploy/testsuite; tar xf 
> iot-testsuite.tar.gz; tar xf iot-testfiles.qemux86 .tar.gz -C iottest; cd
> iottest; python runtest.py -m qemux86 -t 192.168.7.2

step 5 is not necessary in this case

I was able to reproduce the problem. The issue is within meta-qa on runtest.py.

You are using runtest instead of runexported; in runtest you recreate testContext by hand (d and target) but in runexported the object data is stored in a JSON file and read from there.

Also the host_dumper object is never recreated in runtest, so if at some point a test needs it you'll have another issue.

What I would recommend is to recreate the missing objects or attributes in runtest using the exported JSON produced by testimage, otherwise if we keep adding functionality to the testimage class you would experience more errors.

Yes, you are right. I have tried this in runexport.py(need add poky/bitbake/lib to sys.path first in runexport.py). Didn't find this issue.
We were trying to keep the test suite is independent before. So we removed some feature from runexport.py.
Anyway, this is not a upstream bug. Thanks for your clarification. We will change our scripts to keep sync up with upstream.

Thanks, close this ticket.