Bug 10872

Specifically, we need to check that during build the feeds are configured without anything failing, and then that on the resulting image the repo manager can access them without error.

Later, the test should be extended to check for repository and package signatures (that signing works and signature verification works) - PACKAGE_FEED_SIGN and RPM_SIGN_PACKAGES are used to configure that.

We discussed how on-image testing is difficult during selftest, but there is a testcase which boots an image and tries things on it, and this could be added to it:

1) Set up the configuration variables in meta/lib/oeqa/selftest/runtime-test.py, specifically

class TestImage(oeSelfTest):
    def test_testimage_install(self):

2) Add testcases that run dnf on image to meta-selftest/lib/oeqa/runtime/cases/selftest.py 

This shouldn't make testing much longer.

Will work on add this test for smartpm.

Adding 9675 as dependency, once it is implemente we can think on changing this test to dnf.

Regarding the per-arch repodata issue, a couple points:

1) the repodata that is generated in image's private build directory for the purpose of installing packages into the image on the host should not be used to set up remote package feeds, in general. It's transient, contains only those packages that are required by the image, and will be wiped next time an image is built. It's okay to use for the image tests, but nothing else.

2) Generally there is no standard way in oe-core to set up package feeds, and I don't know how people do it. I guess they copy tmp/deploy/rpm to somewhere else, and then run metadata creation tools on it - for which they have custom recipes.

3) I really don't want to complicate indexing and dnf configuration code by iterating over architectures. createrepo_c is perfectly capable of recursing into directories, let's use that facility, unless someone points out a specific reason why this is not the right way (see point 1 - this repodata will not be used for remote feeds anyway).

4) Similarly, we can make the package feed configuration code behave differently than that of opkg/deb: if no arches are given, it will not iterate over the internal list and append them all, but rather append nothing at all. If people want per-arch repodata, they'll need to supply an explicit list of arches to feed creation code, and run createrepo against each directory. Again, I'd like to hear specific reasons first, before making code more complicated than it needs to be.

First, wanted to clarify what is being discussed here:

a. This bug is about adding a test for PACKAGE_FEED_URIS var, and most of the discussion is about actually implementing the setting of feeds in dnf based on this vars.

b. smartpm already has this functionality done, and, as the rest of the package managers, has a way to get a very solid choice of archs for the feeds even when PACKAGE_FEED_ARCHS are not given.

c. dnf is not yet in master and has not yet implemented the use of PACKAGE_FEED vars, which again, this bug is asking to test.

---

So, answering the statements about implementation in dnf and unique repo against a repo per arch:

Only the rpm-repo in the dnf branch has a repodata in the base folder of the repo. Master branch has smart creating a rpm base folder for the repo named 'rpms' and has a repo being created under each arch. This is a temporal repo, for internal reasons only, but still, this is the way that all package managers are expecting the structure and the way upstream repos such as fedora official updates and releases repos are structured.

It is a temporal repo, used for testing. But, even when I am okay with modifying environment conditions to make tests more efficient; modifying the code to adapt to the repo environment is a different thing.

As I have said, currently, smartpm and the rest of the package managers have a way to generate the archs even if PACKAGE_FEED_ARCHS is not given. They do it well, and what is being proposed is that dnf does not generate these archs when they are not given, but simply assume empty archs. What reason, other than because it will be easier to test with the new repo structure generated for the dnf branch, there is to not have dnf generate these arch values also?

---

That being said, I think the previous fragment and the rest of the conversation should be moved to a new bug. There is no reason to continue this talk about implementing this functionality in the dnf poky-contrib branch here. This bug is clearly about testing it. Since the dnf functionality is not done yet and is not in master, that is a bug/test for another time. I will use this bug to test the PACKAGE_FEED_URIS var in smartpm in master and kindly ask for a new bug to be opened to address implementation of the functionality in dnf.

But why is this even an issue for you? If you are writing tests for smart, you don't set PACKAGE_FEED_ARCHS at all, and the code in package_manager.py will expand it into the correct list of architectures, one per feed. You need to test that the expansion looks sane in the resulting config file and that smart is able to access those feeds, and nothing exits with an error.

Similarly, when porting the same test to dnf, you do not set the PACKAGE_FEED_ARCHS at all either, and you still need to do the same two things:
1) check that the config file in the image looks sane
2) check that smart is able to access the feed.

Whether in case of dnf there is a per-architecture feed or not, is irrelevant for the test, no?

It is important because smart knew how to correctly generate these archs and you are proposing to implement this for dnf (which again, is not the purpose of this bug) and not let dnf have an ability smart had.

Your test repo in dnf branch does not work with the PACKAGE_FEED_ARCHS, even if we give the right archs to the variable. You propose to not give the var at all and code around it, which doesn't sound right to me, adapting your code to work with your test repo doesn't seem like the way to go. This is already documented in the yocto manual, and adding dnf means that you need to support this var too.

I am working in this test already, but I am doing it for smart, which already has the correct structure. I will not test this for dnf in this bug, and won't implement the functionality for dnf to do this in this bug either.

If you have doubts about the smart implementation, we can continue this discussion. If your doubts are about dnf, specially about the implementation of this feature and not the test, please open a new bug for it. This is phrased as a bug to add tests, not to discuss what dnf should use as archs if PACKAGAE_FEED_ARCHS is not given.

Okay my bad, I read the code in master now more closely. For testing smart, it doesn't reuse the package feed produced during image creation, but rather creates its own, separate feed (the former is in 'rpms' directory, the latter in 'rpm' - very easy to confuse), and then tests smart against that. This latter feed is created by calling createrepo directly from testimage.bbclass (and doing some other tricks), and so it is set up exactly the way the tests need.

I should add the same thing to dnf testing, where they don't reuse the image creation feed, but rather make their own in a similar way. Then the whole issue where PACKAGE_FEED_ARCHS clash with how image creation feed is made becomes moot. Agree?

Agree. I only want to add that the image created folder is actually 'rpm' and the new repo created in package_manager.py is really the one called 'rpms'

(In reply to comment #10)
> Agree. I only want to add that the image created folder is actually 'rpm'
> and the new repo created in package_manager.py is really the one called
> 'rpms'

I'll change those names to be more descriptive, e.g. "oe-rootfs-repo" and "oe-testimage-repo".

The repo issue is now fixed in https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=akanavin/dnf-rpm4 - dnf test is now using the feeds created by testimage.bbclass (I did slight tweaks to them, but repodata is still per arch).

Sent patch for smart in master

http://lists.openembedded.org/pipermail/openembedded-core/2017-February/132994.html

Working in a v2 patch to address comments.

Fixed with http://lists.openembedded.org/pipermail/openembedded-core/2017-March/134122.html

Already in master http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=2b77735c72cb5100dbd850e79d4f639e82629842

Already in master, with a patch sent by Alex Kanavin, but still has a dependency to #9675 and can't be marked RESOLVED until that one is fixed.

I think you can totally resolve/fix this bug, as 9675 remains open only to update Yocto documentation, the rest of the work is done and merged.

Removed dependency so I could mark this bug as solved.

Test for PACKAGE_FEED_URIS  is added in selftest.

Build Configuration:
BB_VERSION        = "1.33.2"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.2+snapshot-20171204"
TUNE_FEATURES     = "m32 i586"
meta-selftest     = "HEAD:2b77735c72cb5100dbd850e79d4f639e82629842"

STEPS: 
1. git clone git://git.yoctoproject.org/poky"
2. $ git checkout <commit-id>
3. add   meta-sefltest  in vi conf/bblayers.conf
4. git clone git://git.yoctoproject.org/meta-selftest
5.$ bitbake rpm busybox run-postinsts
$ bitbake package-index
6. bitbake-selftest -v

expect-out:
all test pass 
actual expect-out:
all test pass

============================================================================
test_var_exec (bb.tests.codeparser.PythonReferenceTest) ... ok
test_var_reference (bb.tests.codeparser.PythonReferenceTest) ... ok
test_vardeps (bb.tests.codeparser.DependencyReferenceTest) ... ok
test_vardeps_expand (bb.tests.codeparser.DependencyReferenceTest) ... ok
============================================================================
full-log in attachment

Created attachment 4140 [details]
full-log

You are executing bitbake-selftest, but the test is in oe-selftest.

(In reply to comment #21)
> You are executing bitbake-selftest, but the test is in oe-selftest.

ok Im go  prove in oe-sefltes, thanks

 the test oe-sefltest run in ubuntu 16.04 AB GDC failed 
======================================================================
FAIL [392.534s]: test_testimage_install (oeqa.selftest.runtime-test.TestImage)
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ab/autobuilder/yocto-worker/nightly-oe-selftest-ubuntu1604/build/meta/lib/oeqa/selftest/runtime-test.py", line 130, in test_testimage_install
    bitbake('-c testimage core-image-full-cmdline')
  File "/home/ab/autobuilder/yocto-worker/nightly-oe-selftest-ubuntu1604/build/meta/lib/oeqa/utils/commands.py", line 143, in bitbake
    return runCmd(cmd, ignore_status, timeout, **options)
  File "/home/ab/autobuilder/yocto-worker/nightly-oe-selftest-ubuntu1604/build/meta/lib/oeqa/utils/commands.py", line 121, in runCmd
    raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, result.output))
AssertionError: Command 'bitbake  -c testimage core-image-full-cmdline' returned non-zero exit status 1:
Loading cache...done.
Loaded 1322 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies

ERROR: core-image-full-cmdline-1.0-r0 do_testimage: Function failed: do_testimage

----------------------------------------------------------------------

Build Configuration:
BB_VERSION        = "1.33.2"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal"
TARGET_SYS        = "x86_64-poky-linux"
MACHINE           = "qemux86-64"
DISTRO            = "poky"
DISTRO_VERSION    = "2.2+snapshot-20171204"
TUNE_FEATURES     = "m64 core2"
meta-selftest     = "heads/master:2b77735c72cb5100dbd850e79d4f639e82629842"

full-log in attachment

Created attachment 4143 [details]
oe-selftest-log

The error is:

NOTE: runqemu exited with code 1
Output from runqemu:
Could not initialize SDL(No available video device) - exiting

That is a qemu/build setup issue, which you need to fix first before reaching the point where image tests can be run.

the qemu/build setup issue, is solved and run the image test in oe-sefltest and all rerun and all succeeded. 

Build Configuration:
BB_VERSION        = "1.33.2"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.2+snapshot-20171205"
TUNE_FEATURES     = "m32 i586" 
meta-selftest     = "HEAD:2b77735c72cb5100dbd850e79d4f639e82629842"

STEPS: 
1. git clone git://git.yoctoproject.org/poky"
2. $ git checkout <commit-id>
3. add   meta-sefltest  in vi conf/bblayers.conf
4. git clone git://git.yoctoproject.org/meta-selftest
5.$ bitbake rpm busybox run-postinsts
$ bitbake package-index
6.bitbake oe-selftest-image


expect-out:
all test pass 
actual expect-out:
all test pass
================================================================================
NOTE: recipe oe-selftest-image-1.0-r0: task do_image_ext4: Started
NOTE: recipe oe-selftest-image-1.0-r0: task do_image_tar: Started
NOTE: recipe oe-selftest-image-1.0-r0: task do_image_ext4: Succeeded
NOTE: recipe oe-selftest-image-1.0-r0: task do_image_tar: Succeeded
NOTE: Running task 2392 of 2394 (/home/oscar/quemux/meta-selftest/recipes-test/images/oe-selftest-image.bb:do_image_complete)
NOTE: recipe oe-selftest-image-1.0-r0: task do_image_complete: Started
NOTE: recipe oe-selftest-image-1.0-r0: task do_image_complete: Succeeded
NOTE: Running task 2393 of 2394 (/home/oscar/quemux/meta-selftest/recipes-test/images/oe-selftest-image.bb:do_image_qa)
NOTE: recipe oe-selftest-image-1.0-r0: task do_image_qa: Started
NOTE: recipe oe-selftest-image-1.0-r0: task do_image_qa: Succeeded
NOTE: Running noexec task 2394 of 2394 (/home/oscar/quemux/meta-selftest/recipes-test/images/oe-selftest-image.bb:do_build)
NOTE: Tasks Summary: Attempted 2394 tasks of which 1770 didn't need to be rerun and all succeeded.
==============================================================================
full-log

Created attachment 4144 [details]
full-log

You are still verifying the wrong thing. 'bitbake oe-selftest-image' will not execute the test. You need to execute oe-selftest itself.

(In reply to comment #28)
> You are still verifying the wrong thing. 'bitbake oe-selftest-image' will
> not execute the test. You need to execute oe-selftest itself.

What would be the correct steps to verify this error?

(In reply to comment #29)
> (In reply to comment #28)
> > You are still verifying the wrong thing. 'bitbake oe-selftest-image' will
> > not execute the test. You need to execute oe-selftest itself.
> 
> What would be the correct steps to verify this error?

To verify this bug you can execute the "runtime_test" from oe-selftest suite, follow below steps and ensure all test cases are reported as PASS. 

1. echo "SANITY_TESTED_DISTROS=''" >> conf/local.conf
2. oe-selftest -r runtime_test 

as an alternative you can execute a oe-selfete buildset on the AB an verify that that below test cases are PASS.


runtime_test.Postinst.test_postinst_rootfs_and_boot
runtime_test.TestExport.test_testexport_basic
runtime_test.TestExport.test_testexport_sdk
--> runtime_test.TestImage.test_testimage_dnf --> "This is actually the test that is coevring the feature"
runtime_test.TestImage.test_testimage_install

 can execute a oe-selfete buildset on the AB an verify that that below test cases are PASS.

Build Configuration:
BB_VERSION           = "1.37.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "fedora-25"
TARGET_SYS           = "x86_64-poky-linux"
MACHINE              = "qemux86-64"
DISTRO               = "poky"
DISTRO_VERSION       = "2.4+snapshot-20171206"
TUNE_FEATURES        = "m64 core2"       
meta-yocto-bsp       = "heads/master:4469acdf1d0338220f3fe2ecb5e079eea6fda375"
============================================================================
2017-12-06 18:20:37,155 - oe-selftest - INFO - RESULTS - runtime_test.Postinst.test_postinst_rootfs_and_boot - Testcase 1540: PASSED
2017-12-06 18:20:37,155 - oe-selftest - INFO - RESULTS - runtime_test.TestExport.test_testexport_basic - Testcase 1499: PASSED
2017-12-06 18:20:37,155 - oe-selftest - INFO - RESULTS - runtime_test.TestExport.test_testexport_sdk - Testcase 1641: PASSED
2017-12-06 18:20:37,155 - oe-selftest - INFO - RESULTS - runtime_test.TestImage.test_testimage_dnf - Testcase 1883: PASSED
2017-12-06 18:20:37,155 - oe-selftest - INFO - RESULTS - runtime_test.TestImage.test_testimage_install - Testcase 1644: PASSED
2017-12-06 18:20:37,155 - oe-selftest - INFO - RESULTS - selftest.ExternalLayer.test_list_imported - Testcase 1885: PASSED
============================================================================
log in attachment

Created attachment 4151 [details]
all testcases pass