Bug 12609

Summary: dnf failures during testing on Fedora 27
Product: [Build System, Metadata & Runtime] OE-Core Reporter: California Sullivan <california.l.sullivan>
Component: coreAssignee: Alexander Kanavin <alex.kanavin>
Status: RESOLVED WORKSFORME QA Contact:
Severity: normal    
Priority: Undecided CC: meta.mr.watcher, meta.watcher
Version: 2.5   
Target Milestone: ---   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description California Sullivan 2018-03-16 17:19:29 UTC
A couple different builds had failing tests all involving dnf on the new Fedora 27 builder:

Pyro-next: https://autobuilder.yocto.io/builders/nightly-qa-extras/builds/871/steps/Running%20Sanity%20Tests_7/logs/stdio

---
Command: dnf --repofrompath=oe-testimage-repo-noarch,http://192.168.7.1:46709/noarch --repofrompath=oe-testimage-repo-qemux86_64,http://192.168.7.1:46709/qemux86_64 --repofrompath=oe-testimage-repo-core2_64,http://192.168.7.1:46709/core2_64 --nogpgcheck install -y http://192.168.7.1:46709/noarch/run-postinsts-dev-1.0-r9.noarch.rpm
...
Failed:
  run-postinsts.noarch 1.0-r9          run-postinsts-dev.noarch 1.0-r9         

Error: Transaction failed
---


Rocko-next: https://autobuilder.yocto.io/builders/nightly-musl-x86-64/builds/336/steps/Running%20Sanity%20Tests/logs/stdio

---
NOTE: core-image-sato-sdk-1.0-r0 do_testimage: test_dnf_install_dependency (dnf.DnfRepoTest)
NOTE: core-image-sato-sdk-1.0-r0 do_testimage:  ... FAIL
---
NOTE: core-image-sato-sdk-1.0-r0 do_testimage: test_dnf_reinstall (dnf.DnfRepoTest)
NOTE: core-image-sato-sdk-1.0-r0 do_testimage:  ... FAIL
---
| DEBUG: [Running]$ ssh -l root -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o LogLevel=ERROR 192.168.7.6 export PATH=/usr/sbin:/sbin:/usr/bin:/bin; dnf --repofrompath=oe-testimage-repo-noarch,http://192.168.7.5:41915/noarch --repofrompath=oe-testimage-repo-qemux86_64,http://192.168.7.5:41915/qemux86_64 --repofrompath=oe-testimage-repo-core2_64,http://192.168.7.5:41915/core2_64 --nogpgcheck install -y run-postinsts-dev
...
| No package run-postinsts-dev available.
| Error: Unable to find a match
| 
| NOTE:  ... FAIL
Comment 1 Ross Burton 2018-03-17 02:05:45 UTC
Could it be running the host dnf somehow?
Comment 2 Alexander Kanavin 2018-03-19 04:36:36 UTC
The tests are running dnf on the target image against a repository served by a host, so I don't think host dnf is involved. The rocko-next failing test seems to have network connectivity issues:

Added oe-testimage-repo-core2_64 repo from http://192.168.7.5:41915/core2_64
| Added oe-testimage-repo-qemux86_64 repo from http://192.168.7.5:41915/qemux86_64
| Added oe-testimage-repo-noarch repo from http://192.168.7.5:41915/noarch
| Failed to synchronize cache for repo 'oe-testimage-repo-core2_64', disabling.
| Failed to synchronize cache for repo 'oe-testimage-repo-qemux86_64', disabling.
| Failed to synchronize cache for repo 'oe-testimage-repo-noarch', disabling.


The pyro-next is installing and deinstalling a couple of packages in various ways, until it mysteriously fails:

Running transaction
  Installing  : run-postinsts-1.0-r9.noarch                                 1/2Created symlink /etc/systemd/system/sysinit.target.wants/run-postinsts.service \u2192 /lib/systemd/system/run-postinsts.service.
 
  Installing  : run-postinsts-dev-1.0-r9.noarch                             2/2run-postinsts-dev-1.0-r9.noarch was supposed to be installed but is not!
 
  Verifying   : run-postinsts-dev-1.0-r9.noarch                             1/2run-postinsts-1.0-r9.noarch was supposed to be installed but is not!
 
  Verifying   : run-postinsts-1.0-r9.noarch                                 2/2 

Failed:
  run-postinsts.noarch 1.0-r9          run-postinsts-dev.noarch 1.0-r9         

Error: Transaction failed


We need to narrow this down a bit, but I'm not sure how yet.
Comment 3 Alexander Kanavin 2018-03-19 04:45:23 UTC
Especially puzzling is that pyro-next ran the same test several times, and only the last run (involving systemd) is failing:
https://autobuilder.yocto.io/builders/nightly-qa-extras/builds/871/

In either case, I doubt it's F27's fault.
Comment 4 Alexander Kanavin 2018-03-19 09:21:34 UTC
In any case, if you see this again, do let me know, as I'd appreciate narrowing this down. Trying to reproduce locally now, on a debian testing host.
Comment 5 Alexander Kanavin 2018-03-19 10:00:41 UTC
rocko-next worked flawlessly (musl, running testimage for three images in parallel). Will try pyro-next next.
Comment 6 Alexander Kanavin 2018-03-20 02:31:17 UTC
pyro-next works fine as well (with systemd enabled etc.)

So I'm out of ideas at this point, and don't know what else to try. It's possible we're hitting some weird resource limit in F27 (especially if the issue occurs in random places), but without a way to reproduce there's nothing to be done. If you see this again, please do re-open.