10478 2016-10-21 14:19:25 +0000 run-postinsts doesn't update package manager database on target device 2016-12-14 21:23:26 +0000 1 1 7 Build System, Metadata & Runtime OE-Core devtools / tool chain 2.1.2 Other arm RESOLVED FIXED Backport to YP 2.2.1 and 2.1.2 Medium+ normal 2.3 M1 1 niko.mauno jku jose.perez.carranza meta.mr.watcher meta.watcher ross.burton juan.fernandox.ramos.frayle --- --- Don't know oldest_to_newest 67452 0 niko.mauno 2016-10-21 14:19:25 +0000 The /usr/sbin/run-postinsts script, installed by meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb recipe, does not seem to update target device package manager database when it executes postinst scriptlets from under /etc/*-postinst/ during first boot to pristine root filesystem. In our use case the rootfs contains an opkg package manager database (under /var/lib/opkg) as well as several "delayed postinst" scriptlets under /etc/ipk-postinsts/ (apparently created by actions in meta/lib/oe/rootfs.py). If an user then runs 'opkg update' and 'opkg install somepackage' on the target device, the pending postinst actions (from opkg database's point of view) are executed again, in addition to 'somepackage''s postinst actions. Ie. this triggers superfluous execution of "delayed postinst" actions (which can involve eg. downing of network interfaces, ssh server, and so on, via update-rc.d calls included in the pending postinst actions). Following modifications seemed to fix the issue in our opkg use case: --- a/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts +++ b/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts @@ -13,6 +13,9 @@ backend_list="rpm deb ipk" pm_installed=false +dpkg_statusfile="#LOCALSTATEDIR#/lib/dpkg/status" +opkg_statusfile="#LOCALSTATEDIR#/lib/opkg/status" + for pm in $backend_list; do pi_dir="#SYSCONFDIR#/$pm-postinsts" @@ -20,14 +23,14 @@ for pm in $backend_list; do case $pm in "deb") - if [ -s "#LOCALSTATEDIR#/lib/dpkg/status" ]; then + if [ -s "$dpkg_statusfile" ]; then pm_installed=true break fi ;; "ipk") - if [ -s "/var/lib/opkg/status" ]; then + if [ -s "$opkg_statusfile" ]; then pm_installed=true break fi @@ -59,7 +62,13 @@ exec_postinst_scriptlets() { echo "Running postinst $i..." [ "$POSTINST_LOGGING" = "1" ] && eval echo "Running postinst $i..." $append_log if [ -x $i ]; then - eval sh -c $i $append_log + if [ "$pm" = "ipk" -a -s "$opkg_statusfile" ]; then + eval opkg configure $(basename $i | cut -d- -f2-) $append_log + elif [ "$pm" = "deb" -a -s "$dpkg_statusfile" ]; then + eval dpkg --configure $(basename $i | cut -d- -f2-) $append_log + else + eval sh -c $i $append_log + fi rm $i else echo "ERROR: postinst $i failed." 67775 1 jku 2016-10-31 10:06:23 +0000 (In reply to comment #0) Hi Niko, I'll do some tests myself but if you can clear the confusion I have that would be great: * The hard coded path looks like a real bug (but I understood this wasn't your problem as you mentioned /var/lib/opkg path) * The exec_postinst_scriplets changes I do not understand: run-postinsts seems to already contain similar functionality even in krogoth (2.1) branch: | if $pm_installed; then |  case $pm in |  "ipk") |  eval opkg configure $append_log |  ;; |  |  "deb") |  eval dpkg --configure -a $append_log |  ;; |  esac | else |  exec_postinst_scriptlets | fi So exec_postinst_scriplets should only ever get called if $pm_installed is false. It's entirely possible that there's a problem with e.g. how we call opkg configure, but I don't see how the patch could help as the code path modified in the patch can never end up executed when $pm_installed = "true" as far as I can tell... 67783 2 niko.mauno 2016-10-31 15:28:39 +0000 Hi Jussi, Thanks for asking. I hope You will find the following additional details helpful: - In our case this problem manifests on a rootfs which was generated by an image recipe which inherits image.bbclass. - Before 1st boot occurs, an /etc/ipk-postinsts directory with several '00*-<packagename>' scriptlets exists on the generated rootfs. - When 1st boot occurs, during S runlevel /etc/init.d/run-postinsts script is executed, which effectively calls /usr/sbin/run-postinsts <http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts?h=krogoth> - Line 14 sets pm_installed as 'false' - Line 17 sets pi_dir as '/etc/ipk-postinsts' (when for-loop has iterated to the 3rd element of backend_list) - Line 19 tests for existence of pi_dir, succeeds and breaks the for-loop (this occurs before pm_installed would have been set as 'true' on line 31 in the same for-loop) - Line 73 if-statement translates as 'false', which leads to the exec_postinst_scriptlets call. Also in the patch I provided, I tried to carefully maintain the order of scriptlet execution, as commit message of b2c9e7347acdfd0efe1c3dabb853d609233b61b6 mentions about adding postinst script execution order mindfulness. I checked that in our case calling just 'opkg configure' did execute the postinst actions in different order than exec_postinst_scriptlets(), so I decided to rather call 'opkg configure <packagename>' one at a time in the existing exec_postinst_scripts() loop to retain the execution order dictated by Yocto rootfs facility. Furthermore, I tried to keep the effective changes relatively contained, as the 'run-postinsts' script looks like it might use a more fundamental rewrite. 67785 3 jku 2016-10-31 16:34:18 +0000 Thanks, I had indeed missed the early break in the first for-loop. I'll so some tests and talk to Anibal about the postinst execution order and whether we can trust opkg and dpkg to do the right thing on their own ... 67815 4 3516 jku 2016-11-01 12:08:32 +0000 Created attachment 3516 run-postinsts: Use opkg/dpkg to configure when possible I think the early break is buggy. The already existing code for package manager configures seems fine to me, it's just not being used. I'm attaching a patch to modify the behaviour in the first loop -- normally I'd just send it to mailing list but this is somewhat difficult to test (I wish we had some tests for this). I believe calling opkg or dpkg just once is enough: the package managers do take care of proper dependency chains and _should_ run the configures in a correct order. Niko, if you have a possibility of testing this, please do. I'll run some more tests myself and if no problems appear will send the patch to mailing list. 67816 5 niko.mauno 2016-11-01 13:30:13 +0000 (In reply to comment #4) > Created attachment 3516 [details] > run-postinsts: Use opkg/dpkg to configure when possible > > I think the early break is buggy. The already existing code for package > manager configures seems fine to me, it's just not being used. I'm attaching > a patch to modify the behaviour in the first loop -- normally I'd just send > it to mailing list but this is somewhat difficult to test (I wish we had > some tests for this). > > I believe calling opkg or dpkg just once is enough: the package managers do > take care of proper dependency chains and _should_ run the configures in a > correct order. Being unaware of pkg_postinst_${PN} permutations outside our companys' and Yocto upstreams' meta layers, I thought sticking to the Yocto rootfs facility dictated order would presumably err to the safe side. In my mind as well, the order arbitrated by the actual package manager should be favored. My earlier tests showed already that forcing just single 'opkg configure' call instead of exec_postinst_scriptlets() fixed the problem in our case. > Niko, if you have a possibility of testing this, please do. I'll run some > more tests myself and if no problems appear will send the patch to mailing > list. I tried the patch you provided and it seems to do the trick in our case, ie. running 'opkg configure' after 1st boot to pristine rootfs no longer triggers any pending configure actions. (Please consider including the replacement of '/var' with '#LOCALSTATEDIR#' on line 30 of the original run-postinsts file). 68907 6 ross.burton 2016-12-07 16:56:30 +0000 Merged in oe-core b645919f173512f9e75aeb26348d60b63dcdc53c. 3516 2016-11-01 12:08:32 +0000 2016-11-01 12:08:32 +0000 run-postinsts: Use opkg/dpkg to configure when possible 0001-run-postinsts-Use-opkg-dpkg-to-configure-when-possib.patch text/plain 1558 jku RnJvbSAzNzJlNDE3NDJlYjY1MDNlNGIwMzJiZDIxYjViZTk4ZTM4ZmUyYjkwIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBKdXNzaSBLdWtrb25lbiA8anVzc2kua3Vra29uZW5AaW50ZWwu Y29tPgpEYXRlOiBUdWUsIDEgTm92IDIwMTYgMTE6MzY6MTcgKzAyMDAKU3ViamVjdDogW1BBVENI XSBydW4tcG9zdGluc3RzOiBVc2Ugb3BrZy9kcGtnIHRvIGNvbmZpZ3VyZSB3aGVuIHBvc3NpYmxl CgpDdXJyZW50bHkgcnVuLXBvc3RpbnN0cyBzY3JpcHQgaGFzIGNvZGUgdG8gcnVuIHBvc3RpbnN0 IHNjcmlwdHMKdmlhIG9wa2cvZHBrZyBjb25maWd1cmUgYnV0IHRoYXQgY29kZSBpcyBuZXZlciB1 c2VkLiBUaGUgYWR2YW50YWdlCm9mIHVzaW5nIHBhY2thZ2UgbWFuYWdlcnMgaW5zdGVhZCBvZiBq dXN0IGV4ZWN1dGluZyB0aGUgc2NyaXB0cyBpcwp0byBrZWVwIHRoZSBwYWNrYWdlIG1hbmFnZXIg REIgdXBkYXRlZC4KCkZpeCB0aGUgc2NyaXB0IHNvIHRoYXQgdGhlIHBhY2thZ2UgbWFuYWdlcnMg YXJlIHVzZWQgd2hlbiBhcHByb3ByaWF0ZS4KClNpZ25lZC1vZmYtYnk6IEp1c3NpIEt1a2tvbmVu IDxqdXNzaS5rdWtrb25lbkBpbnRlbC5jb20+Ci0tLQogbWV0YS9yZWNpcGVzLWRldnRvb2xzL3J1 bi1wb3N0aW5zdHMvcnVuLXBvc3RpbnN0cy9ydW4tcG9zdGluc3RzIHwgOCArKysrKy0tLQogMSBm aWxlIGNoYW5nZWQsIDUgaW5zZXJ0aW9ucygrKSwgMyBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQg YS9tZXRhL3JlY2lwZXMtZGV2dG9vbHMvcnVuLXBvc3RpbnN0cy9ydW4tcG9zdGluc3RzL3J1bi1w b3N0aW5zdHMgYi9tZXRhL3JlY2lwZXMtZGV2dG9vbHMvcnVuLXBvc3RpbnN0cy9ydW4tcG9zdGlu c3RzL3J1bi1wb3N0aW5zdHMKaW5kZXggMDRiYTM5NC4uNjBiZWFkZiAxMDA3NTUKLS0tIGEvbWV0 YS9yZWNpcGVzLWRldnRvb2xzL3J1bi1wb3N0aW5zdHMvcnVuLXBvc3RpbnN0cy9ydW4tcG9zdGlu c3RzCisrKyBiL21ldGEvcmVjaXBlcy1kZXZ0b29scy9ydW4tcG9zdGluc3RzL3J1bi1wb3N0aW5z dHMvcnVuLXBvc3RpbnN0cwpAQCAtMTYsMjMgKzE2LDI1IEBAIHBtX2luc3RhbGxlZD1mYWxzZQog Zm9yIHBtIGluICRiYWNrZW5kX2xpc3Q7IGRvCiAJcGlfZGlyPSIjU1lTQ09ORkRJUiMvJHBtLXBv c3RpbnN0cyIKIAotCVsgLWQgJHBpX2RpciBdICYmIGJyZWFrCisJaWYgWyAhIC1kICRwaV9kaXIg XTsgdGhlbgorCQljb250aW51ZQorCWZpCiAKKwkjIGZvdW5kIHRoZSBwYWNrYWdlIG1hbmFnZXIs IGl0IGhhcyBwb3N0aW5zdHMKIAljYXNlICRwbSBpbgogCQkiZGViIikKIAkJCWlmIFsgLXMgIiNM T0NBTFNUQVRFRElSIy9saWIvZHBrZy9zdGF0dXMiIF07IHRoZW4KIAkJCQlwbV9pbnN0YWxsZWQ9 dHJ1ZQotCQkJCWJyZWFrCiAJCQlmaQogCQkJOzsKIAogCQkiaXBrIikKIAkJCWlmIFsgLXMgIi92 YXIvbGliL29wa2cvc3RhdHVzIiBdOyB0aGVuCiAJCQkJcG1faW5zdGFsbGVkPXRydWUKLQkJCQli cmVhawogCQkJZmkKIAkJCTs7CiAJZXNhYworCWJyZWFrCiBkb25lCiAKIHJlbW92ZV9yY3NkX2xp bmsgKCkgewotLSAKMi4xLjQKCg==