Bug 3390

Summary: Different PACKAGE_ARCH in recipe only for some package or MACHINE causes missing ipk files in deploy
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Martin Jansa <Martin.Jansa>
Component: configurationAssignee: Richard Purdie <richard.purdie>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: sgw
Version: 1.4   
Target Milestone: 1.4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: ---

Description Martin Jansa 2012-11-06 06:12:22 UTC
I tried to report some of those issues in
https://bugzilla.yoctoproject.org/show_bug.cgi?id=2801
but that one is already RESOLVED FIXED.

1) Example of different PACKAGE_ARCH for package is in opkg:
PACKAGE_ARCH_update-alternatives-cworth = "all"
Building opkg for qemux86-64 and then qemux86 results in warning:
NOTE: Running task 729 of 731 (ID: 11, /OE/oe-core/openembedded-core/meta/recipes-devtools/opkg/opkg_svn.bb, do_package_write_ipk)
NOTE: recipe opkg-1_0.1.8+svnr633-r10.8: task do_package_write_ipk: Started
WARNING: The recipe opkg is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
   /OE/oe-core/tmp-eglibc/deploy/ipk/all/update-alternatives-cworth_0.1.8+svnr633-r10.8_all.ipk
   Matched in
Please verify which package should provide the above files.
NOTE: recipe opkg-1_0.1.8+svnr633-r10.8: task do_package_write_ipk: Succeeded

2) Explicit set of PACKAGE_ARCH to MACHINE_ARCH in recipes:
To reproduce add:
PACKAGE_ARCH_qemux86 = "${MACHINE_ARCH}"
in openembedded-core/meta/recipes-core/udev/udev-extraconf_1.0.bb

$ MACHINE=qemux86-64 bitbake udev-extraconf
...
$ find tmp-eglibc/work -name udev-extraconf\*ipk
tmp-eglibc/work/all-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/all/udev-extraconf-dbg_1.0-r7_all.ipk
tmp-eglibc/work/all-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/all/udev-extraconf-dev_1.0-r7_all.ipk
tmp-eglibc/work/all-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/all/udev-extraconf_1.0-r7_all.ipk
$ find tmp-eglibc/deploy/ipk/ -name udev-extraconf\*
tmp-eglibc/deploy/ipk/all/udev-extraconf-dbg_1.0-r7_all.ipk
tmp-eglibc/deploy/ipk/all/udev-extraconf-dev_1.0-r7_all.ipk
tmp-eglibc/deploy/ipk/all/udev-extraconf_1.0-r7_all.ipk

$ MACHINE=qemux86 bitbake udev-extraconf
$ find tmp-eglibc/work -name udev-extraconf\*ipk
tmp-eglibc/work/all-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/all/udev-extraconf-dbg_1.0-r7_all.ipk
tmp-eglibc/work/all-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/all/udev-extraconf-dev_1.0-r7_all.ipk
tmp-eglibc/work/all-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/all/udev-extraconf_1.0-r7_all.ipk
tmp-eglibc/work/qemux86-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/qemux86/udev-extraconf-dev_1.0-r7_qemux86.ipk
tmp-eglibc/work/qemux86-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/qemux86/udev-extraconf_1.0-r7_qemux86.ipk
tmp-eglibc/work/qemux86-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/qemux86/udev-extraconf-dbg_1.0-r7_qemux86.ipk

But .ipk files with all PACKAGE_ARCH from previous qemux86-64 build are removed:
$ find tmp-eglibc/deploy/ipk/ -name udev-extraconf\*
tmp-eglibc/deploy/ipk/qemux86/udev-extraconf-dev_1.0-r7_qemux86.ipk
tmp-eglibc/deploy/ipk/qemux86/udev-extraconf_1.0-r7_qemux86.ipk
tmp-eglibc/deploy/ipk/qemux86/udev-extraconf-dbg_1.0-r7_qemux86.ipk

3) Implicit set of PACKAGE_ARCH to MACHINE_ARCH
To reproduce, cleansstate udev-extraconf for both MACHINES used in 2), remove changes from 2), copy one file used in SRC_URI to MACHINE subdirectory
mkdir meta/recipes-core/udev/udev-extraconf/qemux86/
cp meta/recipes-core/udev/udev-extraconf/mount.blacklist meta/recipes-core/udev/udev-extraconf/qemux86/

$ MACHINE=qemux86-64 bitbake udev-extraconf
..
$ find tmp-eglibc/work -name udev-extraconf\*ipk
tmp-eglibc/work/all-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/all/udev-extraconf-dbg_1.0-r7_all.ipk
tmp-eglibc/work/all-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/all/udev-extraconf-dev_1.0-r7_all.ipk
tmp-eglibc/work/all-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/all/udev-extraconf_1.0-r7_all.ipk

$ find tmp-eglibc/deploy/ipk/ -name udev-extraconf\*
tmp-eglibc/deploy/ipk/all/udev-extraconf-dbg_1.0-r7_all.ipk
tmp-eglibc/deploy/ipk/all/udev-extraconf-dev_1.0-r7_all.ipk
tmp-eglibc/deploy/ipk/all/udev-extraconf_1.0-r7_all.ipk

$ MACHINE=qemux86 bitbake udev-extraconf
..
$ find tmp-eglibc/work -name udev-extraconf\*ipk
tmp-eglibc/work/all-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/all/udev-extraconf-dbg_1.0-r7_all.ipk
tmp-eglibc/work/all-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/all/udev-extraconf-dev_1.0-r7_all.ipk
tmp-eglibc/work/all-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/all/udev-extraconf_1.0-r7_all.ipk
tmp-eglibc/work/qemux86-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/qemux86/udev-extraconf-dev_1.0-r7_qemux86.ipk
tmp-eglibc/work/qemux86-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/qemux86/udev-extraconf_1.0-r7_qemux86.ipk
tmp-eglibc/work/qemux86-oe-linux/udev-extraconf-1.0-r7/deploy-ipks/qemux86/udev-extraconf-dbg_1.0-r7_qemux86.ipk

But again old ipk from previous build are gone:
$ find tmp-eglibc/deploy/ipk/ -name udev-extraconf\*
tmp-eglibc/deploy/ipk/qemux86/udev-extraconf-dev_1.0-r7_qemux86.ipk
tmp-eglibc/deploy/ipk/qemux86/udev-extraconf_1.0-r7_qemux86.ipk
tmp-eglibc/deploy/ipk/qemux86/udev-extraconf-dbg_1.0-r7_qemux86.ipk
Comment 1 Richard Purdie 2012-11-08 11:41:44 UTC
For 1), I'm not sure we can support this type of syntax. When you think about it, repeatedly overwriting some "all" arch package is a bad thing and can lead to races like you describe. The problem has always been there, we're just specifically identifying it now.

For 2), the key problem is that udev-extraconf is an allarch recipe. If you do this on a normal recipe, things will work but you can't "uninherit" allarch which means peices still apply.

In both cases, I therefore thing we should not support these particular specialist usecases and fine alternatives.
Comment 2 Martin Jansa 2012-11-09 10:09:31 UTC
(In reply to comment #1)
> For 1), I'm not sure we can support this type of syntax. When you think
> about it, repeatedly overwriting some "all" arch package is a bad thing and
> can lead to races like you describe. The problem has always been there,
> we're just specifically identifying it now.

So should I remove just
PACKAGE_ARCH_update-alternatives-cworth = "all"
or do we want separate update-alternatives-cworth recipes which can be allarch, like we had before?
 
> For 2), the key problem is that udev-extraconf is an allarch recipe. If you
> do this on a normal recipe, things will work but you can't "uninherit"
> allarch which means peices still apply.

So can I remove allarch from udev-extraconf recipe to resolve this? Or should I add another recipe which is MACHINE_ARCH by default (e.g. to provide machine specific mount.blacklist)?

> In both cases, I therefore thing we should not support these particular
> specialist usecases and fine alternatives.
Comment 4 Richard Purdie 2012-11-26 10:05:04 UTC
For issue 1), http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=eaa7c3a37111e254217263a26306018aebd7953f was merged which fixes the issue.

For issue 2), http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=668f7e36fe2c681a1b5c7604fe34960956448eec was merged as an alternative fix which also addresses other issues.