Bug 4102 - Incremental builds do not work well when renaming recipes or changing architecture.
Summary: Incremental builds do not work well when renaming recipes or changing archite...
Status: VERIFIED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: configuration (show other bugs)
Version: 1.4
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 2.1 M1
Assignee: Richard Purdie
QA Contact: Costin
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2013-03-25 16:00 UTC by Martin Jansa
Modified: 2016-02-16 12:00 UTC (History)
9 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments
Partial fix for some of the issues (1.53 KB, patch)
2014-03-20 11:49 UTC, Richard Purdie
no flags Details | Diff
errors while building a image-sato (7.95 KB, application/octet-stream)
2015-11-06 15:00 UTC, Costin
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Jansa 2013-03-25 16:00:19 UTC
1) renaming:

http://lists.linuxtogo.org/pipermail/openembedded-core/2013-March/037466.html

http://git.openembedded.org/openembedded-core/commit/?id=c8bbb9983bcc7cfc5332e89c3e8148505b4ca83f
renames mesa-dri to mesa, but sstate code does not have any idea, that it should remove mesa-dri bits from sysroot before building mesa and later when staging mesa it will warn about conflicting files from mesa-dri:

WARNING: The recipe mesa is trying to install files into a shared area
when those files already exist. Those files and their manifest location
are:
   /OE/shr-core/tmp-eglibc/sysroots/qemux86-64/usr/lib/libGLESv2.so
   Matched in manifest-qemux86-64-mesa-dri
.. and lot more ..

from what I remember this means, that old mesa-dri files will be kept in                                                                                      
sysroot until I do build from scratch (or sstate-cache). That can be                                                                                          
pretty bad when mesa is upgraded to some new version required by other                                                                                        
components and my incremental sysroot will still keep old mesa-dri                                                                                            
instead.                                                                                                                                                      
                                                                                                                                                              
Yes, I can cleansstate mesa-dri before updating oe-core to include this                                                                                       
commit or revert this commit to cleansstate and then remove revert..                                                                                          
but that's not something average OE user will understand and do. 

Maybe we should introduce something like:
REPLACES = "${TUNE_PKGARCH}:mesa-dri"
and let sstate search for staged mesa-dri with the same architecture as current mesa (as well as older mesa version).

2) changing architecture:

netbase changed architecture from MACHINE_ARCH to TUNE_PKGARCH and builds were failing:
http://www.mail-archive.com/openembedded-core@lists.openembedded.org/msg34318.html

This is even more important then 1), because thanks to controversial opkg patch to prefer architecture over version in do_rootfs it will continue to use older MACHINE_ARCH package as long as it stays available in deploy/ipk.

There is also RFC to change netbase architecture from TUNE_PKGARCH to allarch, the same issues:
http://lists.linuxtogo.org/pipermail/openembedded-core/2013-March/037300.html

Maybe we should introduce something like:
REPLACES = "${MACHINE_ARCH}:netbase"
for first transition and then add also
REPLACES += "${TUNE_PKGARCH}:netbase"
Comment 1 Martin Jansa 2013-04-03 10:16:55 UTC
Two patches to show better error message before it fails were sent to oe-core ML as RFC (no links, because linuxtogo pipermail is down)

Subject: [RFC][PATCH 1/2] packagedata: show error when there are multiple pkgdata directories
Date: Wed,  3 Apr 2013 12:14:33 +0200

Subject: [RFC][PATCH 2/2] packagedata: Show error when trying to change PE/PV/PR from runtime/pkgdata
Date: Wed,  3 Apr 2013 12:14:34 +0200
Comment 2 Martin Jansa 2013-04-04 15:01:38 UTC
In some cases we would need RREPLACES equivalent for sstate, with different recipes.

systemd is trying to stage a lot of files formerly owned by udev

WARNING: The recipe is trying to install files into a shared area when those files already exist. Those files are:

prevents that, but in this case it would be better other way around, let systemd overwrite files from udev even when udev was first to stage them.

systemd.bb:
REPLACES += "${TUNE_PKGARCH}:udev"
Comment 3 Paul Eggleton 2013-04-04 22:37:36 UTC
I hesitate to suggest this but perhaps rather than having to explicitly state a replacement relationship, what we need is simply a mechanism that removes sstate-managed files and the associated stamps that are no longer provided by the original recipe. That would take care of recipe renames, removals and the removal of layers as well.
Comment 4 Richard Purdie 2013-04-04 23:13:57 UTC
(In reply to comment #3)
> I hesitate to suggest this but perhaps rather than having to explicitly
> state a replacement relationship, what we need is simply a mechanism that
> removes sstate-managed files and the associated stamps that are no longer
> provided by the original recipe. That would take care of recipe renames,
> removals and the removal of layers as well.

Is it practical to do that in practise though?
Comment 5 Paul Eggleton 2013-04-04 23:19:30 UTC
Without further investigation, no idea. We certainly wouldn't want to implement it if it was going to have to do a lengthy check before any build could start.
Comment 6 Martin Jansa 2013-05-13 21:33:40 UTC
pipermail is working again, so here are also list to archive:

Subject: [RFC][PATCH 1/2] packagedata: show error when there are multiple pkgdata directories
Date: Wed,  3 Apr 2013 12:14:33 +0200

http://lists.linuxtogo.org/pipermail/openembedded-core/2013-April/037795.html

Subject: [RFC][PATCH 2/2] packagedata: Show error when trying to change PE/PV/PR from runtime/pkgdata
Date: Wed,  3 Apr 2013 12:14:34 +0200

http://lists.linuxtogo.org/pipermail/openembedded-core/2013-April/037796.html

I must admit that 1/2 is too strict and shows couple of errors which cannot be fixed without changing how packagedata are searched. 2/2 works fine and shows error only in cases where do_package_write_ipk would fail with confusing "No such file or directory" so it at least shows what packagedata need to be removed.
Comment 7 Richard Purdie 2013-06-21 13:29:49 UTC
(In reply to comment #4)
> (In reply to comment #3)
> > I hesitate to suggest this but perhaps rather than having to explicitly
> > state a replacement relationship, what we need is simply a mechanism that
> > removes sstate-managed files and the associated stamps that are no longer
> > provided by the original recipe. That would take care of recipe renames,
> > removals and the removal of layers as well.
> 
> Is it practical to do that in practise though?

Specifically, some recipes can in theory by machine specific in some cases, package_arch in others and in theory all arch in other cases too. How we tell whether a given sstate target should be removed in a multimachine build, I don't know.
Comment 8 Richard Purdie 2013-06-24 15:18:29 UTC
I'm going around in circles on this issue, its very hard to come up with something generic which would work correctly in all cases. I'd like to merge Martin's second patch so that if there are problems, we detect them earlier/better.

For the underlying issue, I'm going to propose we use the OELAYOUT_ABI number in sanity.bbclass to handle things. If we do something incompatible, we bump the number and this will then call the sstate clean code with the appropriate values. If layers need to do this, we add migration code to the core and then bump the layer minor version of core so they can depend on it.

Its not a perfect solution and needs a bit of work but we don't make these kind of changes often so it should be ok?
Comment 9 Richard Purdie 2013-06-28 09:26:02 UTC
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d3ce916c7c2490154f0e749a4466f4df6307acde was merged which is a modified version of one of Martin's patches. This guards against bad things happening to builds.

In the future if we have architecture issues we'll have to deal with them as per my previous comment.
Comment 10 Martin Jansa 2013-08-09 16:32:24 UTC
To manually cleanup old version from TMPDIR (when you have already upgraded to version with different architecture) you can also use:

find <TMPDIR>/pkgdata/ <TMPDIR>/stamps -name recipe-name\* -exec rm -rf {} \;
Comment 11 Alexandru Palalau 2013-10-16 06:47:48 UTC
Verified on master branch.
Commit ID: 2b5bac02196b8bd30d223d508f5f7d4300889fd7
Comment 12 Martin Jansa 2014-01-18 14:09:03 UTC
Reopening as discussed on ELCE, the confusing error in do_package was replaced with better one, but we still don't support package architecture changes in incremental build (without hard-to-explain manual steps done by developer)
Comment 13 Tom Zanussi 2014-03-19 22:53:49 UTC
Here's another apparent manifestation of this bug, details below, but after chatting with RP and Darren, this appears to be happening only because we're switching between a two diffferently-base-named kernel providers, linux-yocto and linux-yocto-dev.  Once we rename linux-yocto-dev to linux-yocto_3.14, the problem won't exist between these two kernels.  There will still be a problem when switching between either of those and the new linux-yocto-dev, and for that the suggestion is to make the manifest warning below a 
failure instead of a warning.

Synopsis: kernel populates sysroot without removing previous contents

poky/master: 1f16ca9fba325f5ddbb1edad5bbad78d990de5e0

The way it showed up was with a perf build failure for qemuppc, but it has nothing specifically to do with perf or qemuppc.

Reproducer: select the qemuppc machine (or any other) and build perf (this uses the 3.10 kernel by default):

[trz@empanada build]$ bitbake perf

Note this file in the sysroot for the 3.10 kernel:

[trz@empanada linux]$ ls -al /home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/perf/util/include/linux/compiler.h 
-rw-rw-r--. 4 trz trz 391 Mar 19 16:10 /home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/perf/util/include/linux/compiler.h

After building the default linux-yocto kernel, switch to linux-yocto-dev:

In meta/conf/machine/include/qemu.inc:

PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto-dev"

See the warning and error below.  

Note that compiler.h has been moved to tools/include/linux/compiler.h for 3.14:

[trz@empanada linux]$ ls -al /home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/include/linux/compiler.h 
-rw-rw-r--. 4 trz trz 694 Mar 19 17:33 /home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/include/linux/compiler.h

But also note that the old compiler.h is still there in tools/perf/util/include/linux/compiler.h:

[trz@empanada linux]$ ls -al /home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/perf/util/include/linux/compiler.h
-rw-rw-r--. 4 trz trz 391 Mar 19 16:10 /home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/perf/util/include/linux/compiler.h

And note the include search order:

-I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/perf/util/include -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/perf/arch/powerpc/include -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/include/

So the 3.14 is picking up the compiler.h left over from 3.10, which doesn't have the definition of __weak and thus we get the build error.



WARNING: The recipe linux-yocto-dev is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
   /home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/pkgdata/runtime/kernel-module-crc-ccitt
   Matched in manifest-qemuppc-linux-yocto.packagedata
   /home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/pkgdata/runtime/kernel-module-cast-common
   Matched in manifest-qemuppc-linux-yocto.packagedata
   /home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/pkgdata/runtime/kernel-module-nf-nat-sip
   Matched in manifest-qemuppc-linux-yocto.packagedata
   /home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/pkgdata/runtime/kernel-module-nls-cp863
   Matched in manifest-qemuppc-linux-yocto.packagedata
   /home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/pkgdata/runtime/kernel-module-ebt-arpreply.packaged
   Matched in manifest-qemuppc-linux-yocto.packagedata
   /home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/pkgdata/runtime/kernel-module-iptable-mangle.packaged
   Matched in manifest-qemuppc-linux-yocto.packagedata
   /home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/pkgdata/runtime/kernel-module-xt-tcpudp.packaged
   Matched in manifest-qemuppc-linux-yocto.packagedata
   /home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/pkgdata/runtime/kernel-module-nls-koi8-u.packaged
   Matched in manifest-qemuppc-linux-yocto.packagedata
   /home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/pkgdata/runtime/kernel-module-iptable-raw.packaged
.
.
.
| powerpc-poky-linux-gcc -o /home/trz/yocto/dev-kernel-test/build/tmp/work/qemuppc-poky-linux/perf/1.0-r8/perf-1.0/util/path.o -c  -Wbad-function-cast -Wdeclaration-after-statement -Wformat-security -Wformat-y2k -Winit-self -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wno-system-headers -Wold-style-definition -Wpacked -Wredundant-decls -Wshadow -Wstrict-aliasing=3 -Wstrict-prototypes -Wswitch-default -Wswitch-enum -Wundef -Wwrite-strings -Wformat -O6 -fno-omit-frame-pointer -ggdb3 -funwind-tables -Wall -Wextra -std=gnu99 -fstack-protector-all -D_FORTIFY_SOURCE=2 -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/perf/util/include -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/perf/arch/powerpc/include -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/include/ -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/arch/powerpc/include/uapi -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/arch/powerpc/include -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/include/uapi -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/include -I/home/trz/yocto/dev-kernel-test/build/tmp/work/qemuppc-poky-linux/perf/1.0-r8/perf-1.0//util -I/home/trz/yocto/dev-kernel-test/build/tmp/work/qemuppc-poky-linux/perf/1.0-r8/perf-1.0/ -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/perf/util -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/perf -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/lib/ -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -DHAVE_LIBELF_SUPPORT -DHAVE_LIBELF_MMAP_SUPPORT -DHAVE_ELF_GETPHDRNUM_SUPPORT -idirafter =/usr/include/slang -DHAVE_SLANG_SUPPORT -DHAVE_TIMERFD_SUPPORT -DHAVE_LIBBFD_SUPPORT -DHAVE_ON_EXIT_SUPPORT -DHAVE_BACKTRACE_SUPPORT -I/home/trz/yocto/dev-kernel-test/build/tmp/work/qemuppc-poky-linux/perf/1.0-r8/perf-1.0/ util/path.c
| powerpc-poky-linux-gcc -o /home/trz/yocto/dev-kernel-test/build/tmp/work/qemuppc-poky-linux/perf/1.0-r8/perf-1.0/util/rbtree.o -c  -Wbad-function-cast -Wdeclaration-after-statement -Wformat-security -Wformat-y2k -Winit-self -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wno-system-headers -Wold-style-definition -Wpacked -Wredundant-decls -Wshadow -Wstrict-aliasing=3 -Wstrict-prototypes -Wswitch-default -Wswitch-enum -Wundef -Wwrite-strings -Wformat -O6 -fno-omit-frame-pointer -ggdb3 -funwind-tables -Wall -Wextra -std=gnu99 -fstack-protector-all -D_FORTIFY_SOURCE=2 -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/perf/util/include -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/perf/arch/powerpc/include -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/include/ -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/arch/powerpc/include/uapi -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/arch/powerpc/include -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/include/uapi -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/include -I/home/trz/yocto/dev-kernel-test/build/tmp/work/qemuppc-poky-linux/perf/1.0-r8/perf-1.0//util -I/home/trz/yocto/dev-kernel-test/build/tmp/work/qemuppc-poky-linux/perf/1.0-r8/perf-1.0/ -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/perf/util -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/perf -I/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/lib/ -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -DHAVE_LIBELF_SUPPORT -DHAVE_LIBELF_MMAP_SUPPORT -DHAVE_ELF_GETPHDRNUM_SUPPORT -idirafter =/usr/include/slang -DHAVE_SLANG_SUPPORT -DHAVE_TIMERFD_SUPPORT -DHAVE_LIBBFD_SUPPORT -DHAVE_ON_EXIT_SUPPORT -DHAVE_BACKTRACE_SUPPORT -I/home/trz/yocto/dev-kernel-test/build/tmp/work/qemuppc-poky-linux/perf/1.0-r8/perf-1.0/ -Wno-unused-parameter -DETC_PERFCONFIG='"etc/perfconfig"' ../../lib/rbtree.c
| util/path.c:29:15: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'strlcpy'
|  size_t __weak strlcpy(char *dest, const char *src, size_t size)
|                ^
| make[1]: *** [/home/trz/yocto/dev-kernel-test/build/tmp/work/qemuppc-poky-linux/perf/1.0-r8/perf-1.0/util/path.o] Error 1
| make[1]: *** Waiting for unfinished jobs....
| make[1]: *** wait: No child processes.  Stop.
| make: *** [all] Error 2
| make: Leaving directory `/home/trz/yocto/dev-kernel-test/build/tmp/sysroots/qemuppc/usr/src/kernel/tools/perf'
| ERROR: oe_runmake failed
| WARNING: /home/trz/yocto/dev-kernel-test/build/tmp/work/qemuppc-poky-linux/perf/1.0-r8/temp/run.do_compile.19478:1 exit 1 from
|   exit 1
| ERROR: Function failed: do_compile (log file is located at /home/trz/yocto/dev-kernel-test/build/tmp/work/qemuppc-poky-linux/perf/1.0-r8/temp/log.do_compile.19478)
ERROR: Task 6 (/home/trz/yocto/dev-kernel-test/meta/recipes-kernel/perf/perf.bb, do_compile) failed with exit code '1'
NOTE: Tasks Summary: Attempted 737 tasks of which 711 didn't need to be rerun and 1 failed.
No currently running tasks (737 of 743)

Summary: 1 task failed:
  /home/trz/yocto/dev-kernel-test/meta/recipes-kernel/perf/perf.bb, do_compile
Summary: There were 2 WARNING messages shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
Comment 14 Richard Purdie 2014-03-20 11:49:58 UTC
Created attachment 1844 [details]
Partial fix for some of the issues

This issue is complex and multi-dimensional. If we try and fix all issues in one go, we'll likely fail to deal with all the cases.

With that in mind, this is an attempt to fix some of the issues. The attached patch saves out PROVIDES information into the sysroot. The existing QA sanity checks will currently warn about overlapping files.

It would be trivial to turn those warnings into fatal errors either unconditionally or in the case the files are in /sysroot-providers/.

It would allow a new trick where if you rename mesa-dri to mesa, you could put PROVIDES += "mesa-dri" into the recipe and it would then trigger this mechanism.

It does not deal with the package architecture issues. For those, its very hard to know when a package should exist and when it should not. I'm beginning to suspect that the only way to handle that would be to look at version information and error if older versions of things were present in the feeds. That way, any architecture change would need a PR bump but that would trigger the removal code. That is something for a different patch though. It may be worth splitting these issues into separate bugs.
Comment 15 Martin Jansa 2014-03-20 12:16:47 UTC
Can we use similar PROVIDES syntax for architecture changes?

E.g. when changing foo.bb from allarch to TUNE_PKGARCH I would add

PROVIDES = "all:${PN}"

in the recipe or when changing from TUNE_PKGARCH to allarch I would add

PROVIDES = "${TUNE_PKGARCH}:${PN}"

And it would show warnings/errors when files from "old" architecture exist?
Comment 16 Richard Purdie 2014-03-27 09:37:00 UTC
Its worth thinking about this issue as two separate things:

a) How do we clean out "old" data from the machine specific sysroot

b) How do we clean out "old" packages.

For the first part, I think we should only ever have one specific provider installed in the sysroot at any one time. Firstly, we should start to error if we have more than one provider of something, then we can make it an error and/or automatically uninstall the conflicts. I'd note we have problems with gcc-cross since this is built multiple times and stored in the native sysroot which gives rise to "conflicts" currently.

For the second, I think the key is versioning. Perhaps we should only ever have one set of versions in the deploy/xxxx directories. If the version changes (including a PR change), older versions are removed.

This would mean that an architecture change would become a manual PR bump, something we can easily handle.

How does that sound?
Comment 17 Martin Jansa 2014-03-27 11:46:13 UTC
Sounds good to me.

a) is what made me to write this ticket because it was "silently" breaking some builds

b) having just one set of packages will "fix" controversial use of --prefer-arch-to-version in meta/classes/package_ipk.bbclass:OPKG_ARGS
Comment 18 Richard Purdie 2014-09-24 13:49:05 UTC
Some of this has been done, some work remains. We're past feature freeze now so targeting as 1.8
Comment 19 Richard Purdie 2015-06-15 07:06:14 UTC
This has been addressed with http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=ccb2d1b481c0fba69c06a0c0f2553e2a45ee937a and surrounding commits.
Comment 20 Richard Purdie 2015-06-15 07:06:26 UTC
Resolved a per last comment
Comment 21 Costin 2015-11-04 15:23:02 UTC
tested on latest commit:fc45deac89ef63ca1c44e763c38ced7dfd72cbe1 on an Ubuntu 15.04 machine

The steps performed are those found on comment#13:
1) MACHINE=qemuppc bitbake perf # no problem here, builds well
2)perform the following change in meta/conf/machine/include/qemu.inc:
PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto" 
--->
PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto-dev"
3) repeat step 1 and the error occurs:
"
...
ERROR: The recipe linux-yocto-dev is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
   /home/autobuilder/costin-poky/build/tmp/sysroots/qemuppc/sysroot-providers/virtual_kernel
 Matched in manifest-qemuppc-linux-yocto.populate_sysroot"

reopening ...
Comment 22 Richard Purdie 2015-11-06 09:50:05 UTC
(In reply to comment #21)
> tested on latest commit:fc45deac89ef63ca1c44e763c38ced7dfd72cbe1 on an
> Ubuntu 15.04 machine
> 
> The steps performed are those found on comment#13:
> 1) MACHINE=qemuppc bitbake perf # no problem here, builds well
> 2)perform the following change in meta/conf/machine/include/qemu.inc:
> PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto" 
> --->
> PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto-dev"
> 3) repeat step 1 and the error occurs:
> "
> ...
> ERROR: The recipe linux-yocto-dev is trying to install files into a shared
> area when those files already exist. Those files and their manifest location
> are:
>   
> /home/autobuilder/costin-poky/build/tmp/sysroots/qemuppc/sysroot-providers/
> virtual_kernel
>  Matched in manifest-qemuppc-linux-yocto.populate_sysroot"
> 
> reopening ...

I did more investigation. If a recipe "disappears" from bitbake's view, the code works correctly and things are uninstalled from the sysroot. The trouble is both linux-yocto and linux-yocto-dev can in theory be built at the same time, even if that would be a bad thing to do.

If you modify the test slightly and delete the linux-yocto recipe when building linux-yocto-dev, it will work correctly.

I believe the situation was improved by the patches but have a further idea about changing the way virtual/ providers work that should be able to improve the way this code works in the future which is 2.1 material.
Comment 23 Costin 2015-11-06 13:26:36 UTC
Just tested with the patch from here:
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=e97352fa020762201a10a493d213bfab3f1291d3
Works well now. Marking as WaitingForUpstream
Comment 24 Paul Eggleton 2015-11-06 13:54:42 UTC
FYI, "WaitForUpstream" is when we're waiting on some upstream project e.g. gcc or the kernel; for a bug where a patch is just out for review on our mailing list the status to use is IN PROGRESS REVIEW.
Comment 25 Costin 2015-11-06 15:00:29 UTC
Created attachment 2843 [details]
errors while building a image-sato

errors while building a image-sato
Comment 26 Costin 2015-11-06 15:03:17 UTC
Please disregard my last attachment. I made an error on my local build machine
@Paul, thanks
Comment 27 Richard Purdie 2015-12-08 10:42:18 UTC
Fix previously discussed merged: http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=0c5d2398fca382a0cbdc73adf50e741cef2287bc
Comment 28 Costin 2016-02-16 12:00:43 UTC
Hello,

Tested on latest master: 36b43b233c841b3c09fc17e04686b4b758780963
Followed steps as in comment#21.
Works fine.

Marking as verified.