Bug 9414

Patchset has been sent for review: http://lists.openembedded.org/pipermail/openembedded-core/2016-April/thread.html

The recipe is failing when MACHINE is changed:
$ bitbake bmap-tools-native
WARNING: Host distribution "openSUSE-13.2" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Parsing recipes: 100% |########################################################################################################| Time: 00:00:11
Parsing of 882 .bb files complete (0 cached, 882 parsed). 1314 targets, 44 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION        = "1.30.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal"
TARGET_SYS        = "x86_64-poky-linux"
MACHINE           = "genericx86-64"
DISTRO            = "poky"
DISTRO_VERSION    = "2.1"
TUNE_FEATURES     = "m64 core2"
TARGET_FPU        = ""
meta
meta-poky
meta-yocto-bsp
meta-selftest     = "ed/oe-core/bmap-tools-9414:7416c3ba19f26fe6eea485fefad4db4881f829ad"

NOTE: Preparing RunQueue
NOTE: Executing SetScene Tasks
ERROR: bmap-tools-native-3.2-r0 do_deploy_setscene: The recipe bmap-tools-native is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
   /home/ed/git/yocto/poky/build/tmp/deploy/tools/bmaptool-3.2
 Matched in manifest-qemux86-64-bmap-tools-native.deploy
 /home/ed/git/yocto/poky/build/tmp/deploy/tools/bmaptool
 Matched in manifest-qemux86-64-bmap-tools-native.deploy
Please verify which recipe should provide the above files.

I'll remove preparing standalone binary and installing it into tmp/deploy/tools from the recipe. the tool still can be run as any native oe tool, so removing standalone version doesn't influence anything.

sent v2 for review: http://lists.openembedded.org/pipermail/openembedded-core/2016-April/120687.html

better to add "inherit allarch" than removing standalone version of tool.
Otherwise, user end-up in situation where bmap created, but there is no easily usable tool to use generated bmap for the image.

For reference, the ostro recipe (with do_deploy) is here:

https://raw.githubusercontent.com/ostroproject/meta-ostro/master/meta-ostro/recipes-devtools/bmap-tools/bmap-tools_3.2.bb

I figured out the issue with the do_deploy error above - since the bmap-tools recipe is doing this in a non-machine-specific manner it should set the following:

do_deploy[stamp-extra-info] = ""

This overrides the default value of "${MACHINE}" set in deploy.bbclass which causes the task to re-execute when MACHINE changes (since what's in stamp-extra-info gets added to the name of the stamp file, so if it changes it doesn't see that it's stamped as already having been completed).

I should mention also, inherit allarch would not be appropriate for this recipe - when building for the target it *is* arch-dependent.

For clarification, the reason it's arch-dependent (even though it's written in an arch-independent scripting language) is that the signature depends on python which will rebuild for the target architecture and thus influence the signature of tasks within this recipe; this is something we don't currently optimise for.

(In reply to comment #4)
> better to add "inherit allarch" than removing standalone version of tool.
> Otherwise, user end-up in situation where bmap created, but there is no
> easily usable tool to use generated bmap for the image.

I can run the tool this way: PATH=tmp/sysroots/x86_64-linux/usr/bin/ bmaptool. It works just fine.

I wouldn't call this method of tool invocation as "convenient" :)
Better to fix recipe, especially, because one line fix available from Paul.

Sent v3 for review: http://lists.openembedded.org/pipermail/openembedded-core/2016-May/120834.html

there is no much sense to separate bmap-tools recipe into two commits, but end result is ok.

There are at least 3 concerns about standalone script mentioned on the mailing list:
http://lists.openembedded.org/pipermail/openembedded-core/2016-May/120921.html
http://lists.openembedded.org/pipermail/openembedded-core/2016-May/120922.html
http://lists.openembedded.org/pipermail/openembedded-core/2016-May/121472.html

I'd propose to fix standalone script generation and send it for review separately.

The core functionality has been accepted to poky master. I think it's enough to resolve this issue.

For issue mentioned in http://lists.openembedded.org/pipermail/openembedded-core/2016-May/121472.html I had chance to talk directly with Richard. Common understanding that it should be changed to
"addtask deploy before do_build after do_install"

As for other comments from Chris, we need to revisit how tool is used in wic code.
My understanding so far was that it uses python library from sysroot, not the standalone tool from deploy dir. Is it so Ed ?

wic is calling native bmaptool to generate .bmap file:
   exec_native_cmd("bmaptool create %s -o %s.bmap" % (full_path, full_path),
                    self.native_sysroot)

However, I'm not sure Chris is talking about wic in his comment:

You're right that this is very handy, but we hit a problem when building
from shared state. This uses do_deploy to deploy bmaptool into
DEPLOY_DIR_IMAGE, but IMAGE_DEPENDS only depends on do_populate_sysroot,
not do_deploy, so an image build from sstate won't write this script out.

I'm also concerned about Richard saying in his comment:
I'm not sure if we want to do this in the first place.

I think it's better to discuss this with Richard and Chris before sending the patch again.

That's very interesting. Ed, from your code it seems that you're calling native version of the tool and it is executed from native sysroot. thus, it should be correct usage, and it shouldn't be dependant on any content in deploy directory...
Thus, dependency on do_populate_sysroot is expected one...

Are you able to reproduce same error as Chris having with building image from sstate ?

No, I didn't try to reproduce it.

Chris, can you show us how to reproduce the issue with standalone bmaptool script you're talking about here http://lists.openembedded.org/pipermail/openembedded-core/2016-May/120921.html ?