Bug 7632

Summary: ROOTFS_POSTPROCESS_COMMAND no longer accepts raw commands, just functions
Product: [Documentation] Reference Reporter: Laszlo Papp <lpapp>
Component: handbookAssignee: Scott Rifenbark <srifenbark>
Status: VERIFIED FIXED QA Contact: Bogdan Alexandru Voiculescu <bogdanx.a.voiculescu>
Severity: normal    
Priority: Medium+ CC: alexandru.c.georgescu, bluelightning, bogdanx.a.voiculescu, poky.doc.watcher, poky.watcher, Qi.Chen, srifenbark
Version: unspecified   
Target Milestone: 1.9 M2   
Hardware: x86   
OS: Multiple   
Whiteboard: 11 May 2015: RESOLVED
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Done (doc changes complete)

Description Laszlo Papp 2015-04-20 13:24:54 UTC
I am now in the process of updating our software stack from a dylan based solution to daisy. I have just happened to come through this issue during the process.

It seems that ROOTFS_POSTPROCESS_COMMAND no longer accepts raw commands as it used to. As a workaround, we need to wrap our commands into a function. One could argue that it is also better style, but still.

WARNING: Function mkdir /home/lpapp/Projects/Yocto/poky-dylan-9.0.1/build/tmp/work/foo-linux-gnueabi/foo-core-image/1.0-r0/rootfs/opt doesn't exist
WARNING: Function chown foo:foo /home/lpapp/Projects/Yocto/poky-dylan-9.0.1/build/tmp/work/foo-linux-gnueabi/foo-core-image/1.0-r0/rootfs/opt doesn't exist
WARNING: Function [ -d /home/lpapp/Projects/Yocto/poky-dylan-9.0.1/build/tmp/work/foo-linux-gnueabi/foo-core-image/1.0-r0/rootfs/tmp ] && rmdir /home/lpapp/Projects/Yocto/poky-dylan-9.0.1/build/tmp/work/foo-linux-gnueabi/foo-core-image/1.0-r0/rootfs/tmp doesn't exist
WARNING: Function ln -s /var/tmp /home/lpapp/Projects/Yocto/poky-dylan-9.0.1/build/tmp/work/foo-linux-gnueabi/foo-core-image/1.0-r0/rootfs/tmp doesn't exist
WARNING: Function [ -d /home/lpapp/Projects/Yocto/poky-dylan-9.0.1/build/tmp/work/foo-linux-gnueabi/foo-core-image/1.0-r0/rootfs/run ] && rmdir /home/lpapp/Projects/Yocto/poky-dylan-9.0.1/build/tmp/work/foo-gnueabi/polatis-core-image/1.0-r0/rootfs/run doesn't exist
Comment 1 Paul Eggleton 2015-04-21 08:51:14 UTC
As I said on IRC I'm leaning towards this being a regression that we should fix rather than something we would note in the manual.
Comment 2 Scott Rifenbark 2015-05-04 21:19:09 UTC
Someone else should own this bug if we are going to treat it as a regression that should be fixed.  

Scott
Comment 3 Paul Eggleton 2015-05-05 09:08:23 UTC
I've discussed this with Richard and he would prefer to leave the behaviour as-is, as with the change we now have the ability to call python functions, where that was not possible previously. So it turns out we really do need to change the documentation.

What this means is:

1) Change the variable entry for the following variables so that we note that it's expected to be a semicolon-separated list of *functions* to call and not arbitrary shell commands:

ROOTFS_PREPROCESS_COMMAND
ROOTFS_POSTPROCESS_COMMAND
SDK_POSTPROCESS_COMMAND
POPULATE_SDK_POST_TARGET_COMMAND
POPULATE_SDK_POST_HOST_COMMAND
IMAGE_POSTPROCESS_COMMAND
IMAGE_PREPROCESS_COMMAND
ROOTFS_POSTUNINSTALL_COMMAND
ROOTFS_POSTINSTALL_COMMAND

It appears that some of these are currently undocumented so we'll need to add them.

2) Somewhat late, but in the daisy migration notes we should mention this behaviour change so that anyone using them to port their metadata over will know what to do. This should mention that arbitrary shell commands simply need to be wrapped in a shell function when migrating.
Comment 5 Paul Eggleton 2015-05-05 21:39:42 UTC
* Can we replace "Added by classes to run post processing commands" with "Specifies a list of functions to call"?

* The entry for ROOTFS_PREPROCESS_COMMAND uses ROOTFS_POSTPROCESS_COMMAND in its example (copy-paste?)

* In some of the examples you have "functions" instead of "function".

* "which points to the root filesystem image" -> "which points to the directory that will become the root filesystem image"

* For ROOTFS_POSTUNINSTALL_COMMAND, can we extend "has removed unnecessary packages" with "(when runtime package management is disabled in the image, several packages are removed including base-passwd, shadow and update-alternatives)".

* For IMAGE_PREPROCESS_COMMAND and IMAGE_POSTPROCESS_COMMAND, we need to be a bit clearer about when these run. I'd suggest "created the image" -> "created the final image output files" or similar.

* For the migration entry: as I said above we need to mention for migration purposes that arbitrary shell commands simply need to be wrapped in a shell function when migrating, e.g.:

my_postprocess_function() {
   echo "hello" > ${IMAGE_ROOTFS}/hello.txt
}
ROOTFS_POSTPROCESS_COMMAND += "my_postprocess_function; "

* Also on the migration entry, can we rename it from "Variable Entry Behavior" to something more specific such as "Preprocess / Postprocess Command Variable Behaviour" ?
Comment 6 Scott Rifenbark 2015-05-05 23:04:35 UTC
Hi, 

I have implemented Paul's changes.  I will leave in IN PROGRESS REVIEW for a while longer.

Thanks, 
Scott
Comment 7 Paul Eggleton 2015-05-07 13:29:52 UTC
Looks good, I think we're probably done - thanks. Could you also please update the 1.8 manuals to include these changes?
Comment 8 Scott Rifenbark 2015-05-11 13:33:44 UTC
Updated for fido branch last week.  I have marked as RESOLVED and set the doc flag to "done." 

Thanks 
Scott
Comment 9 Bogdan Alexandru Voiculescu 2015-09-01 13:44:26 UTC
verified