Bug 9823

Here's a second version with some clarifications. One thing I'm still curious about is whether e.g. do_deploy[dirs] is still effective when the task is accelerated.


The line

  SSTATETASKS += "do_deploy"

adds some required sstate-related processing (implemented in sstate.bbclass)
to before and after do_deploy.

The line

  do_deploy[sstate-inputsdirs] = "${DEPLOYDIR}"

says that the do_deploy task places its output in ${DEPLOYDIR} when run
normally (i.e., when not using the sstate cache). This output becomes the input
to the sstate cache.

The line

  do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"

says that the contents of the sstate cache should be copied to ${DEPLOY_DIR_IMAGE}.

If do_deploy is not already in the sstate cache, or if its input checksum
(signature) has changed from when the output was cached, it will be run to
populate the sstate cache, after which the contents of the sstate cache is
copied to ${DEPLOY_DIR_IMAGE}. If do_deploy is in the sstate cache and its
signature indicates that the cached output is still valid (i.e., if no relevant
task inputs have changed), then the contents of the sstate cache will be copied
directly to ${DEPLOY_DIR_IMAGE} by do_deploy_setscene instead, skipping
do_deploy.

The task definition

  python do_deploy_setscene () {
      sstate_setscene(d)
  }
  addtask do_deploy_setscene

is glue logic needed to make the above settings effective. sstate_setscene(d)
takes the flags above as input and accelerates do_deploy via the sstate
cache if possible. If the task was accelerated, sstate_setscene(d) returns True.
Otherwise, it returns False, and the normal do_deploy task runs. See the
(setscene section)[https://www.yoctoproject.org/docs/2.1/bitbake-user-manual/bitbake-user-manual.html#setscene]
of the BitBake User Manual for more information.

The line

  do_deploy[dirs] = "${DEPLOYDIR} ${B}"

creates ${DEPLOYDIR} and ${B} before do_deploy runs. See the Variable Flags
section
[https://www.yoctoproject.org/docs/2.1/bitbake-user-manual/bitbake-user-manual.html#variable-flags]
of the BitBake User Manual.

In cases where sstate-inputdirs and sstate-outputdirs would be the same, you
can use sstate-plaindirs. For example, to preserve the ${PKGD} and ${PKGDEST}
output from do_package, write

  do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"

Hi, 

Please look the new section over here http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#shared-state.  I was not sure what to do with some of the old existing explanation.  So, I commented out what I thought was being replaced by the new stuff.  Just let me know of any changes here. 

Thanks, 
Scott

Looks good. There might be a comma missing after "i.e." in "i.e. if no relevant task inputs have changed", but I'm not sure if this is hard rule.

Only information I can see that got lost is the following:


This method, as well as the following example, also works for multiple directories.

     do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
     do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"
     do_package[sstate-lockfile] = "${PACKAGELOCK}"

These methods also include the ability to take a lockfile when manipulating shared state directory structures since some cases are sensitive to file additions or removals. 
            

Maybe two points could be added to replace it:


 * sstate-inputdirs and sstate-outputdirs can also be used with multiple directories. For example, the following declares PKGDESTWORK and SHLIBWORK as sstate input directories (from which the sstate cache is populated), and PKGDATA_DIR and SHLIBSDIR as the corresponding sstate output directories:

     do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
     do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"

 * These methods also include the ability to take a lockfile when manipulating shared state directory structures, for cases where file additions or removals are sensitive:

     do_package[sstate-lockfile] = "${PACKAGELOCK}"

Would be nice if someone more familiar with the sstate implementation looked over it too, just to be safe.

One more nit:

"The do_deploy[sstate-inputsdirs] = "${DEPLOYDIR}" line causes the do_deploy to place its output in ${DEPLOYDIR} when run normally."

would be better as

"The do_deploy[sstate-inputsdirs] = "${DEPLOYDIR}" declares that do_deploy places its output in ${DEPLOYDIR} when run normally."

That line by itself does not cause do_deploy to place its output in ${DEPLOYDIR}, but tells the sstate mechanism where the input is found.

Ok, 

I added those two points.  See http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#shared-state for the final.  Also, on the comma after the "i.e." stuff... there is debate on that usage.  I have chosen to not use that comma throughout the YP set.  If some exist... I will will hunt them down and exterminate them :)

I also added Richard Purdie to the CC list should he like to look at this shared state stuff.  Or, suggest someone else. 

Thanks, 
Scott

We collided on the last point you made... Let me get that addressed....

Ok - see http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#shared-state.  All up-to-date at this point. 

Thanks, 
Scott

Fine by me now. Thanks!

Or... one last nit:


Setting SSTATETASKS to "do_deploy" adds some required sstate-related processing...


would be more accurate as


Adding "do_deploy" to SSTATETASKS adds some required sstate-related processing...

Got it... http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#shared-state

I will leave the bug in the IN PROGRESS REVIEW state for a while to see if we get any more review comments..

Thanks for your help in this!

Scott

Found one small typo:

  do_deploy[sstate-inputsdirs]

should be

  do_deploy[sstate-inputdirs]

Dang... sorry. 

http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#shared-state

Thanks, 
Scott

Mostly fixed with yocto-docs 518352f88c8dda16f2915a7bb9901ffd7686d739 and typo in 49e7cb239f4f4f70dc5738686f1bca135772bcbe.

Ross, 

Setting doc flag to "done."

Scott