Bug 2024 - Need to update sstate documentation
Summary: Need to update sstate documentation
Status: RESOLVED FIXED
Alias: None
Product: Reference
Classification: Documentation
Component: handbook (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium normal
Target Milestone: 1.2
Assignee: Scott Rifenbark
QA Contact:
URL: http://git.yoctoproject.org/cgit.cgi/...
Whiteboard: 15-March-2012: Fixed/Resolved
Depends on:
Blocks:
 
Reported: 2012-02-22 15:09 UTC by Richard Purdie
Modified: 2012-03-15 22:43 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Richard Purdie 2012-02-22 15:09:51 UTC
After reading http://www.yoctoproject.org/docs/latest/poky-ref-manual/poky-ref-manual.html#shared-state-cache I'm reminded there have been code changes we need to account for in the manual on the sstate signature generation.

I'm opening this bug to remind me to provide the information to Scott about this (and so Scott can remind me if I forget).
Comment 1 Richard Purdie 2012-02-22 19:39:28 UTC
Scott, I think we need to update section 3.2.2. Checksums (Signatures) to end with:

Thus far, this section has limited discussion to the direct inputs into a task. Information based on direct inputs is referred to as the "basehash" in the code. However, there is still the question of a task's indirect inputs, the things that were already built and present in the build directory. The checksum (or signature) for a particular task needs to add the hashes of all the tasks on which the particular task depends. Choosing which dependencies to add is a policy decision. However, the effect is to generate a master checksum that combines the basehash and the hashes of the task's dependencies.

At the code level, there are a variety of ways both the basehash and the dependent task hashes can be influenced. Within the BitBake configuration file, we can give BitBake some extra information to help it construct the basehash. The following statements effectively result in a list of global variable dependency excludes - variables never included in any checksum:

     BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH"
     BB_HASHBASE_WHITELIST += "DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS"
     BB_HASHBASE_WHITELIST += "FILE_DIRNAME HOME LOGNAME SHELL TERM USER"
     BB_HASHBASE_WHITELIST += "FILESPATH USERNAME STAGING_DIR_HOST STAGING_DIR_TARGET"
           
This example is actually where WORKDIR is excluded since WORKDIR is constructed as a path within TMPDIR, which is on the whitelist. 

The rules for deciding which hashes of dependent tasks to include though dependency chains are more complex and this is generally done with a python function. The code in meta/lib/oe/sstatesig.py shows two examples of this and also illustrates how the user can insert their own policy into the system if they desire. This file defines the two basic signature generators OE-Core uses, "OEBasic" and "OEBasicHash". By default, there is a dummy "noop" signature handler enabled in BitBake. This means that behaviour is unchanged from previous versions. OECore uses the "OEBasic" signature handler by default through this setting in the bitbake.conf file:

     BB_SIGNATURE_HANDLER ?= "OEBasic"

The "OEBasicHash" BB_SIGNATURE_HANDLER is the same as the OEBasic version but adds the task hash to the stamp files. This results in any metadata change that changes the task hash, automatically causing the task to be run again. This removes the need to bump PR values and changes to metadata automatically ripple across the build. Currently, this behavior is not the default behavior for OE-Core but is the default in Poky.
            
Its also work noting that the end result of these signature generators is to make some dependency and hash information available to the build. This includes:

     BB_BASEHASH_task-<taskname> - the base hashes for each task in the recipe
     BB_BASEHASH_<filename:taskname> - the base hashes for each dependent task
     BBHASHDEPS_<filename:taskname> - The task dependencies for each task
     BB_TASKHASH - the hash of the currently running task
Comment 2 Scott Rifenbark 2012-03-15 22:43:41 UTC
Replaced the ending part of the section per Richard Purdie's suggestions.  I changed a few little areas as needed but not many.  

You can examine the change at http://www.yoctoproject.org/docs/latest/poky-ref-manual/poky-ref-manual.html#checksums

Scott