Bug 10141

Hello,

BitBake includes a mechanism for viewing the input signature (variable
dependencies, including automatically added ones) of a task, but it's hard to
discover and eluded me for the longest time (hence ugly workarounds like
https://lists.yoctoproject.org/pipermail/yocto/2016-April/029756.html).

Here's some suggested fleshing out/rearranging of the documentation to make
things easier to discover and understand:

 * Rename '2.3. Debugging Build Failures' to '2.3. Debugging Tools and Techniques'
   (Unrelated, but I think it makes sense as this section is about more general
   debugging than just dealing with build failures.)


 * Rename '2.3.3. Viewing Dependencies' to '2.3.3. Viewing Dependencies Between Recipes and Tasks'


 * Add a new section '2.3.4. Viewing Task Variable Dependencies' with the following contents:

    As mentioned in the
    [Checksums (Signatures)](https://www.yoctoproject.org/docs/2.2/bitbake-user-manual/bitbake-user-manual.html#checksums)
    section of the BitBake User Manual, BitBake tries to automatically determine
    what variables a task depends on so that it can rerun the task if the value of
    any of the variables changes. This is usually reliable, but if you do things
    like construct variable names at runtime, then you might have to manually
    declare dependencies on those variables (using the
    [[vardeps]](https://www.yoctoproject.org/docs/2.2/bitbake-user-manual/bitbake-user-manual.html#variable-flags)
    variable flag).
    
    If you are unsure whether a variable dependency is being picked up
    automatically for a given task, you can list the variable dependencies
    BitBake has determined as follows:
    
      1. Build the recipe containing the task with
      
           $ bitbake recipename
    
      2. Inside the [STAMPS_DIR] directory, find the sigdata file corresponding to
         the task. sigdata files contain a pickled Python database of all the
         metadata that went into creating the input checksum for the task. As an
         example, for the [do_fetch] task of the db recipe, the sigdata file might
         be found in the following location:
         
           ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1
    
         For tasks that are accelerated through the shared state
         ([sstate](http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#shared-state-cache))
         cache, an additional siginfo file is written into [SSTATE_DIR] along with
         the cached task output. siginfo files contain exactly the same information
         as sigdata files.
    
      3. Run 'bitbake-dumpsigs' on the sigdata or siginfo file:
    
           $ bitbake-dumpsigs ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1
    
    In the output of the above command, you will find a line like the following,
    which lists all the (inferred) variable dependencies for the task. This list
    also includes indirect dependencies from variables depending on other
    variables, recursively.
    
      Task dependencies: ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]', 'base_do_fetch']
    
    Note that functions (e.g. base_do_fetch) also count as variable dependencies.
    They in turn depend on the variables they reference.
    
    The output of bitbake-dumpsigs also includes the value each variable had, a
    list of dependencies for each variable, and
    [variable whitelisting](https://www.yoctoproject.org/docs/2.2/bitbake-user-manual/bitbake-user-manual.html#var-BB_HASHBASE_WHITELIST)
    information.
    
    There is also a 'bitbake-diffsigs' command for comparing two siginfo/sigdata
    files. This can be helpful when trying to figure out what changed between two
    versions of a task. If 'bitbake-diffsigs' is called with just one file, it
    behaves like 'bitbake-dumpsigs'.
    
    You can also use BitBake to dump out the signature construction information
    without executing tasks by using either of the following BitBake command-line
    options:
    
        ‐‐dump-signatures=SIGNATURE_HANDLER
        -S SIGNATURE_HANDLER
    
    Note {
    Two common values for SIGNATURE_HANDLER are "none" and "printdiff" to only dump
    the signature or to compare the dumped signature with the cached one,
    respectively.
    }
    
    Using BitBake with either of these options causes BitBake to dump out .sigdata
    files in the stamp directory for every task it would have executed instead of
    building the specified target package. 


 * Replace the contents of '4.3.4.1. Debugging' with the following:

    Seeing what metadata went into creating the input signature of an sstate task
    can be a useful debugging aid. This information is available in siginfo files
    in [SSTATE_DIR]. See [2.3.4. Viewing Task Variable Dependencies] for how to
    view and interpret the information stored in these files.


'4.3.4.1. Debugging' was the only section to have sigdata/siginfo information
before. Most of the information applies to all tasks and not just to sstate
tasks though, so it makes sense to move it.

The --dump-signatures stuff is copied verbatim from the old version. I actually
couldn't get --dump-signatures to work as (I think) intended. It might be
buggy.

Cheers,
Ulf